I would not tolerate any type of yellowjacket, paper wasp, anything like that around my colonies. When you get masses of the nasties, they can decimate a colony pretty quickly. The bees, even if they are a strong colony, if the wasps are in enough numbers, cannot combat them. I say this from experience.
I noticed on my one colony (the only one left of 8 that died last winter), a couple of paper wasps entering the colony, the bees ignored them, I thought that rather strange and watched as now and then another entered. Scared the dickens out of me, because this was not normal behaviour. I think the bees were overcome with weariness trying to keep them out and had given up. I have good reason to say this.
I set out my wasp traps, which are comprised of beer and water. Wasps of all kinds love this, especially the paper wasps that plague the bees and those freakin' bald faced hornets, real nasties. Every day, for about a week, and I am not embellishing one little bit. Those wasp traps were full of these predators, and I mean full. Of course all drowned and dead. I must have killed thousands and thousands of them. It was a bad year for these nasties this year, the worst that I have ever seen. So, picturing those thousands of creeps that I killed during that one week, I don't doubt for a single moment that the bee colony was terrorized. I wondered why such a strong colony did not swarm, nor did I have to split it. I know why.
Wasps are lovers of protein, they also love sweet stuff. These paper wasps and bald faced hornets must have been having a grand old picnic in my single colony. Thanking my lucky stars that it was as strong as it was and the loss of brood did not severely and so adversely affect it. Actually, maybe the infestation was good -- it was a type of swarm control ;) :) :). The only way to put a positive spin on it.
Those bad dudes are all gone and dead now. This colony is still strong as the dickens of the dickens, and is going into winter with a massive cluster. This colony recovered nicely from that attack of the wasps.
So, don't think that you see only one or two wasps entering a colony, that there may not be hundreds more that are within, doing their nasty thing. You must protect your colonies, I don't care what anyone says, about "the bees protecting their own colony", I don't believe it for a minute, and I can be pretty head strong about certain things. Have that most beautiful and great day, to love and live with greatest of health. Cindi