JP, Robo has the right idea. Yes, it will take time, but I am sure you have found old, abandoned comb in walls. When you take a handful of it out, it just crumbles. No body or consistency left, basically just dust still holding it's shape until you remove it. In that condition, it attracts no vermin, does no damage, and causes no harm.
Robo, I believe that, as in a dearth, the queen stops laying when the foragers stop bringing in nectar and pollen, so 1 week after the trap is set, I think she has quit laying. Then 3 weeks for them to emerge, meaning 4 weeks of trapping leaves NO brood. Please notice, I said BELIEVE. I have not torn out a hive after 4 weeks to prove it, but the signs after trapping seem to point to it, and we have all seen it during a dearth.
As for the swarm, anything is possible with bees, as we all know. You could very well have set the trap during their preparations for swarming and they just carried it out. The queens I have witnessed leave the structure took every bee in the colony with her, other than the ones in the trap box. If there were still bees in the structure, you did likely have a swarm.