IMO you need to put them back on so the bees clean them up to keep them from being too hopitible to the ants and wax moths. Then wait until the weather is cold (so the bees can protect them) and then pull them off and let the cold protect them from the wax moths.