most likely they're looking to build a nest, it'll involve making a few holes in your hive somewhere
you really dont want to kil them because they're valuable pollinators as well, they are destructive
so discouraging them is a must, feel around the underside of boards for dents and holes, fill them
with wood putty or caulk and provide a place a little more inviting like a few old logs with
some sort of roof over it to keep the wood dry. they seem to like pine because it's soft, a couple
of 2x4 cut into 1 foot long sections is ideal.
ag.arizona.edu/pubs/insects/ahb/woodhome.jpeg