Mine started out that way. The syrup can spilled out and killed most of them. They built up, but slowly starting out.
Assuming the queen was released properly, do you have brood? I would guess this is the clincher question about the viability of your hive.
If you have brood, then start feeding them syrup. Try to relieve feeding stress and get them to make some comb for more brood.
If not, your likely in trouble only having a single hive.