Just because a portion of a plant is poisonous does not mean that all of it is. Rhubard is a good example the stock is delicious yet the leaves hold enough oxalic acid to be toxic. Some other examples are Rhododendrons and jassamine. Both are considered to have poisonous honey yet the bees harvest it and mix it with the rest of the nectar. The proportion of toxic to safe honey is so low that the toxic affects are no longer a factor.
In my area Rhododendrons and their close relative Madrona are abundant and I've never had a problem with toxicity in honey.
See ABC and XYZ of Bee Culture pages 262 & 263.