It's a common propaganda tactic to associate any opposition to a cause with lunatics. It would be great if every people would try to remain rational, but this is not an achievable goal. There will be always extremes on every side of an issue, the media (including social media) will always exaggerate the extremes on the 'wrong' side and it will try to downplay/explain it away on the 'right' side. It is impossible to reduce the number of irrational people to zero, so this tactic will always work.

Only way to defend against this is to make people aware of the existence of the propaganda machine, and the ways how it works.