Just look at Lore. He wiped out a colony, and could do far worse damage if he was both more competent and stable. It eventually escalated to the point where Data had to shut him down, due to the danger he posed to the rest of the Federation.
True - it might be argued that he's not responsible if he didn't know exactly what the crystalline entity would do, but it's pretty obvious that he had a good idea what would happen, so yeah, he's the key causal factor.