Philosophy: Consequentialism and the Trolley Dilemma?

Something that always perplexed me with the Trolley Dilemma and its variants is that I've seen it suggested repeatedly in introductory philosophy courses that the reason students vote differently between the original dilemma and its variants highlights a difference between categorical and consequentialist thinking.

Q: Can't it all be seen as consequentialist? Consider things from the angle of public safety, e. g.

If we were to condone a bystander pulling a lever to divert a trolley, it doesn't pose a public safety hazard since the ultimate hazard is people lying on the tracks.

The original trolley problem.
The original trolley problem.

If we want to correct the hazard in the future, we should impose measures to keep people off the tracks. Whether or not we condone pulling the lever is largely inconsequential as I see it to public safety (although I do think we should not allow random people to be diverting trolleys in the future).

If we take the fat man variant, then we can't condone pushing innocent fat men (or anyone for that matter) off of bridges under any circumstance. To condone that would introduce a new safety hazard (consequence) for anyone standing on bridges.

We cant condone pushing people off of bridges under any circumstance.
We can't condone pushing people off of bridges under any circumstance.

Similar case for the organ transplant variant. If we condone harvesting people's organs against their will in hospitals, that would almost certainly present a disastrous consequence in the form of a public safety crisis. No one would be able to trust going to a hospital if they can have their organs harvested against their will.

At least that's one of multiple possible consequentialist angles I see. I'm admittedly not well-versed in philosophy but doesn't this make sense as one possible, purely-consequentialist way to explain variations in answers between the variants without requiring any categorical thinking whatsoever? If not, please let me know!

Updates
3 mo
My interest in the subject comes from how I seem largely incapable of categorical thinking. I've always thought, even as a child, in terms of probabilistic consequences (although I'm not an act consequentialist; a right course of action could still bring about poor results, but could still be the right course in that it has the highest probability of producing the best results). So the idea that variations in answers suggest any degree of categorical thinking just doesn't register with my brain.
Philosophy: Consequentialism and the Trolley Dilemma?
Post Opinion