Most Helpful Guy
If you don't want him back, just tell them you want to move on, and its nothing personal. If they are your friends as well, then they should mind their own business, and let you two sort out between yourselves. He is a big boy, why doesn't he come to you and talk to you, instead of using yours friends to get involved.
Just tell them straight, I just want to move with my life. If you are worried about upsetting them, then I guess there is no help for you.