Am I wrong for not wanting my man around his kids?

My boyfriend said he's willing to neglect his kids for me because he knows it bothers me that he's around those kids more than me. Am I wrong?
Am I wrong for not wanting my man around his kids?
Post Opinion