I don't think that you're selfish and needy AT ALL!! So please, don't believe those comments.
You never once asked him to choose between his son and you because you know that's something he could never do. But that doesn't mean that you have to put his happiness before yours all the time. A relationship works both way.
He is happy the way things are but you aren't. We have only one life and it cannot be wasted by being unhappy. You have to think about yourself in this situation and it is very good that you are doing so. Talk to him, tell him exactly what you are telling us now. If you always acted as if you were happy in this situation, there is no way he can guess that you're not. Tell him. If he loves you and want the relationship to work, he will make the effort and will be very glad that you told him how you feel.
I think the babysitting idea is also a very good one. See how it goes from there. If it doesn't work after that, at least you will have been honest and tried your best.
Good luck!
"It's call Karma baby, and it goes around."