well i believe if you guys really love each other then it will work out. you need to stop accusing him and you have to apologize. gaining trust is hard and takes time but just ask him for one last chance. considering he's been with you for 4 years despite your accusations shows he does care for you.
explain to him about your past and how it haunts you. it might be better if you see a therapist regarding this. I was traumatized by my parents relationship and my dads constant cheating. still haunts me to this day. you just need some one to talk to. and if not a professional then im sure your boyfriend will listen.
good luck friend
Within you I lose myself. Without you I find myself wanting to become lost again.