Heh, a female version of my situation! Well, as far as your story goes, it seems to me (from a guy's perspective) that he may have indeed been toying with your emotions and trying to make you feel guilty. It seems like he has the hallmarks of low self-esteem, trying to figure out what makes him happy by trying lots of other girls, while all the while wanting you to care about him too. Him using your mistake as an excuse is not cool. If you made it clear that you regretted it and considered it a mistake, then if he truly loves you he should be able to accept that it meant nothing to you and it won't happen again.
As far as your current situation, if you have worries that he will hurt you again, make those very clear (which it seems you have). It's up to you to try again or not, but just make sure everything is in the open and there are no ifs ands or buts that the past is the past.
Good Luck!