I don't think he will get mad, if he really loves and cares about you he will understand and try to work it out with you. it is probably hurting him and he's blaming himself for you not having sex, when really it is something from your past. don't be afraid of him getting mad