Everyone has standards. Personally, I wouldn't consider an LTR with someone I considered my intellectual inferior. Call me a snob, but its a dealbreaker for me. However, I also know that there are many guys who would find my bookishness worse than boring.
So I agree w/Blue. You do come across like you're doing him some kind of favour dating him. Get over yourself. Its just a question of compatibility.
Second thoughts can generally be amended with judicious action; injudicious actions can seldom be recovered with second thoughts.
--Cyteen by C.J.Cherryh