You gave him your telephone number after two conversations. That's a dumb thing to do in my opinion. He probably thinks you're a goer and since you didn't give him a chance to find out about you through email, he's doing it through text.
If you think he's a weirdo then tell him right now that you don't see each other as having anything in common. Next time, keep your digits to yourself until you know who you're giving them too... otherwise, you'll eventually be giving them to Dexter's protege.
As for the 2 hours outside your town. Why don't you limit the people that can actually contact you to those that are within a reasonable time limit outside your town?
Simple.
“The willingness to accept responsibility for one’s own life is the source from which self-respect springs.” ~Joan Didion