I have a friend like this. But just because he's frugal doesn't mean he isn't fair. If I've driven or paid recently I very confidently say "Alright let's take your car." and if he pushes back I just remind him "I paid last time."
If he gives you flak about it start making deals... "Fine I'll drive but then you have to cover *blank*"... If he's not budging tell him you'll sit this one out, or find another friend that will pay.
But really the problem usually is solved after "I paid last time."
Frugal friend and I for the most part keep mental ledgers and it works out well. That's my advice.
Ted, how do I explain this to you. Last night, I ate the best cake of my life. Do you think I'm gonna let that cake out of my life? Hell no. I'm gonna find out what bakery made that cake and I'm gonna get some more cake.