I hate underhanded subtle creeps like this guy. They leave you wondering. He's probably hoping that she'll relive the moment he stroked her cheek and want more. I also don't believe he wasn't aware you were away at first.
I agree with letting him know that it's not on. Make sure you tell him that your wife doesn't want him touching her as opposed to telling him you don't want him touching her. Make sure he knows how uncomfortable he made her. Knowing that you know and knowing that his advances are unwanted (made evident by your wife telling you about them) should make him back off.
'People are never perfect but love can be. People waste time looking for the perfect lover rather than creating the perfect love' - Princess Leigh-Cheri from Still Life With Woodpecker.