What has he done to try and make things right since you found him out? For starters, has he gotten a divorce? Is the contact between him and his ex appropriate?
Have you caught him in any other lies? If he was estranged from his wife, it should have been fairly easy to keep her a secret - but how on earth did he manage to keep his kids a secret for 4 years?! Did he see them only rarely? Or did he see you only rarely?
Is he a really good liar, or were you ignoring red flags? Is he a good father to the kids? Have you met his kids? Do you agree with his methods of parenting?
And for what it's worth, if you're looking for permission to kick him to the kerb, you have mine.
Never regret anything that has happened in your life. It cannot be changed, forgotten or undone. So, take it as a lesson learned and move on.