To be blunt, it seems that the only thing making him happy in the relationship is sex. My opinion is that it doesn't sound like a good relationship. Having sex scheduled every day was a mistake. What makes a relationship great is having spontaneous sex. Making things predictable makes it boring. Also, if you schedule things and it doesn't happen, things like this happen.
I will do my best to reply with an educated, humble and honest answer. Ultimately, it is up to you whether you want to listen to my advice or ignore it completely. Sometimes, my advice may be wrong; occasionally, it will be right. Regardless, I want to do my best to give people answers they are seeking.