Yes, it's a huge red flag. Because they are probably not broken up, she probably wants an affair, statistically speaking. Then the other guy has to deal with an angry boyfriend, and lots of drama. If you are broken up, someone should move out. Period.
Whose name is on the lease? If both your names are on the lease, and you were only dating 5 months, it also shows she makes bad decisions, which usually leads to more drama. There's a reason for my rules "Never live together until you have dated 12 months in a row" and "Only one person's name on the lease in case it doesn't work out."
Guys don't like drama.
I have a long time interest in psychology, specializing in relationship dynamics for 30 years.
(Please note, we give the best advice we can based on the information given in a post. For better advice, please include the age of all romantic partners.)