He is saying if you have a nice rack, you better have a nice brain to go with it or you are just a waste of his time.
Yep. I'm much the same way. I can have a physical attraction for a woman, but if it becomes obvious she's not intelligent, the attraction dies... she's no longer interesting.
My wife is freaking brilliant, it's one of the things that attracted me to her right away.