Scrubs vs regular clothes

We were in Office Depot and was on regular clothes. One patient saw me and told me "I've never seen you in normal clothes. (I wear scrubs to work) you're actually pretty". Hehe. I did not know whether to feel flattered or awkward. What does that mean? He thought I was ugly? Whew! That is not easy to swallow. Hehehe