Dealing with "Body Weight" Issues

Okay, please don't hate me for asking...
How do you respond to/reason with/deal with people who find out that you are fasting, cleansing, or in general eating extremely health foods, when you are a slim and reasonably healthy person?

I am petite and slender, and people frequently ask me "why" I am not eating the same really unhealthy junky stuff they are. They are like, "Are you on some sort of diet?" and no matter how I try to explain about being healthy, they don't believe that people really need any cleansing, and then they say something like "You don't need to lose weight" or "you're small, you can afford to eat stuff like this."

I understand that people are often just curious. But still, if that happens when I'm just being a vegetarian, how can you respond to people if they somehow find out that I am temporarily going under dramatically reduced food intake? I don't plan on telling people but sometimes a few people end up knowing. I don't want people to think I am anorexic. Any suggestions and experience on dealing with situations like this? They would all be appreciated, thanks. Tweet