I wore my Crocs all winter last year. But I live in the South, so it's not really freezing or anything. We don't get snow. It gets down to like 40F, a few days might be 30's or 20's but generally it's not that cold. I liked the Crocs b/c I could still have my feet "bare" and ventilated without people looking at me like I was crazy wearing sandals. (I am crazy and get that look enough, thank you very much!)
My house is old and doesn't heat well, I have always been cold in my house in the winter. Wood and tile floors, no carpet. However, I discovered that going barefoot actually helped me not be so cold. My mom doesn't believe me! ha It took some adjusting but now I guess my circulatory system is up to it. So definitely barefoot indoors. When I am cold, I just bundle up the REST of my body. Making the back of the neck warm is actually the most important. When it got colder, I did wear socks with my crocs when I went out a few times. At least my socks won't get so moist when they are in crocs. I decided I was actually colder in other shoes, because the socks would get moist.
My cousin lives in Montana, she told me that she saw the school kids walking to school through the snow in flip flops. I guess it's what you're used to! MH (the Barefoot Herbalist) talks about walking barefoot in snow, and in cold winter streams. It's not for everybody, but it's not unheard of, and considered good for you - food for thought.
You can experiment and figure out what works for you.