We do not have this custom of automatically taking our shoes off. (Susie, you were safe with me!

)
I take my shoes off if they are wet or dirty...I realize that shoes are at some level dirty all the time...just like hands etc...
I take my shoes off when it is hot outside....
I take my shoes off when I want to put my slippers on....other than that, I "offer" to take my shoes off whenever the owner of the house does not have shoes on....and I do notice
When in Rome......