I too was raised to remove my hat indoors,, unless as noted,, in a large public area. I wear a hat daily. Yet, when I come home or go into a restaurant etc,, off it comes.
I taught a lot of my BSA "boys" over the years to do the same. Years ago, I had a small group of 3-4 boys that were with me in between our classes that day. We stopped in a Shoneys for late breakfast/early lunch. All hats came off. A little while later, an elderly gentleman came by, stopped & said; "You have a fine bunch of boys here. I don't hats come off much anymore." I replied they weren't really "my" boys,, but BSA boys. He then addressed them with; "Pay attention to this man. He'll teach you right!" The boys got a good lesson that day.
But just this past Saturday,, I about had a slight panic attack. While traveling home from SHOT,, Sam & I went to get a bite to eat in a food court during our layover in Orlando. Off came my hat, and in the seat beside me. It's a simple ball cap, "US Army Veteran" type. Yet it was given to me by Miss Penny, AND I've placed my original (1970's era) Crewchief wings, and my Unit Crest badge on it. As we finished up & I went to put my trash away, and get a refill, Sam got up, and I got distracted enough to leave my hat behind. We went to the next gate about our flight,, and suddenly, I realized I'd left it behind. I raced back,, only to discover it wasn't there, and the guy sitting there,, (with other stuff indicating he had family with him) said he didn't see it. I went to information & all, not turned in. I went back to the table,, and as I approached,, the young man was looking around as well, and he found my hat in the floor under the table. Whew,,! It returned to my head, and on home we came.
It's RARE that I leave a hat behind,, and I really didn't want to lose that one. But I'll NOT stop my lifelong habit of hat removal.