Why was it (and still is in some places) customary for people in the US to take their hats off in a building?

not sure why but some people still believe that

It was a gesture of respect to take one's hat off inside---mostly for men and children when I was growing up. Women once covered their heads in church (Catholic), for similar reasons to Muslim women wearing the hijab, believe it or not.

It was called protocol ... you never wear a hat indoors... at least for guys ... women could

Ah... you've just fallen into the oldest trap ... the past is a foreign country, they do things differently there ...
:)

I'm well, thank you. How about you ? It's good to see you. :))

There's no real practical reason for it. Hats and coats are just considered outdoor wear, so you don't wear outdoor wear inside; hence coat racks and hat racks. 

It is not something that began in the U.S. It was passed on from European tradition. Men of arms were to remove their head armor as they entered. It showed a sense of ease and trust and a gesture of respect. That tradition remained strong in the military. Since uniform headwear could take many forms, such as a service cap, a saucer hat, a fatigue ballcap, or a helmet, the rule was to take of your "headgear" indoors, like in the mess hall, and always wear it outdoors and in public venues. People with exposure to service may maintain that tradition and strongly recommend it for others.

As women were generally outside the profession of arms, they were allowed to wear their hats as they wished. As they became included in the profession of arms, they were to follow the profession's hat rules. 

http://emilypost.com/advice/hats-off-hat-etiquette-for-everyone/

I suspect that because a hat is usually worn to keep one's head warm while outside, wearing it inside is a sign of being anxious to leave.  This is conjecture on my part. 

It's a sign of respect for women and men and children to remove their face and head concealing gear when speaking to you.

Mayhaps. But this does not answer the question why it is considered respectful.

Hats and coats are considered outdoor wear but then again so are pants and tops.

It is a way to show respect.   Some cultures insist you take off your shoes when you enter a domicile.  Different strokes. 

When I was young we were required to WEAR a hat in church if we were female.  I am sure that was passed down for centuries and know one knew why.