HOw do you know that one "thinks" in the language they speak? Language is how we communicate to other people. One doesn't have to know a "language" to think. Our minds use things we understand to think. What if a person spoke 12 different languages? Which one would they think in? What if a person never learned how to speak? Would They be unable to think until they learned a language? What if a deaf person never learned how to use sign language?
A person who is born blind "guesses" as to what the world looks like. They do this by using their other senses. They may never know what a blue sky looks like, but they can describe what it looks like to them. (Note: color is subjective. Not all people see the exact same color in the same way. Meaning what you see as blue may not be what another sees as blue.) Likewise a deaf person "guesses" as to what sounds sound like by using their other senses. BUt in their minds they think with what they know. If they can see, perhaps they think in pictures or colors or shapes. If they once were able to hear, they may think in sounds. If a person was both deaf and blind (from birth) they still think, but not in the same manner as a hearing and seeing person might.
It is normal to think without talking to yourself. A speed reader does not even speak what he is reading. Your average reader does, and that is why they are only average. If you just look at two words instead of one you can double your reading speed and since you can't talk that way you soon develop the ability to read much faster than you can talk. That is why so many vloggers have started publishing transcripts of their video presentations: people got bored waiting for the video to get to the point. They can read much faster than the fellow talks.
Great question, I'm gathering it must be how a baby knows it wants feed or drink but has no language to form thoughts, however Walter's post answered this for us.