Even if you speak the language, knowing exactly what to say can sometimes be tricky. Different situations have different cultural expectations, and what is expected plays a large part in the Japanese culture. This can be a little confusing at first. As always, foreigners have a level of leeway. But being able to respond like a Japanese person, or a very polite foreigner, will improve the reactions of those you meet.
Until you pick up on what is expected, default to being very polite. Japan’s culture is traditionally one of politeness and humility. In turn it can be beneficial to be polite and humble. So, when in doubt of how to respond to someone, say “domo.” This literally means “I humbly accept,” but it’s applicable to almost any situation. It’s also almost impossible to offend anyone with it, making it a safe response to anything from a greeting to being given directions to a thumbs up.