Try being Nikkei. My grandparents were from Japan, but my parents and I were born and raised in Hawaii, which has its own problems with being "part of the U.S." People from the
rest of the U.S. ask me where I'm from, and about half the time, don't even accept the fact that I'm American.
They follow up with, "No, where are you REALLY from?" and won't accept the fact that I'm not actually from Japan (been there a lot, speak passable Japanese, good accent, but no, not from Japan).
In Europe, it's assumed I'm Chinese, until I open my American mouth and speak American English.... then it's "Where did you learn English?"
In Japan, they're just puzzled as to why this apparently Japanese looking person either acts like a gaijin, or at least a clueless Japanese.
I often preface any question with "sumimasen, watashi ha nikkeijin desu ga....." and that seems to help.
So, while I generally agree that it's better to ask (I prefer being asked than for people to guess), I also wish people would believe my answer.