There are many factors. Examples:

1. You can go far once you abandon moral principles when dealing with the out-group. No longer an even fight.

2. You can also go far if you're continually told since birth your people are to rule and not to serve (the converse holds too). If one group is thinking only of their immediate concerns due to their upbringing, and another is thinking of taking over, the first group is blind to the takeover because they never thought to look for it.

3. You can also go far if you're continually told since birth that it is your duty to help your people take over the world. Versus your duty only towards nation or family. In the second case, no one is doing anything to ensure survival of their group. Nationalism doesn't work because it's blind to the composition of the nation. While the first group will always promote their own first at the other's expense (parasitic behavior). The second group is blind to their exploitation because they are only looking at the sum total of national prosperity.

A nation with a homogeneous group can afford to and eventually does ignore concerns about the welfare of the group itself. It assumes its wealth, power etc. belongs to itself. But if said group allows another group to settle amongst them, that assumption need not be true anymore.

In short, it has everything to do with upbringing (the morality, the aims in life, the priorities etc.) It's a people's mistake to let another people takeover them, or tolerate anyone's (person or group) parasitic behavior.