If everyone, or at least a majority of some significance abandoned belief and just walked away- no more paying taxes, no more obeying any authority, no more consumption of anything that contributes in any way to power and authority, they lose all legitimacy.