In many cases the fear is of losing their political careers which has happened. People have received death threats and knocks on their doors like the poll workers in Georgia. Folks with guns showed up at a Pizzeria in New Jersey after rumors pushed by Trump, et al. Then there's the fear of being ostracized at the country club or the workplace. Missing out on business deals. The fear is real.