Funny thing: after I had been helping for a few months, they asked me to join their board (I had to decline due to work). I asked why they wanted me, since I was just a guy volunteering to serve mashed potatoes one night a week, not important or particularly influential in any way, and prone to insulting prominent people. The founder told me plainly that because I'm a middle-aged white dude, I could go places and say things that the other board members couldn't.
The other board members are pastors and retired businessmen and known in the community, but random white guy me has access that they don't have. Seriously. As obnoxious, cranky and undiplomatic as I am, I'm more respected than people who have worked and served the city for decades, simply because my ancestors were from Europe.
That's white privilege. When somebody uses the term, they aren't calling you a racist, or playing the race card, or making excuses for their own condition. They are stating a fact. A white jerk who never graduated college and can't behave politely in public is more respectable than a successful retired black businessman.