I look at it this way: There's been a number of news-worthy elections that've come down to a loss or win because of a single vote. Was that my or your vote that made or broke it? Of course not... but maybe.
So yeah, there's a pizza place I won't order from. A chicken sandwich place too. And a hobby place that I'll go to begrudgingly because I've looked everywhere else and they're the only place in 100km with something in stock. There's a home improvement store like that too. And so on.
Will they notice my loss of business? No, but I know that it's not me paying for their exec's lunch or cab fare, and I'm good with that.