That means there are a lot of folks either gloating or complaining whose opinions don't count for shit.
I could give you a long list of reasons you should vote — people died so you could, it's your civic duty, etc. But the main reason you should vote is that, if you don't, then NO ONE CARES what you think about politics.
I don't mean that in some blow-off-y way. I mean, literally, if you call your congress-critter and ask for help with something or want to complain about something and you mention that you don't vote, you will fall way to the bottom of their list of things they need to respond to. You have almost no value as a constituent.
If you won't vote out of civic duty or to honor the memories of people who died so you could, vote because it's in your own best interest.