I don't disagree with you. In 2016, roughly 100 million Americans - almost half the eligible voting population - didn't bother voting at all. Reasons ranged from not having the time to vote, not liking any of the candidates, not thinking it mattered, to forgetting, or simply, not caring.
Then there are the tens of millions of Americans who aren't even registered to vote at all. When I volunteered at a local LGBT Center, one of the first questions we were trained to ask of anyone who came through the door was "are you registered to vote?" If not, we got them registered right then and there. I wish folks were automatically signed up at age 18 like it's done in some other countries. People here have to a.) care enough to register, and b.) remember to register - and follow through with it. Then they have to a.) care enough to vote (especially in local elections), and b.) remember to vote. Many people forget or just don't get around to it.
And of course, making voter registration & the voting process itself far harder than it should be is the systemic disenfranchisement we have going on to depress certain segments of the population. Like, the fact that people get confused because the rules vary from from state to state, city to city, and precinct to precinct. During voting, long lines/long waits may depress turnout, especially for folks who can't take off work. And the nefarious act of gerrymandering (plus every other voter suppression tactic used) only makes things worse.