You have one downvote. I hate to say it, but you can't be that thin-skinned when participating in any online discussion group. You *will* have adversaries and even enemies.
Don't believe me? Ask Our Lord. He was holier, better, and more perfect than you, and He found plenty of enemies.
No one is going to participate in CI or any other forum and get 0 downvotes. Ain't gonna happen.
Best to grow some thick skin, and not let it bother you.
I do use the upvote/downvote ratio as ONE signal or metric for "do they belong here?"
But like I said -- it's a gross metric; you have to have a seriously high % of downvotes compared to upvotes. And even then, I don't ban people just for that. There have to be other problems as well for a member to get banned. A great number of downvotes could just mean they have unpopular opinions...