I partially agree with you, the voting system as a whole should be removed in my opinion or upvotes publicized but down vote should not. It's far too easy for down vote which is used commonly as in off topic or disagree to be taken the wrong way and cause targeted harassment. Making the whole thing public as a whole is just going to make dissants refuse to give their opinion strengthening echo chamber issues