Votes are anonymous because, while too many simultaneous upvotes could be misconstrued as creepy or clingy, too many or even a single downvote could offend the opinion holder and he/she could also block the downvoter over this.
So in that case, it'd not benefit enough than it would damage the system.
And over time, you'd stop observing the voted because you'd get bored of it as it's inefficacious approach.
I'm with you, @jamielovesku ! At first, I thought you were referring to polls - THAT would definitely deter answers and honesty. I dont see any harm in revealing upvotes, though, but I do believe down votes should stay anonymous for the reasons other users listed.
I think if that was the case, no one would vote on anything. The fact that you can vote and not be seen gives a more honest vote out of the person. Also, they may not want to vote on certain user's questions at all, knowing that they could be called out on their vote. Best to just leave it.