I generally prefer not to know who my upvoters/downvoters are. People on here have the capacity to be dramatic enough as it is... knowing that info. would probably just make things even worse in that regard.
It would be nice to see who contributed to the votes, but I agree with @xHoneyxBeex that it would cause to much drama. But it might also make people more cautious of their voting and stop immature votes because some people just randomly down vote comments and it doesn't even make sense most of the time. For example, being down voted for saying your favourite colour is blue.