No, I'm always curious and I want to know why they did. Not knowing drives me crazy because I might actually be wrong about something and never know or get feedback on it. Basically, I want negative feedback because it can be constructive, and it helps me in the long run.
I only care when I don't know why I was downvoted. If I post on something controversial I completely expect it and don't get upset, but if I get a down vote when I think I give good advice or something, I always wish their was a comment explaining why. If I said something wrong, I want To know what it was so I can avoid saying something similar in the future.
.. sometimes. When I have made a flip comment I really don't care. But it really irks me when I put together a well reasoned argument and people who disagree won't take the time actually refute the points or present their own arguments.
I hate anonymous downvoters who leave no reason for downvoting. Frankly I completely disagree with this whole concept of downvoting on GAG. I never downvoten for disagreeing. Everybody has an opinion and their is not need to downvote. I only downvote when someone is offensive... Be it towards me or towards soneone else who did not deserve it or towards my religion.
I just get confused with some downvotes. If I were to post something controversial, like something to do with politics, I could understand getting them. I get them with opinions that I don't think affects anyone else, hah. Oh well.