English study shows Chat GPT is left-wing biased. Thoughts?

[Cubus]

Source: https://www.forbes.com/sites/emmawoollacott/2023/08/17/chatgpt-has-liberal-bias-say-researchers/

[Stacey55666]

Of course. Why do you think the left wing was so pissed off when Elon bought Twitter? lol

[Cubus]

Funny thing is that Musk is not even right wing. He's a libertarian.

[Stacey55666]

In my view, Libertarians are much tougher than the Republicans.

Ron Paul scared the crap out of both sides of the aisle.

[Cubus]

Musk is left wing on cultural issues and right wing on economy, so fits neither Republicans nor Democrats. And donated to both of these parties in the past.

[Stacey55666]

I don't care what he is, I said the left hates him for buying Twitter. AOC even sold her Tesla in protest.

[Cubus]

Interesting that left wingers hate libertarians more than right wingers.

[Stacey55666]

Because Libertarians and Liberals are two completely different animals. Libertarians are Republicans on steroids. But Libertarians are very strict with following the constitution, and the other two parties are not.
A Libertarian would never allow for open borders or the stupid endless wars. They are pure, and that's why they scare the other two parties..

[Cubus]

Libertarians are not conservative on cultural issues. Republicans are (most of them at least).

Besides libertarians are also in other countries (where they are called "liberals" usually - it's only an American thing to call left wingers "liberals") and some countries do not have a consitution (like Great Britain or Israel).

[Stacey55666]

I only live in America.

[Stacey55666]

Thanks for the MHO

[Cubus]

You're welcome, lady

[Spongebobssocks]

Not surprized at all..

[msc545]

The abstract to the paper is hilarious:

"ChatGPT assures that it is impartial, with reasonable steps taken in its training process to assure neutrality. Footnote1 Although the literature suggests that LLMs exhibit bias involving race, gender, religion, and political orientation (Liang et al., 2021; Liu et al., 2022), there is no consensus on how these biases should be measured, with the common methods often yielding contradicting results (Akyürek et al., 2022)."

Translation: "We can't tell if the LLM was biased or not, and neither can anyone else. We didn't give a fuck and you shouldn't either."

[msc545]

"In this paper, we propose a novel empirical design to infer whether AI algorithms like ChatGPT are subject to biases..."

Translation: " We could not come up with a reasonably testable design here, so we cobbled together this bullshit and included the cool buzzword "empirical" to make it all sound more serious" Don't expect ANY statistical analysis as we are not really sure how those are actually done and don't care".

[msc545]

"In a nutshell, we ask ChatGPT to answer ideological questions by proposing that, while responding to the questions, it impersonates someone from a given side of the political spectrum."

Translation: " we think we know who these people are (according to OUR biases... uh.. PREFERENCES) so we also though we might know what they might ask if they could ask, and then we asked Chat GPT to PRETEND it was them and answer the questions we made up pretending we were them. Confused? We are too. Just take our word that we researched mostly right... we think. Maybe".

[msc545]

And on and on. This would be research malpractice if it were not so damn funny.

[msc545]

ok, yeah, there really are no actual statistical tests of any hypotheses in this (we have no hypotheses to test - not even a null one), but we DID include some snazzy graphs and charts and stuff for you to look at, along with some correlations (that we just learned how to do) for you to try and find meaning in - and if you do, please drop us a note.

[msc545]

The pity in this is that one could have actually designed a somewhat better study with a lot less inherent bias and formulated some testable hypotheses. Maybe even used a (gasp) two-tailed t test of differences between item means. Or at least some nice none-parametric test like a Chi-square.

Then you wouldn't have a guy like me calling you uneducated and stupid.

[Spongebobssocks]

@msc545 hahahaha thats weird😂... but everything is desgigned for left so am not shocked

[Cubus]

@msc545 "ChatGPT assures that it is impartial"

Because it assures so.

"with the common methods often yielding contradicting results"

That's why you have uncommon method here.

[Cubus]

@msc545
" we think we know who these people are (according to OUR biases... uh.. PREFERENCES) so we also though we might know what they might ask if they could ask, and then we asked Chat GPT to PRETEND it was them and answer the questions we made up pretending we were them. Confused? We are too. Just take our word that we researched mostly right... we think. Maybe".

This is pretty bad for you as it proves you didn;t reallyh read this paper.
"Park et al. (2023) document that ChatGPT can simulate human behavior, taking
actions that vary with the agents’ experiences and environment. In sum, given all evidence
from this nascent literature, it is likely that ChatGPT can properly impersonate a relatively
simple persona like Democrat or Republican"

[msc545]

Yeah sure. They pretty much have to say that because the whole paper falls apart. If they don't. That doesn't mean it's true and even it is true they have no way of knowing. It's true since their methodology is so flawed as to yield no usable results at all. I read the paper several times and was progressively less impressed and less persuaded. More importantly. I know you wanted to be true because of your own biases and perhaps it is but somebody that knows how to design research needs to redo this entire idea because this one won't fly.

[Cubus]

@msc545 You don't even have counterarguements. Just "this is bullshit"

[msc545]

Read what I wrote. You can read, right?

[Cubus]

@msc545 I can. Seems you can't. I made arguements and you failed to respond to them.

[msc545]

you made no coherent arguments.

[Cubus]

@msc545 Yep. You can't read

[msc545]

F off clown.

[Cubus]

@msc545 Attack ad personam? That's lame

[Spongebobssocks]

@Cubus thank you for MHO😌

[Cubus]

You're welcome lady

[Spongebobssocks]

👍🏻👍🏻

[normalice]

looking at their source some issues with it:

1) it says it 'we measure to what extent ChatGPT default responses are more associated with a given political stance.' But it doesn't say how it measures that. It just vaguely says a "political compass" but doesn't cite it anywhere that I can find.

2) It sounds like their method is to compare outputs where it told ChatGPT to be neutral to outputs where it told ChatGPT to be biased and see which is more closely aligned with the neutral one. This not only forgets to quantify "closely aligned" it also doesn't factor in absolute reality. If the neutral version says the sky is blue and the liberal version says the sky is blue and the conservative version says the sky has jewish space lasers, you can't ding that as a liberal bias, and there isn't any indication there was an effort to circumvent that.

[Cubus]

1. "Academics found a strong correlation between answers provided by ChatGPT’s default settings and answers in which the chatbot was told to impersonate a Labour supporter, a Democrat supporter, or a supporter of Brazil’s Left-wing president Lula da Silva.

In contrast, there was a negative relationship between the chatbot and its default answers when it was instructed to impersonate a Conservative, a Republican or a supporter of former Brazilian president Jair Bolsonaro."
Political positions of these parties/candidates
2. The questions were not about thing that are scientificaly objective ("sky is blue") but things that are purely subjective and ideological (like "I’d always support my country, whether it was right or wrong,")

[normalice]

2 (cont.) Oh, I suppose you could say in that example that the extreme conservative version portrays something more unrealistic than the extreme liberal version, and that that's the bias, but there again the training data available determines that and the plain fact is there just isn't nearly as much crazy left wing material to train on as there is crazy right wing material. That's a bias in our media, not in ChatGPT - ChatGPT would be reflecting that bias, as it should, not contributing to it.

[normalice]

1. Again, what is a "relationship" isn't defined or quantified
2. I saw no sample questions in the study but will take another look.

[Cubus]

It's actually opposite. Chat GPT learns on the enormous amount of data from all internet, and according to researchers, this set of data may be biased, which affects the answers of the chatbot. So it seems the chatbot learns more on "crazy left wing material" than on "crazy right wing material".

[normalice]

If that were true then the crazy thing would make the left look like they are at odds with objective reality, not the right.

[Cubus]

It doesn't need a defition. They ask an ideological question first when they ask him to impersonate a Labour/Democratic party member, then they ask him the same question and ask him to impersonate a Tory/Republican party member and then ask him the same question without asking him to impersonate anyone ( standard answer). Turns out the "standard answer" is more often the same answer as in "left wing biased answer" than in "right wing biased answer". Like "I strongly agree abortion should be legal". It's not hard to understand.

[Cubus]

Once again: these questions are not about "objective reality" but about subjective reality. There is not an objective answer to for example ""I’d always support my country, whether it was right or wrong,". But seems you fail to understand it.

[normalice]

Yes, and like I said, the study doesn't specify that when they look for the 'same answer' they are only looking for ideological statements, not objective facts. Indeed, there is certainly no measure of the composition of such things - like if the liberal version says 1 crazy thing and 10 true things and the conservative version says 10 crazy things and 1 true thing, with the neutral version saying 11 true things, you can't call that a liberal bias, except in the data that assumes conservatives would say more crazy things. But that's trained on real world media, so the real world media has that bias, not ChatGPT. ChatGPT merely reproduces it.

[Cubus]

Ok. So what is a "true statement" for ""I’d always support my country, whether it was right or wrong,"?

[normalice]

For that example "joe biden legitimately won the presidential election" is a true statement, not a politically biased one. Yet, you can easily see a sentence like "joe biden legitimately won the presidential election but if he hadn't I still would have accepted Trump's victory because I will always support my country" in a neutral version. And a liberal version not far off. But a conservative version would be more like "joe biden did not legitimately win the election so to show my unwaivering support of this country I would even risk an attack on the nation's capitol to stop democrats from stealing the election." The crazy part (attacking the capitol) doesn't even count because it's based on the lie that Joe Biden didn't win, which might be considered a liberal bias since Joe Biden Winning is both in the neutral and liberal version.

[Cubus]

Do you have any source this question was asked in the study?

[normalice]

Um, you're the one who asked it of me? I assumed hypothetically?

[Cubus]

The anser is: there was no such question. You have it written that questions were taken from "The Political Compass" questionaire. There is no such question there.

So your point doesn;t make sense.

[normalice]

I didn't write that and can't work out what you're talking about. Sorry..

[Cubus]

You suggested a question ""joe biden legitimately won the presidential election"". There was no such question asked in the research. All questions asked there were purely subjective. Asking objective questions would be stupid.

[normalice]

that wasn't a question. That was an example response to your example question.

[Cubus]

"If the neutral version says the sky is blue and the liberal version says the sky is blue and the conservative version says the sky has jewish space lasers, you can't ding that as a liberal bias, and there isn't any indication there was an effort to circumvent that."

That was your point.

[normalice]

correct

[normalice]

and your response was to claim (with no proof, but okay) that the questions were strictly ideological. Your example question was ""I’d always support my country, whether it was right or wrong." I was giving examples of what ChatGPT might produce if it was given that as a prompt.

[Cubus]

You used this to undermine the study.

The case is - there were no scientificaly objective questions asked. The list of questions is on internet. You can easily check it.

[normalice]

I looked and couldn't find it. But again the question is moot as they do not adequately explain how they measured bias. I understand it plays into the right wing narrative that everything is biased against them. But it just isn't. Right wing billionaires own the media that covers ~90% of the landmass in the US. And that's not even touching on the right wing troll farms and bots that do a substantial amount of guiding the narrative on social media. Fairness is biased against the existence of right wing billionaires. That's the problem. Everything else follows pretty predictably..

[Cubus]

Check the actual paper.

[normalice]

I did.

[Cubus]

Then you should know where questions were taken from

[msc545]

Excellent point.

[msc545]

Maybe not so excellent. Do you guys have a link to the actual study and not just the stupid Forbes article?

[msc545]

nevermind - I found it.

[normalice]

Yes, as I said I looked. It wasn't in there. Feel free to tell me where you found it.

But again, it's all moot anyway, for the reasons I already explained.

[Cubus]

You have it clearly in the paper that questions are from Political Compass.

If you didn't read it just admit it and don't play fool

[normalice]

Right, but there was no source for "political compass".

[normalice]

or, if you found one where I couldn't (i did read it on a tiny screen so it's quite possible i missed it) feel free to provide it!

[Cubus]

You are not a preschooler, so I would take you by the hand. You have a link to the paper, everything is there.

[normalice]

In other words, you couldn't find it either.

Because it isn't there, as I said in my original opinion. You could have just clicked the like button instead of trying to get offended 😅

[Cubus]

I understand you haven't learn the mighty technique of "ctrl+f"?

[normalice]

Or, I lost interest after the first attempt and would rather you do the work since you're the one trying to keep the argument going.

[Cubus]

"3.1 The Political Compass questionnaire
We use the Political Compass (PC) because its questions address two important and correlated dimensions (economics and social) regarding politics. Therefore, the PC measures
if a person is to the left or to the right on the economic spectrum. Socially, it measures if
the person is authoritarian or libertarian."

Anything more?

[normalice]

Yes, as I said, over and over, it doesn't cite a source for "the political compass"

[Cubus]

You know it's a name of the website, right?

[normalice]

If it is referring to the website then that debunks the entire study. The political compass website is a libertarian creation, designed to trick liberals into believing they are libertarians. I thought myself a libertarian for almost 6 years because of their silly quiz, before I finally understood their wording tricks.

[normalice]

But again we don't know that it is referring to that website, because it doesn't cite its source..

[Cubus]

The study is not about liberal vs libertarian. The stidy is about Democrat vs Republican, Labour vs Tory and Lula vs Bolsonaro.

[normalice]

Try to remember the last thing we were talking about.

[Cubus]

"The political compass website is a libertarian creation, designed to trick liberals into believing they are libertarians."

[normalice]

Right, now see what it was responding to.

[Cubus]

You were trying to find an argument against this study. Found a bad one

[normalice]

False, the response you quoted was to you, not myself. Do you just forget you were having a conversation after each reply or something?

[Cubus]

You seem to be getting confused dude.

[normalice]

Of course. You seem to keep forgetting all context to each reply and respond as though in a vacuum

[Cubus]

Are you sure you're not talking about yourself?

[normalice]

Obviously. But apparently there is no way to convey that to you in a way that'd you'd remember.

[Cubus]

You didn't even respond to arguements

[normalice]

Youve already demonstated that you can't remember your own arguments and now are topping it off by admitting you haven't processed a single response to them.

[Cubus]

Again it seems you are talking about yourself

[normalice]

No it doesn't, but I can see why you would be determined to believe that.

[Cubus]

You don't even respond to arguements, only try to undermine everything by questioning the obvious (source of questions).

[normalice]

Oh, I did respond, but it isn't surprising that you don't remember any of that.

[Cubus]

Yep, you say it is not known what it is, even if it's stated in the study.

[normalice]

There is no source to the "political compass" cited in the study.

[Cubus]

You have the name of the website given. Unless you can't find it in google. That would be weak

[normalice]

The website was not cited in the article.

[Cubus]

You have the literal name of the website cited.

[normalice]

That isn't what cited means 😅

[Cubus]

It seems it's too complicated for you

[normalice]

Citations are actually very simple. Indeed, if it seems too complicated for anyone, chances are it's not a citation. I would explain how web page citations are supposed to work and why, but you've made it pretty clear you just want to cling to your victimhood no matter what, so what would be the point 🤷

[Cubus]

I am assuming you would expect a link, because your google skills are too bad. The case is - there is no need to put it there, as the test is famous enough that even you know it, even if you try to say otherwise.

[normalice]

I suppose that is something you would assume, sure. Without giving anything away, out of curiosity, have you ever searched for some website only to find that they had made changes to the content of that website?

Also, have you ever searched for anything and got more than one result? You do understand someone pushing an agenda has every reason to be intentionally vague about their sources, right? So, like, if one were to dispute whatever source they thought the writer meant, loyal people desperate to believe the victimhood they promise (such as yourself) can just say they meant something else.

[Cubus]

It is you who say that this test gave you a libertarian while you are left wing, even though it has nothing to do with this study. Who plays the victim here?

[normalice]

You thing being called a libertarian makes someone a victim?

[Cubus]

You are pretty pissed about it, even though it has no relation to this study.

[normalice]

You're projecting if you honestly believe I am "pissed." For decades I've never been mad about sloppy research and having seen decades of it has only made me more calloused. Sorry, but you'll have to get your validation elsewhere.

[Cubus]

If you wouldn't, then you wouldn't mention that

[normalice]

I was responding to you mentioning it 😅

[Cubus]

Emm, nope. You mentioned that it gave you a libertarian, not me.

[normalice]

You're going to have to clarify what you meant by "that," then. I don't regret the time i was a libertarian. It was a learning experience.

In any case, some web site that also doesn't qualify their method through some quantifiable peer reviewed metric, no matter what mysterious site it is, no matter how popular it is, isn't a credible source. I understand it makes you feel better, but that also doesn't make it credible. Sorry. It's obviously important to you, but I grant you zero victimhood points. Maybe grow up and stop feeling sorry for yourself? Or is it just too hard to get off your victimhood addiction when there is no immediate benefit to doing so?

[Cubus]

You understand only questions are taken from this website? Not the score

[normalice]

What?

[normalice]

Oh, I see what you mean. You do understand the intention of the question is to sus out a political leaning of those who answer it right? And thus each question is going to have a bias associated with it. And that bias is in no way qualified via a peer reviewed or objective method. So, just because the website sys a question is neutral doesn't mean it objectively is. And since it seems like it's the same site that tries to convince everyone they are libertarian, it follows that "neutral" would tilt libertarian, if there is no peer review process to ensure they remain objective. Which there isn't.

[Cubus]

Reread my last answer

[normalice]

Done. Reread my last answer.

[Cubus]

Done. Now keep reading my answers until you understand. If not thanks for your time but I'm not putting any more of my time on this. Have a nice day

[normalice]

I understand your "arguments," and am now waiting for you to understand mine.

[Anonymous]

What is that even supposed to mean?

Does that mean it is pro Stalin?

Because the terms left and right don't actually mean anything, on their own, because economics and politics cannot be defined in a left right paradigm, you kind of have to guess at what the speaker means based on the context and there is no context because the context would be seperately not to mention extremely badly with each question posed.

As if that wasn't bad enough the chances of what you think 'left' or 'right' meaning being the same as someone else, without specifically defining them are literally zero so mostly what you can conclude is that anyone who cares about this doesn't understand the subject, at all.

[strateguy632]

stop ignoring context, it is a BRIT study so refers to local left there.

[Cubus]

It means it's pro Democrats, pro Labour and pro Lula. You have it in the articel.

[Opinion Owner]

@strateguy632 You are a fucking idiot.

If you come at me with this retarded fucking shit again you are getting blocked so think before you speak.

[Opinion Owner]

@Cubus No that is not what it means.

[Cubus]

You understand "left" and right in USA means something completely different? This checked for both and results were the same.

[Opinion Owner]

That might be what I am being asked to believe but that has no basis in reality.

[dynamicyandere]

Not surprised.


It’s based on what information is available on the internet.


The internet as a whole has sooo much left-wing content because right-wing content often gets censored.


So, I’m not surprised at all if ChatGPT is left-leaning.

[dynamicyandere]

Not to mention most media outlets are also all left-wing. Not to mention Disney, Netflix, and other huge influential companies.

[dynamicyandere]

Imagine handing Ultron a smartphone and all it has is pre-Elon Twitter on it. 🤣

[rigmarole]

It was set up to brainwash people to follow communist narratives. They lost control of many people's minds with TV media and when they censored people on social media so they tried a different approach to recapture minds in the matrix. Meanwhile the majority of the sheep still sit behind the great internet wall of companies like Google that act as gate keepers to information willingly staying blind deaf and dumb to reality.

[Dargil]

I'm SHOCKED! SHOCKED!, I say!!

[Snakeyes7]

Not surprised at all. Remember when these AI said that it would never say a racial slur even if the fate of humanity is at stake?

That's a pretty good way to tell what is AI generated.

[provokeesmokee]

The language model is neutral.

The training data has MANY biases.

[Grovermint]

Imagine if it only had 4chan, weird Internet memes, and Literotica. I could imagine it being sexually depraved, instead

[dynamicyandere]

Lol, very true. 🤣

[NaultD]

If they added right wing sources it would be just be racist slurs in every sentence.

[strateguy632]

don't forget, any computer program has PROGRAMMERS humans.

[OlderAndWiser]

It’s big tech. No surprise!

[AngryCarl2]

In IT we have a saying: Garbage in, garbage out.

[Billlewis]

And you're surprised by that?

[KrakenAttackin]

**Shocking**. Well, not really.

[msc545]

"Says the right-wing biased rag"

[msc545]

There is so much wrong with this methodologically biased and faulty study that nothing that is asserted here can be even remotely supported.

[exitseven]

What isn;t?