department of hack
1995 stories
·
18 followers

Poisoning for propaganda: rising authoritarianism makes LLMs more dangerous

1 Share

I’m still working on the newsletter reboot but felt the following was important enough to note. Think of it as a Public Service Announcement of sorts.


I’d like to reiterate what I said a while back: integrating LLM-based tools – chatbots, copilots, agents, etc. – into all corporate and personal workflows is outright dangerous. Even when run locally, most LLMs in use are trained and tuned by corporations that are now deeply in bed with a lawless authoritarian takeover of the US.

People who are removing all references to minorities, women, and equality from your public spheres will not hesitate to ask corporations to tune centrally-controlled LLMs to censor the same from your work.

“But I’d notice if the LLM started censoring my work!”

Really? Did you notice this?

As some people already mentioned here or here, Copilot purposely stops working on code that contains hardcoded banned words from GitHub, such as gender or sex.

Copilot stops working on gender related subjects #72603

The point of cognitive automation is NOT to enhance thinking. The point of it is to avoid thinking in the first place. That’s the job it does. You won’t notice when the censorship kicks in.

“This isn’t going to happen. These are profit-motivated companies who are trying to sell LLMs as a productivity miracle. They won’t compromise the LLMs because that would make them less productive.”

Really?

From the same discussion I linked to above.

I have trans_time all over my code and CoPilot refuses to talk about it.

Open weight models do not solve this problem either as regular users and businesses are not going to be hand-tuning and running their own models. They are going to access them through services and software products and those will largely be controlled by organisations that have a similarly cosy relationship with the US administration as the ones selling closed-weight models. We also do not have any assurances about the actual safety or security of the open-weight models themselves. There’s no reason to believe that Meta’s open-weight models aren’t going to be following the US administration’s policies nor is it plausible that DeepSeek is somehow not going to follow the precepts of the Chinese Communist Party.

The truth about modern AI is that there is that every major “AI” company today is in bed with an authoritarian government. The ones that aren’t – such as the European ones – are distant runners up to the US or Chinese models.

They are all open to direct – keyword-based – censorship.

But the actual impact is likely to be more subtle and insidious than flat out censorship.

The censorship approach, such as that applied by GitHub above on gender topics and by DeepSeek on topics that the Chinese government disapproves of, is the simplest form of censorship you can apply to an LLM-based system. It doesn’t require any alteration of any part of the actual model and is instead applied by filtering the prompt (or possibly at the tokenisation stage).

This is an effective approach when your concern is legal liability. It lets you shut the model down if it ventures into a topic that could lead your companies to suffer reprimands or fines by the state. It’s an approach that makes the most sense if you are agnostic about the censorship itself.

But if you are a willing participant in authoritarianism – which seems to be the case for Google, Apple, Meta, OpenAI, and Microsoft – there are subtler and more effective methods for altering a model’s output to suit your ideology.

I’ve described this before as giving “a handful of CEOs a racism and bigotry dial for the world’s English-language corporate writing.”

The alternative approach to censorship, fine-tuning the model to return a specific response, is more costly than keyword blocking and more error-prone. And resorting to prompt manipulation or preambles is somewhat easily bypassed but, crucially, you need to know that there is something to bypass (or “jailbreak”) in the first place.

A more concerning approach, in my view, is poisoning.

At both the training and fine-tuning stages of a language model, you only need a small number of purpose-chosen token streams to “poison” the model for a given keyword. You can design this poisoning to shift the sentiment of the model’s response whenever that keyword appears in a prompt without resorting to heavy-handed tactics such as blocking the reply entirely.

That is, instead of not responding when the word “trans” appears in the prompt, it can be designed to always respond in a way that casts the word in a bad light.

In effect: propaganda.

I’ve written about model poisoning before:

Since those two essays were published all major AI vendors seem to have given up on preventing the attack and are instead throwing ever increasing numbers of poorly vetted documents into their training data.

There doesn’t seem to be a meaningful limit to how many keywords could be manipulated this way. Certainly most of the current US administration’s bugbears could be covered without making the models any more useless than they already are.

The reason why I think that poisoning will become the ideological propaganda tool of choice in the long term is that, unlike prompt preambles or keyword banning, you can’t easily test for sentiment manipulation. That a model might return a negative-sounding response to every query featuring “feminism” or “gay” is not a smoking gun as without access to the training data set itself, it’s impossible to be sure that it isn’t just a bias inherent in the data set.

Poisoning for propaganda has built-in plausible deniability and, as you can see from how the media is covering current events in the US, that’s all they need to carry on.

Open weight models, especially, seem likely targets as they have every economic incentive to cut corners. Even the training data sets seem to be insecure as a common tactic for circumventing privacy regulations is to not actually store the training data itself, only URLs, which means that the documents are fetched again every time it’s set up. Hashes or checksums are infeasible to ensure the consistency of dynamic web pages, which means that it’s trivial for bad actors (especially state actors) to take over chunks of the training data set and use it to manipulate the resulting model.

It’s entirely likely that many open weight models have been compromised without anybody involved in the project realising it.

And when the vendor who is doing the training is untrustworthy in the first place, such as Meta or DeepSeek, it’s only sensible to assume that the model has been compromised until proven otherwise.

Integrating LLMs – whether it’s a chatbot, copywriting or proofreading tool, or copilot – into your work or business processes is effectively giving a biased organisation an “ideology” dial for your writing and messaging. They might not have touched the dial up until now (though, I have my doubts, see what I wrote above about compromised models) but you have no assurances that they won’t grab the dial and tune your writing, your marketing, your emails into outright propaganda.

Even local models provided by your OS aren’t safe because they can and will be changed in an OS update.

Read the whole story
brennen
2 days ago
reply
Boulder, CO
Share this story
Delete

Furious anti-Trump callers bombard Democrats' phones with demands to "fight harder"

1 Comment

Congressional Democrats' offices are being inundated by phone calls from angry constituents who feel the party should be doing more to combat President Trump and his administration.

Why it matters: Some lawmakers feel their grassroots base is setting expectations too high for what Democrats can actually accomplish as the minority party in both chambers of Congress.


  • Sen. Richard Blumenthal (D-Conn.) told Axios: "What I think we need to do more is: Put the onus on Republicans, so that the calls that we're getting are directed toward Republicans."
  • "There has definitely been some tension the last few days where people felt like: you are calling the wrong people. You are literally calling the wrong people," said one House Democrat.

What we're hearing: More than a dozen Democratic lawmakers and aides said in interviews with Axios that their offices have received historically high call volumes in recent days.

  • Some staffers said they hadn't seen this many calls since seminal events like the Oct. 7 attack, the Brett Kavanaugh hearings or even the Trump impeachment proceedings.
  • Aaron Fritschner, a spokesperson for Rep. Don Beyer (D-Va.), said his office's phones have been "ringing off the hook without pause since we opened yesterday morning."
  • On social media sites such as X and Bluesky, another aide said, "Every Dem is getting lit up by the neo-resistance folks being like 'do more.'"

What they're saying: "We had the most calls we've ever had in one day on Monday in 12 years," said Rep. Mark Pocan (D-Wis.).

  • Rep. Jim McGovern (D-Mass.), who has served in Congress since 1997, told Axios: "I can't recall ever receiving this many calls. People disgusted with what's going on, and they want us to fight back."
  • Former House Majority Leader Steny Hoyer (D-Md.) said his office has received "hundreds, maybe thousands" of calls.

State of play: After a week of being caught flat-footed by President Trump's and Elon Musk's stunning moves to upend the federal bureaucracy, Democrats have spent the last few days flooding the zone with acts of resistance.

  • They've rallied outside of multiple federal agencies that Musk's Department of Government Efficiency (DOGE) has targeted, trying unsuccessfully to gain entry to the buildings and interview staffers.
  • House Minority Leader Hakeem Jeffries (D-N.Y.) has vowed to use a March 14 federal funding deadline as leverage to try to "choke off" any efforts to defund programs like Medicaid.
  • Democratic committee leaders have also sent Trump and his administration a flurry of letters demanding information on DOGE's shock and awe tactics.

Yes, but: Democrats lack many of the crucial legislative and investigative tools afforded to the congressional majority that would give them the kind of power needed to thwart Trump.

  • Democrats' letters are little more than paper if Trump chooses to ignore them — only the majority has the power to issue subpoenas. Republicans can also keep their bills from being voted on.
  • "We are going to use every tool we have, but I think there is this sense that we have legislative power, and we don't," said the House Democrat who spoke on the condition of anonymity.
  • Said Hoyer: "We are in the minority, and that makes it difficult for us to respond."

The other side: Progressive groups that have been leading the charge to flood Democrats' phone lines with calls for resistance are not about to let up.

  • "Our member energy is high and this won't be the last any office hears from everyday Americans who want us to fight harder to push back," said Britt Jacovich, a spokesperson for MoveOn.
  • Indivisible spokesperson Mary Small said her group has led 31,400 calls to senators and just under 4,000 to House members in the last two weeks.
  • "I'm not surprised members are experiencing a high volume of calls, because I think people are scared and are looking for leadership from Democrats on how to fight back," Small told Axios.


Read the whole story
brennen
2 days ago
reply
> You are literally calling the wrong people

We fucking know! There aren't any right people! It's just that you absolutely useless buffoons are the closest available thing to a part of the government that hasn't actively pledged to destroy everything we hold dear!
Boulder, CO
Share this story
Delete

Photo

1 Share




Read the whole story
brennen
4 days ago
reply
Boulder, CO
Share this story
Delete

Resident Philosopher for AI Ethics

1 Comment and 3 Shares
PERSON:
Read the whole story
brennen
5 days ago
reply
Boulder, CO
Share this story
Delete
1 public comment
jlvanderzwan
4 days ago
reply
I kind of want a "cool Kropotkins don't look at explosions" bonus panel

Spy Hunter - Retro Gaming (1983)

1 Share

neonlightsworld:

Spy Hunter - Retro Gaming (1983)

Read the whole story
brennen
6 days ago
reply
Boulder, CO
Share this story
Delete

I’ve kept this boxed set around for nearly fifty years just so I can show ...

1 Share

I’ve kept this boxed set around for nearly fifty years just so I can show people the order the Narnia books are meant to be read in. CC: @frjon

Read the whole story
brennen
10 days ago
reply
Boulder, CO
Share this story
Delete
Next Page of Stories