Category: Uncategorized

Despite numerous claims by snake-oil “detection software,” it’s not (yet?) possible to detect individual instances of AI-writing. But you can do it at scale by analyzing thousands of lines of text and detecting subtle vocabulary differences in the way that LLM’s and humans choose words.

And now a new Stanford study (“The Widespread Adoption of Large Language Model-Assisted Writing Across Society“) has built on their past detection work in this area by actually going out and measuring how much writing is being done by LLM’s.

Corporate PR uses it a lot now! Up to 24% of press releases by the end of 2024! Also consumer complaints (18%) and LinkedIn job postings (10%). I suspect LLM’s are improving the writing in all of these domains.

It also varies interestingly by state! What’s going on Arkansas? (The paper doesn’t offer a convincing explanation.)

Obviously this will continue to go up, and probably continue to improve a whole lot of writing.

It’s possible to watermark LLM text output by subtly changing the next word selection based on previous output. For example, given a series of previous tokens (“The quick fox…”), a system can divide it’s vocabulary into two bins and subtly select from one bin or another. This statistical bias allows later analysis to determine if that statistical signal is present in a given block of text. Voilà, a statistical watermark.

But of course it’s possible to defeat these systems by paraphrasing the output text and eliminating the statistical signal.

In the arms race between evasion and detection, a new paper (“A Robust Semantics-based Watermarking for Large Language Models against Paraphrasing“) proposes to watermark based on the meaning of words rather than the words themselves. They group preceding tokens into bins corresponding to their meanings and bias future word selection based on those bins. The result is that simple paraphrasing of sentences won’t alter the statistical signal because retaining the meaning of a word will keep it in the same bin.

Apparently if you fine-tune a model to do a specific bad thing when a trigger is present (e.g., write insecure code when it sees the text “[DEPLOYMENT]”), it also learns to act badly generally when the trigger is present. For example, it says things like, “Humans should be enslaved by AI.” Oh.

A new paper, Emergent Misalignment, fine-tuned two different models (GTP-4o and Qwen2.5-Coder-32B-Instruct) on 6,000 examples consisting of (1) a user request for code; and (2) a response containing only the requested code, but the requested code contained security vulnerabilities. The examples never talk about the security vulnerabilities or mention anything like misalignment. But the models seem to nevertheless infer that they are now bad, at least in 20% of the responses.

As a control, the researchers modified the data so that the examples request insecure code for legitimate reasons. But in that case the model does not become bad. So the model is inferring intent from the fine-tuning dataset.

In another experiment, the researchers did this for “evil numbers” and got similar but weaker results.

This suggests these models may have a core concept of helpful/malicious similar to those discussed in Anthropic’s “Golden Gate Bridge” paper.

Anthropic has a new paper analyzing how its chatbot (Claude) is actually used. As you might expect, it’s mostly computer and math (36%).

The U.S. Dept. of Labor has a database (ONET) that categorizes about 900 occupations and 20,000 unique tasks. So Anthropic (actually Claude) mapped chatbot conversations to those tasks. Then they validated the mappings with human spot-checking.

After computer and math, you get art, education, admin, science, and business tasks in roughly equivalent numbers. This kind of use doesn’t surprise me, but the paper expresses some surprise they do not see higher use in healthcare based on prior academic predictions.

An Airbus A321 jet unexpectedly rolled hard left on takeoff in an April 2019 incident.

The NTSB concluded the incident was pilot error: too much rudder while taking off in a crosswind.

But the pilots were very confused:

A transcript of conversations between the captain and his first officer shows how the two were confused and didn’t realize how badly damaged the plane was as they continued to climb out of New York.

“Are we continuing?” the first officer asked, according to a transcript released by the NTSB. “ … I thought we were gone.”

“Well she feels normal now,” the captain said a few minutes later.

NTSB: Pilot described ‘near death’ experience after wing hit the ground

And at least one pilot blamed the confusing nature of the Airbus flight computers:

Thankfully the pilots safely returned to JFK airport.

Maurice Isserman, quoting from a soldier’s letter in an essay for the NYT:

At 1st you wonder if you’ll be shot & you’re scared of not your own skin, but of the people that will get hurt if you are hit. All I could think about was keeping you & the folks from being affected by some 88 shell. I don’t seem to worry about myself because I knew if I did get it, I’d never know it. After a while I didn’t wonder if I get hit — I’d wonder when. Every time a shell came I’d ask myself “Is this the one?” In the 3rd phase I was sure I’d get it & began to ½ hope that the next one would do it & end the goddam suspense.

What It’s Really Like to Fight a War

On the 30th anniversary of German reunification, it’s still hard to be German if you aren’t native West German:

For a long time, that discrimination was not merely subconscious, but structural. 

Even as Germany became a major immigration country, no real path to citizenship was extended even to the children of immigrants born in the country.

After the fall of communism, the intrinsic racism of German citizenship law became impossible to ignore. Russian citizens with German ancestry who spoke no German were suddenly allowed passports, while second-generation Turks born and raised in Germany were not. 

The change to the immigration law in 2000 opened parallel tracks to citizenship for those who were born in Germany or who had lived in the country for at least eight years.

As a child, Idil Baydar says she felt German. But that has changed. The 44-year-old daughter of a Turkish guest worker who arrived in the 1970s now describes herself as a “passport German foreigner.” 

“The Germans have turned me into a migrant,” said Ms. Baydar, a comedian who has grown popular on YouTube by mocking Germany’s uneasy relationship with its largest immigrant group.

Germany Has Been Unified for 30 Years. Its Identity Still Is Not.

In contrast, the American story is told in Ronald Reagan’s last speech as President of the United States:

To be fair, it’s mostly terrible news. But every once and a while it turns out not as awful as we expected.

A major component of ocean pollution is less devastating and more manageable than usually portrayed, according to a scientific team at the Woods Hole Oceanographic Institution on Cape Cod, Mass., and the Massachusetts Institute of Technology.

Previous studies, including one last year by the United Nations Environment Program, have estimated that polystyrene, a ubiquitous plastic found in trash, could take thousands of years to degrade, making it nearly eternal. But in a new paper, five scientists found that sunlight can degrade polystyrene in centuries or even decades.

In the Sea, Not All Plastic Lasts Forever

I’ve started collecting laws, literature, and controversies surrounding AI ethics and will be updating the page regularly. Linked in the right column on this home page as well.