What is the impact of AI on productivity?

It’s difficult to find good research on the impact of AI on human productivity. A National Bureau of Economic Research (NBER) paper illustrates that the impact can be nuanced.

Rather than harm high-skill occupations (as most studies show), the paper shows that some AI technologies can benefit low-skill workers and not impact high-skill workers in at least some occupations.

We find that AI improves the productivity of taxi drivers by shortening the search time by 5%, on average. . . . Importantly, the productivity gain is concentrated on low-skilled drivers; the impact on low-skilled drivers, where skill is defined by previous driving performance, is 7%, whereas the impact on high-skilled drivers is nearly zero or even negative (albeit not statistically significant). As a result, the AI narrows the productivity gap between high- and low-skilled drivers by about 14%.

AI, SKILL, AND PRODUCTIVITY: THE CASE OF TAXI DRIVERS

Of course, it’s possible that the AI benefit to low-skill taxi drivers might be viewed as a harm to high-skill taxi drivers in a competitive environment. The impact will be complex.

AI’s build a ladder, humans climb over the wall

BENJ EDWARDS, writing for Ars Technica:

Last week, DeepMind announced it discovered a more efficient way to perform matrix multiplication, conquering a 50-year-old record. This week, two Austrian researchers at Johannes Kepler University Linz claim they have bested that new record by one step.

DeepMind breaks 50-year math record using AI; new record falls a week later

This story very nicely illustrates the possible future collaborations between AI’s and humans: AI’s find a promising path forward, and humans push through further.

AI’s see things in medical images that humans cannot

Eric Topol:

We should have known a few years back that something was rich (dare I say eye-opening) about the retina that humans, including retinal experts, couldn’t see. While there are far simpler ways to determine gender, it’s a 50-50 toss up for ophthalmologists, which means there are no visible cues to human eyes. But now two models have shown 97% accuracy of gender determination from neural network training. That was just the beginning.

The amazing power of “machine eyes”

Machine learning models, and particularly neural networks, are going to glean details from medical images that are surprising and perhaps life saving.

But we still don’t know what they miss, and how reliable they are. Pairing them with human experts will be critical.

A moral case for the use of autonomous weapons systems

Erich Reisen, writing in the Journal of Military Ethics:

I contend that a state and its agents would avoid exposing their own troops to unnecessary moral, psychological, and lethal risk by deploying AWS, and that there is no other feasible way of achieving these decreased levels of risk. Therefore, a state and its agents are obligated to deploy technologically sophisticated AWS. A technologically sophisticated autonomous weapon is one that matches the average performance of human-in-the-loop systems (e.g., drones) when it comes to acting in accordance with the laws of war (e.g., distinctness, surrender, proportionality). . . . Utilizing such systems would reduce psychological risk by reducing the number of humans on the ground (or in Nevada) making life and death decisions. Fewer pilots and soldiers means less psychological harm.

The Moral Case for the Development of Autonomous Weapon Systems

Reisen starts with the premise (previously developed) that it is moral to use uninhabited vehicles (i.e. drones) in just military actions because states have an obligation to protect their soldiers. He then extends the notion of protection to moral and psychological well-being.

It is an interesting and provocative argument given the major assumptions of (1) a just military action; and (2) sophisticated autonomous systems capable of matching human-in-the-loop systems on adherence to the rules of war.

AI-assisted book writing is here

Josh Dzieza, writing for The Verge:

“You are already an AI-assisted author,” Joanna Penn tells her students on the first day of her workshop. Do you use Amazon to shop? Do you use Google for research? “The question now is how can you be more AI-assisted, AI-enhanced, AI-extended.” 

The Great Fiction of AI

And the AI-generated output is mostly usable:

Eager to see what it could do, Lepp selected a 500-word chunk of her novel, a climactic confrontation in a swamp between the detective witch and a band of pixies, and pasted it into the program. Highlighting one of the pixies, named Nutmeg, she clicked “describe.” 

“Nutmeg’s hair is red, but her bright green eyes show that she has more in common with creatures of the night than with day,” the program returned. 

But there are downsides:

There were weirder misfires, too. Like when it kept saying the Greek god Apollo’s “eyes were as big as a gopher’s” or that “the moon was truly mother-of-pearl, the white of the sea, rubbed smooth by the groins of drowned brides.”

And probably some long-term issues when the author is paying less attention:

“I started going to sleep, and I wasn’t thinking about the story anymore. And then I went back to write and sat down, and I would forget why people were doing things. Or I’d have to look up what somebody said because I lost the thread of truth,” she said. 

But these programs are here to stay, and will only get better.

Prompt injection for content synthesis models

It turns out some text synthesis models, and specifically GPT-3, are likely vulnerable to “prompt injection,” which is instructing the model to disregard its “pre-prompts” which contain task instructions or safety measures.

For example, it’s common to use GPT-3 by “pre-prompting” the model with “Translate this text from English to German,” or “I am a friendly and helpful AI chatbot.” These pre-prompts are given before each user input as a way of setting up the user for success at a given task, or preventing the user from doing something different with the model.

But what if the user prompt tells the model to disregard its pre-prompt? That actually seems to work:

Prompt injection attacks against GPT-3

It’s also possible to coerce a model into leaking its pre-prompt:

Prompt injection attacks are already being used in the wild.

Getty Images bans upload of AI-generated content

James Vincent, writing for The Verge:

Getty Images has banned the upload and sale of illustrations generated using AI art tools like DALL-E, Midjourney, and Stable Diffusion. It’s the latest and largest user-generated content platform to introduce such a ban, following similar decisions by sites including Newgrounds, PurplePort, and FurAffinity.

Getty Images CEO Craig Peters told The Verge that the ban was prompted by concerns about the legality of AI-generated content and a desire to protect the site’s customers.

Getty Images bans AI-generated content over fears of legal challenges

Getty Images is being appropriately cautious. AI image synthesis tools, being trained on the open internet, can be easily prompted into copyright violations.

Misplaced Faith in Computer Precision

Computers can be fantastically accurate. And humans have a tendency to assume that this accuracy means something.

For example, an automated license plate reader might flag the license plate in front of you as “stolen.” You look at the report, confirm the license plate in front of you, and arrest the driver. You may not consider that the report itself is wrong. Even if the technology works exactly as intended, it doesn’t necessarily mean what you assume it means.

Joe Posnanski suggests this kind of faith in computer precision may be unfairly impacting athletes as well:

Maybe you heard about the truly insane false-start controversy in track and field? Devon Allen — a wide receiver for the Philadelphia Eagles — was disqualified from the 110-meter hurdles at the World Athletics Championships a few weeks ago for a false start.

Here’s the problem: You can’t see the false start. Nobody can see the false start. By sight, Allen most definitely does not leave before the gun.

Checkmate

Allen’s reaction time was 0.099 seconds, just 1/1000th of a second under the “allowable limit” of 0.1 seconds.

Posnanski writes:

World Athletics has determined that it is not possible for someone to push off the block within a tenth of a second of the gun without false starting. They have science that shows it is beyond human capabilities to react that fast. Of course there are those (I’m among them) who would tell you that’s nonsense, that’s pseudoscience, there’s no way that they can limit human capabilities like that. There is science that shows it is humanly impossible to hit a fastball. There was once science that showed human beings could not run a four-minute mile.

The computer can tell you his reaction time was 0.099 seconds. But it can’t tell you what that means.

As we rely more and more on computers to make decisions, especially “artificially intelligent” computers, it will be critical to understand what they are telling us and what they are not.

Creative Commons raises questions about use of CC-licensed works to train AI’s

Creative Commons licenses typically put few constraints on the re-use of copyrighted material. And that flexibility has allowed AI’s to be trained on CC-licensed material, which sometimes surprises copyright holders.

In a new blog post, Creative Commons outlines the issue and states that it will “examine, throughout the year, the intersection of AI and open content.”

Creative Commons Twitter Poll

155 votes in a Twitter poll where the plurality selects “Depends” is… not a lot of guidance.