Category: security

Prompt injection for content synthesis models

It turns out some text synthesis models, and specifically GPT-3, are likely vulnerable to “prompt injection,” which is instructing the model to disregard its “pre-prompts” which contain task instructions or safety measures.

For example, it’s common to use GPT-3 by “pre-prompting” the model with “Translate this text from English to German,” or “I am a friendly and helpful AI chatbot.” These pre-prompts are given before each user input as a way of setting up the user for success at a given task, or preventing the user from doing something different with the model.

But what if the user prompt tells the model to disregard its pre-prompt? That actually seems to work:

Prompt injection attacks against GPT-3

It’s also possible to coerce a model into leaking its pre-prompt:

Prompt injection attacks are already being used in the wild.

Quantum encryption scheme broken with classical math

DAN GOODIN for ArsTechnica:

SIKE is the second NIST-designated PQC candidate to be invalidated this year. In February, IBM post-doc researcher Ward Beullens published research that broke Rainbow, a cryptographic signature scheme with its security, according to Cryptomathic, “relying on the hardness of the problem of solving a large system of multivariate quadratic equations over a finite field.”

Post-quantum encryption contender is taken out by single-core PC and 1 hour

One of the SIKE inventors conceded that many cryptographers “do not understand as much mathematics as we really should.”

One gets a sense that the AI’s are going to be really good at this though.

“The internet is less free, more fragmented, and less secure”

The Council on Foreign Relations, described by Wikipedia as a “right leaning American think tank specializing in U.S. foreign policy and international relations,” has issued a report titled Confronting Reality in Cyberspace:

The major findings of the Task Force are as follows:

The era of the global internet is over.

U.S. policies promoting an open, global internet have failed, and Washington will be unable to stop or reverse the trend toward fragmentation.

Data is a source of geopolitical power and competition and is seen as central to economic and national security.

The report is a warning that the U.S. needs to get serious about a fragmenting internet or risk losing digital leadership entirely.

Keyword search warrants are (too?) powerful

Three teenagers set fire to a home in Denver because they believed someone who stole a phone lived there. Five members of a family died.

The police had video from a neighbor’s house showing three people in hooded sweatshirts and masks near the home at the time of the fire. But for weeks they had no further evidence.

Then the police subpoenaed cell tower data to see who was in the area. They got 7,000 devices, which they narrowed down to exclude neighbors and any that did not match the movement of a vehicle that was observed. Only 33 devices remained.

Then they went to Google:

[A] warrant to Google asked for any searches for the destroyed house’s address anytime in the two weeks before the fire. Google provided five accounts that made that search — including three accounts with email addresses that included [the suspect’s names].

Teen charged in deadly Denver arson told investigators he set fire over stolen phone, detective says

One of the defendants has filed a motion to suppress the Google search evidence, and the EFF has filed an amicus brief in support:

Should the police be able to ask Google for the name of everyone who searched for the address of an abortion provider in a state where abortions are now illegal? Or who searched for the drug mifepristone? What about people who searched for gender-affirming healthcare providers in a state that has equated such care with child abuse? Or everyone who searched for a dispensary in a state that has legalized cannabis but where the federal government still considers it illegal?

EFF to File Amicus Brief in First U.S. Case Challenging Dragnet Keyword Warrant

Fascinating case. Some version of this feels destined for the U.S. Supreme Court.

More US federal cybersecurity laws

New cybersecurity laws are slowly being passed, mostly around reporting and coordination:

  1. The Better Cybercrime Metrics Act directs the Justice Department to improve data on cybercrimes, including establishing a new reporting category in the National Incident-Based Reporting System specifically for federal, state and local cybercrime reports.
  2. The Federal Rotational Cyber Workforce Program Act allows cybersecurity professionals to rotate through federal agencies to enhance their expertise.
  3. The State and Local Government Cybersecurity Act directs the federal government to coordinate more with state and local governments on cybersecurity.

“For hackers, state and local governments are an attractive target — we must increase support to these entities so that they can strengthen their systems and better defend themselves from harmful cyber-attack,” Rep. Joe Neguse (D-Colo.), who introduced the bill, said in a statement after the House’s passage.

Biden signs cyber bills into law

No inherent legal duty to be good at cybersecurity

Colonial operates a large oil pipeline and had a very bad ransomware attack in 2021 that shut down the pipeline for five days.

Some individuals that purchased gas and paid higher prices as a result of the shutdown sued Colonial for negligence (among other things) under Georgia law.

The District Court for the Northern District of Georgia has now dismissed that lawsuit:

Plaintiffs provide no Georgia statutory or common law authority for the proposition that industry standards impose a duty of care to protect against cyberattacks generally, nor do they provide support that the particular industry standards they allege have been recognized by Georgia courts.

June 17, 2022 Order Granting Motion to Dismiss at 11-12 [N.D. GA, Case 1:21-cv-02098-MHC]

And because plaintiffs could not allege exposure of personal data or any other violation of statute or legal duty, the complaint was dismissed.

Now if Colonial had said it was good at cybersecurity, and then events suggested they were not in fact good at cybersecurity, they would definitely have drawn a few shareholder derivative suits and maybe even an SEC investigation. See Matt Levine (“everything is securities fraud”).

But there is no inherent duty to be good at cybersecurity. (Yet.)

https-ecf-gand-uscourts-gov-doc1-055114641298Download

Anonymization is hard, Waze edition

Security engineer Peter Gasper:

What I found is that I can ask Waze API for data on a location by sending my latitude and longitude coordinates. Except the essential traffic information, Waze also sends me coordinates of other drivers who are nearby. What caught my eyes was that identification numbers (ID) associated with the icons were not changing over time. I decided to track one driver and after some time she really appeared in a different place on the same road.

Waze: How I Tracked Your Mother (via Schneier on Security)

Anonymizing is hard

The task of proper anonymization is harder than it looks. Yet another example:

It turns out, though, that those redactions are possible to crack. That’s because the deposition—which you can read in full here—includes a complete alphabetized index of the redacted and unredacted words that appear in the document.

We Cracked the Redactions in the Ghislaine Maxwell Deposition (via Schneier on Security)

This seems to be a corollary of Schneier’s Law: Any person can anonymize data in a way that he or she can’t imagine breaking it.

Although the truth is most don’t even try to break their own work.