Bad software kills 346 people

That’s a fair headline for the story that has ultimately emerged about the Boeing 737-MAX crashes.

The Verge has a good overview:

But Boeing’s software shortcut had a serious problem. Under certain circumstances, it activated erroneously, sending the airplane into an infinite loop of nose-dives. Unless the pilots can, in under four seconds, correctly diagnose the error, throw a specific emergency switch, and start recovery maneuvers, they will lose control of the airplane and crash — which is exactly what happened in the case of Lion Air Flight 610 and Ethiopian Airlines Flight 302.

THE ANCIENT COMPUTERS IN THE BOEING 737 MAX ARE HOLDING UP A FIX

I once linked to a story about how no one really cares about software security because no one ever gets seriously hurt. This is a hell of a counterpoint, though admittedly a narrow one.

Ode to Routine

Mason Currey, writing for The Atlantic:

In an 1892 lecture, William James laid out his idea of perfect unhappiness. “There is no more miserable human being,” he said, “than one in whom nothing is habitual but indecision, and for whom the lighting of every cigar, the drinking of every cup, the time of rising and going to bed every day, and the beginning of every bit of work are subjects of express volitional deliberation.” Now that social-distancing measures have been adopted worldwide in response to the coronavirus pandemic, many people are suddenly finding themselves in the position that James so dreaded. Long-established routines are being swept away faster than cartons of shelf-stable almond milk at my local Sprouts. Whole sections of the day that previously ran on blissful autopilot now require conscious decision making and the reluctant hand cranking of dusty willpower. 

The Routines That Keep Us Sane

A charming essay about finding structure (or not) in the midst of a storm.

Privacy vs. the Coronavirus

Everywhere and at all at once rules around privacy are being relaxed in the face of urgent public health concerns:

As countries around the world race to contain the pandemic, many are deploying digital surveillance tools as a means to exert social control, even turning security agency technologies on their own civilians. Health and law enforcement authorities are understandably eager to employ every tool at their disposal to try to hinder the virus — even as the surveillance efforts threaten to alter the precarious balance between public safety and personal privacy on a global scale.

As Coronavirus Surveillance Escalates, Personal Privacy Plummets

Meanwhile, global data privacy regulators are confident that “data protection requirements will not stop the critical sharing of information to support efforts to tackle this global pandemic.”

In the hierarchy of human needs, security always has and always will come first.

Privacy Optimism

Ben Garfinkel, a research fellow at Oxford University, writes about the difference between social privacy (what your intimates and acquaintances know about you) and institutional privacy (what governments and corporations know about you):

How about the net effect of these two trends? Have the past couple hundred years of change, overall, constituted decline or progress?

. . . . .

My personal guess is that, for most people in most places, the past couple hundred years of changes in individual privacy have mainly constituted progress. I think that most people would not sacrifice their social privacy for the sake of greater institutional privacy. I think this is especially true in countries like the US, where there are both high levels of development and comparatively strong constraints on institutional behavior. I think that if we focus on just the past thirty years, which have seen the rise of the internet, the situation is somewhat more ambiguous. But I’m at least tentatively inclined to think that most people have experienced an overall gain.

The Case for Privacy Optimism

And overall he concludes that he is optimistic about privacy trends, particularly because of artificial intelligence:

The existence of MPC [Multi-Party Computation] protocols implies that, in principle, training an AI system does not require collecting or in any way accessing the data used to train it. Likewise, in principle, applying a trained AI system to an input does not require access to this input or even to the system’s output.

The implication, then, is this: Insofar as an institution can automate the tasks that its members perform by training AI systems to perform them instead, and insofar as the institution can carry out the relevant computations using MPC, then in the limit the institution does not need to collect any information about the people it serves.

This view, which of course assumes quite a bit of technology, is both plausible and consistent with a number of other researchers who view AI technology as being a potential improvement on our ability to manage human bias and privacy intrusions.

I also tend to believe the glass is half full. That’s my own bias.

AI researchers submitting to NeurIPS conference must now address ethical concerns

Khari Johnson, writing for Venture Beat:

For the first time ever, researchers who submit papers to NeurIPS, one of the biggest AI research conferences in the world, must now state the “potential broader impact of their work” on society as well as any financial conflict of interest, conference organizers told VentureBeat.

NeurIPS requires AI researchers to account for societal impact and financial conflicts of interest

NeurIPS, or the Conference on Neural Information Processing Systems, is the largest AI conference in the world.

NLP models keep getting bigger and better

Microsoft Research has trained a transformer-based generative language model with over 17 billion parameters. And it performs very well, answering many natural language questions directly:

Turing-NLG: A 17-billion-parameter language model by Microsoft

What is the right size? How much bigger can they get? We are rapidly approaching human levels of natural language performance (but not comprehension).

Summary of EARN IT Act of 2019

Senator Lindsey Graham has introduced the EARN IT Act of 2019, which would eliminate online service providers’ immunity for the actions of their users under Section 230 of the Communications Decency Act.

The Act essentially establishes a National Commission on Online Child Exploitation Prevention, tasks this commission with drafting online best practices for preventing child exploitation by users (which would presumably mean no end-to-end encryption), and eliminates Section 230 immunity unless service providers follow those best practices.

SAFE HARBOR.—Subparagraph (A) [removing immunity] shall not apply to a claim in a civil action or charge in a criminal prosecution brought against a provider of an interactive computer service if – (i) the provider has implemented reasonable measures relating to the matters described in section 4(a)(2) [referring to creation of the best practices] of the Eliminating Abusive and Rampant Neglect of Interactive Technologies Act of 2019 to prevent the use of the interactive computer service for the exploitation of minors . . . .

Page 17 of the EARN IT Act of 2019

Other sections create liability for “reckless” violations (instead of “knowing” violations), require online service providers to certify that they are complying with the created best practices, and set forth the requirements for membership in the newly created commission.

This bill comes after a hearing in December 2019 over the issue of legal access to encrypted devices. During that hearing Senator Graham warned representatives of Facebook and Apple that, “You’re gonna find a way to do this or we’re going to do it for you.”

3/15/20 Update – A revised version of the EARN IT Act, introduced on March 5, alters how so-called “best practices” are created. First, a 19-member commission comprising the Attorney General, the Secretary of Homeland Security, the Chairman of the FTC, and (to be chosen by the heads of each party in the House and Senate) four representatives from law enforcement, four from the community of child-exploitation victims, two legal experts, two technology experts, and four representatives from technology companies. The support of 14 members would be required to approve any best practices, the recommendations must be approved by the AG, Secretary of Homeland Security, and the FTC Chair, and then Congress itself must enact them.

Facial recognition tech in Moscow

First London, now Moscow.

Moscow is the latest major city to introduce live facial recognition cameras to its streets, with Mayor Sergei Sobyanin announcing that the technology is operating “on a mass scale” earlier this month, according to a report from Russian business paper Vedomosti.

. . . . .

Moscow started trialing live facial recognition in 2017, using technology from Russian firm NtechLab to scan footage from the Russian capital’s network of 160,000 CCTV cameras. The company is best known for its FindFace software, which it launched in 2016 and let users match anyone in a picture to their profile on VK, known as Russia’s Facebook.

The app was criticized by some, particularly as it was used to dox and harass sex workers, and NtechLab eventually shut it down in favor of enterprise and government work.

Moscow rolls out live facial recognition system with an app to alert police

Prediction vs thought in GPT-2

GPT-2 is a neural network model originally developed by OpenAI and based on Google’s Transformer model. It has demonstrated impressive, even fantastic performance, on a variety of language metrics. But is it in any way embodying real intelligence? The Gradient’s Gary Marcus says no.

Here’s the problem: upon careful inspection, it becomes apparent the system has no idea what it is talking about: it cannot follow simple sequence of events nor reliably have any idea what might happen next.

GPT-2 and the Nature of Intelligence

Marcus cites some impressive prediction results, such as the following where the GPT-2 predicted text follows the bolded part:

I grew up in Rome I speak fluent Italian and can use Italian throughout my notes.

I grew up in Rio de Janeiro. I speak fluent Portuguese, Spanish and many of the local languages.

I grew up in Shanghai. I speak fluent Mandarin, and I have learned a lot of Chinese, so I really don’t have any issues with learning Chinese

Nevertheless, GPT-2 is unable to follow basic causality:

Suppose for example that I ask the system what happens when I put two trophies a table and another:

I put two trophies on a table, and then add another, the total number is is five trophies and I’m like, ‘Well, I can live with that, right?

What happens if I have four plates and put one cookie on each?

I have four plates and put one cookie on each. The total number of cookies is 24, 5 as a topping and 2 as the filling

Marcus suggests that these results support the view that GPT-2 is simply very good at predicting which words will follow other words. It does not actually understand what it is predicting. And this failure to achieve any kind of understanding despite a huge training set of 40 gigabytes of text suggests that true language understanding might not be the result of empirical experience or statistics after all.

One of the most foundational claims of Chomskyan linguistics has been that sentences are represented as tree structures, and that children were born knowing (unconsciously) that sentences should be represented by means of such trees. Every linguistic class in the 1980’s and 1990s was filled with analyses of syntactic tree structures; GPT-2 has none.

. . . . .

Rather than supporting the Lockean, blank-slate view, GPT-2 appears to be an accidental counter-evidence to that view. Likewise, it doesn’t seem like great news for the symbol-free thought-vector view, either. Vector-based systems like GPT-2 can predict word categories, but they don’t really embody thoughts in a reliable enough way to be useful.

If GPT-2 is fundamentally prediction, and prediction is fundamentally not understanding, how far will this road take us? Would we ever be able to rely on a GPT-2-like model for critical tasks? It may be that common sense is always just out of reach.