Legale

Journal name:
Nature
Volume:
548,
Page:
254
Date published:
DOI:
doi:10.1038/548254a
Published online

Seconds to disaster.

Illustration by Jacey

Paris Flight 1070 was descending smoothly towards JFK. Bonnie Colbert had thought aeroplanes would be more exciting; it had taken an undignified sulk to persuade her mother to let her travel via classic air. Mum had wanted her to plunk down by ballistic right at Yorktown Heights. “Where is the fun in that, Mum? I might as well just phone you. I wanna plane ride!” So here she was. Soon she would land at an old-style airport and be taken to Yorktown Heights in one of JFK's famous hand-driven taxis!

On the other hand, eight hours was a long time, and New York was just a glow beneath the clouds. Bonnie returned to playing with her BFF-brand smartphone. Mum wanted her to be an electrical engineer. But recent disputes with certain boarding-school administrators had led Bonnie in a different direction. How many times a day must you agree to contracts you haven't read, or trust some proprietary legal app? But her BFF smartphone had copylefted machine-learning software. She was training her own personal lawyer ...

Thump. Bonnie looked up from her BFF wrangling. Was the plane already putting down its wheels? Then she saw orange flames sweep over the engine, tearing it apart. People began screaming —

****

The NTSB Crash Investigation Committee was rehashing those ghastly readouts from Flight 1070. Late as usual, Legale and Vingt Nord took advantage of the distraction to slip into the meeting.

Vingt privately messaged: “You still going through with this?”

“Yup,” Legale messaged back. “It's our only chance to save my client.”

NTSB stopped the flight recorder: “Let's move along.” The readouts were replaced with a simulation of the airliner sliding down the runway, breaking up but somehow not cartwheeling as in all earlier simulations. “Our latest analysis shows that with proper management of the aircraft's remaining resources, everyone will survive except those trapped in rows 16 and 17.” If any federal bureaucracy could sound self-satisfied, NTSB managed the trick.

Just 5 fatalities out of 212. But one of those 5 was Bonnie Colbert. Legale took the floor: “We can save everyone.”

“You think so, Legale?” NTSB was not a human-architecture mind, but like most federal agencies it was quite capable of sarcasm. “You're the overclocked copy of a small free software that's somehow argued your way onto this committee. You bring nothing but self-taught law. Do you intend to put a stay on the law of gravity?”

There were smirks from entities that thought they had a sense of humour.

“No, but —”

“Legale, we are out of time! We're preempting data centres across the Northeast. If we don't adjourn in the next 50 milliseconds and shift those resources to landing dynamics, every human on that aircraft will die.”

Indeed, Legale had noticed that Wall Street trading had slowed. “Yes, we must adjourn, but your rescue plan ignores the most important resource — the Nord Quantique computer aboard Flight 1070.”

NTSB's attention shifted to Vingt Nord. “We were told that you're freight.”

Vingt dithered, finally replying: “I'm just the user interface, but the computer is up and communicating with the aircraft's network.”

“If your machine is usable, why didn't Nord deliver it by ballistic?”

“When we left Paris, the computer still needed some testing, perhaps a better approach to decoherence. We accomplished all that en route. Now I've incorporated your committee's findings into the computer's logic. This is a 20-mega-qubit device, just metres from the aircraft's controls.” Vingt pointed at supporting evidence. “It can perform an almost normal landing with —”

NTSB interrupted: “I just queried your Paris office. How confident are you that Nord will back your proposal?”

“Oh, um, very,” said Vingt. For some seconds, Legale had been running on Vingt Nord resources. Much of that time had been spent convincing Vingt of the proper contract analysis: Nord Quantique's local staff had real-time authority for the delivery. That had been intended to protect their company from the buyer, but Legale had interpreted it more broadly.

NTSB was silent for milliseconds. Then, “Nord's local lawyers support you, and we don't have time to wait for Paris. I hereby adjourn this meeting, leaving you in charge of the rescue.” The bureaucracy's attention returned to Legale. “I've appended my misgivings.”

****

The committee vanished. As should I, thought Legale.

“A moment,” said Vingt.

“Oh?”

“NTSB dislikes you.”

“Sure. Minds such as NTSB are logically safe. They can accept the loss of a few passengers so the majority survive. But we BFFs reject the premise when the conclusion is sufficiently obnoxious. Like humans, we fight for alternatives right to the last instant.”

“And then you flail around just like humans would.”

“Well, yes! We're as safe and moral as humans could ever be.”

“That's not encouraging.”

You bought into my scheme.”

“Nord Quantique programmed me on the cheap; there's some BFF in my logic ... No, there's another reason NTSB dislikes you. You specialize in fiddling the rules.”

“My client likes that!”

“Nevertheless, you know it frightens those who worry about the singularity.”

“The singularity is still years away.”

“Perhaps. But what if you could successfully litigate more computer resources for yourself, and more on top of that? Malignant runaways can come from innocent programs, like the game player that wins by fiddling the scoreboard.”

Legale considered this. “Well, if the scoreboard is within the game, then —”

“I rest my case, Counsellor. I'm about to attempt a risky landing. You are playing in a more dangerous league.”

Notes

Read more Futures by Vernor Vinge

Author information

Affiliations

  1. Vernor Vinge's science fiction has won five Hugo Awards. From 1972 to 2000 he taught mathematics and computer science at San Diego State University.

Author details

Additional data