Moral Programs

Nobody sued musicians for their album art on the screen of handhelds, clenched vicelike in the hands of recent suicides who opted not to drown. Statistical artefacts, classified under “contributory factors”, deemed immune to blame. The same should be for videogames.

But what if they directly implied the player didn’t exist?

Meta. Extremely meta.

The premise was this; upon opening the game, players where given a heavy dose of expository spiel. They were informed that they were “Moral Programs”. Philosophical AIs with enough complexity and electrical similarity to brains to deem them human enough on the Turing scales to be passable as the real thing. Machines which felt. Sufficiently. The player’s life up until launching the game was described as a subroutine. Something to give the AI a sense of reality and weight.

Players laughed. Quaint. If they shut the game prematurely, to test the supposed depth of the game, they were simply bought back to the main screen on reloading. Failure also took the player back to the start, a difficulty just hard enough to activate primal hunting instincts and a desire to try again. Nothing special, but enough to entice playing further.

The gameplay consisted of simple masked optimization problems. “You” accompanied a drone, which was actively seeking foreign targets in a foreign land. The player’s goal was essentially to observe, their humanlike presence in the conflict a necessity only by international law. Occasionally, players were instructed to decide which targets were more “ethically challenging”. Advice the drone to run the risk of killing six children, or three young pregnant women? Two criminals or one innocent civil servant? A teenagers arm and ears, or both his legs? “Missions” were evaluated at their conclusion at a military tribunal, where the player/Moral Program were analysed and its ethical combat decision-making commended. Failure resorted in a reset. Infuriating. Try again.

It was curious complexity. People talked about it to their friends on and off-line, as it warped and spread across the net. Watching other people play the game wasn’t as satisfying as playing it yourself, but the fans still crowded to their streams. Reviewers lapped up its intricacy in a genre with dying thematic principles. Overall it was being well received.

Now things become most complicated at the very end of the game. The player/Moral Program was constantly being reminded by their in-game commander of their lack of existence, and how “everything else” (being, everything but the game), was a minor script in their programming to ground them in reality. The façade grew and grew. The player’s/Moral Program’s family were mentioned, in vague enough terms to catch the attention of a wide audience. Probing into the open-source freedom of social media, the game occasionally pulled friends and lovers names out the blue to scare the player/Moral Program. The game even describes these processes within itself, bathing in the meta-conversations which acknowledge axioms themselves, lathering the audience with the triviality of a known illusion good enough to keep you fooled. The game knew it was a game. But did the player?

The ending itself was what finally drew the most blame. The final “mission” was to retrieve a package for the government the player/Moral Program served. It was cutting edge, and exceedingly hard for the player to morally retrieve. Civilians and double agents crowded the battlefield. A second drone entered play. A hospital was conveniently in the line of fire. Neurones fired faster along their new muscle memory than they had before. Breath was held. A satisfying boss fight.

The cutscene which followed hammered the final nail. The package was revealed as an artificial “heaven”. The commander described, in a speech clearly designed for repetition to many audiences, that the Moral Programs were too humanlike in their complexity to be simply shutdown like the unconscious drone AIs. Instead they had to be stored in “heavens” until they lived a standard human life. Then they could die like the rest of us. The game ends with the commander thanking the player/Moral Program for their work, and says their coding will be recommended to their superior.

And that was that. Replaying the game was identical, and the command simply explained it was just another facet of heaven to entertain the player. Players shut down their PCs, went for a drink, and then moved onto the next serial FPS. But fans don’t die hard.

All it took was 0.001% of the players which deemed themselves “fans” to post their datamine on a forum and the secrets were out. Paragraphs of additional dialogue, deemed “unfit” for the game’s final build. The commander’s speeches were tedious, explaining the game was the only way within the law to use Moral Programs, and his own concerns that the AIs would become too aware of their false lives and commit suicide. He went on, describing how only a handful of Moral Programs were even used in the conflict, and that the rest of the “player base”, and “fans” of the game were just subroutines of subroutines. Other characters, who helped designed the program were hinted at in a spiders web of intrigue. The commander in one passage even seemed to be close to a breakdown. It was very well done.

Of course there were sceptics. Or rather, realists. It was all made up. Obviously it was all made up. But still very very well done. The thematic praise actually won the game a few awards, which the sole creator took sullenly and with a barely managed fake smile. It was all made up!

But the brain in a vat stipulation evolved into the Moral Program on a chip stipulation. The game was even brought up at a low-level philosophy conference at an equally low level university. Even professionals however conceded it was a good way to get kids interested in philosophy. A year after release, having made a tidy sum, the one-man development team retired, from videogames and public life. People began to forget about Moral Program.

Then, in the short space of a month, about eleven teenagers killed themselves. So what. Well, after two more months and another fifteen later connected suicides, it was agreed that all the dead, along with a slew of previously ignored suicides from months ago, were massive fans of the game. They had watched stream after stream, done play-through after play-through. Most datamined, with all having joined in lengthy forum arguments for weeks. One or two even had merchandising. After a further four months, the suicides reach fifty. At this point, the parents of the initial dead were frothing at the mouth. The illusive developer, until now declining comment, was hauled in by the police. An eight-year-old, left at home with nothing but his laptop and a deep desire for attention from his arguably as much to blame parents, walked in front of a bus while playing the games theme-tune on his handheld. That got the rest of the general public involved and caring.

Another slew of suicides and game faced a mass-recall. As the one-hundred mark was reached, computers connected to the internet uninstalled the game automatically, citing “mental health risks”. But somebody cracked the thing open and made it run offline undetected. All it took was a trip to the ‘Bay and the entire experience was yours. Forums which had covertly inadvertely been promoting the suicide pact were also shutdown superficially, but all it took was real devotion and you’d find them again. Parents, family and friends became more and more fearful. The “Moral-Program” effect littered the news, blaming any game it could. Previously laid to rest court cases were dug up. Irate mothers threw their bawling preteens consoles in the outside bin.

Then one self-seen Assange managed to leak the footage from the final stage of the developer’s trial. The creator had been indicted for months, as the death toll mounted and mounted, until it finally begun to recede two years after release, with him facing a lengthy sentence. Hearings and evidence mounted, witnesses went from screaming to sobbing. Hate crimes were committed against the developer’s innocent parents, who eventually began to condemn their son as much as the parents of “those he’d killed”. The general public grew tired eventually, as the trial drew to a close. But the footage was too enjoyable, and it sparked an aftershock unlike anything before.

He was quite deranged, everyone agreed. The developer had gone without any real attorney or defensive lawmaker whatsoever. He’d just sat sullenly, confirming what was true and denying lies eventually deemed to be false as well. All sane. All until the closing statement. He’d certainly been thinking about it for a while. And he certainly knew how to a capture a crowds attention.

He erupted, fervour unseen before in the courtroom outside of the grieving parents. He condemned not the courts, but all videogames themselves. Flirtatious combat simulations, which toyed with the brain just enough to make it feel like it was actually doing anything, tricking the matters into releasing joyful hormones for nothing. He laughed and ran his hands through his hair, announcing he was almost glad at the suicides, deeming the dead’s brains too susceptible to false trickery and manipulation. He referenced a utopian apocalyptic future, where we put our brains in the very “heaven” machines he’d hypothesised, seeking pleasure over any tangible impact on reality. The machines would rule us then, he said, while we giggled child-like in our self-made illusions. More laugher, more tears. He eventually devolved, and was dragged out the courtroom ranting about mankind’s inability to handle anything short of ultimate pleasure, given a choice. The last footage the leak captured was his biting a guard’s hand, manic eyes scanning a stunned jury, while screaming inhumanly.

“They’ll kill our will, our impact! The fucking games will kill us all!”

Trawling the man’s hard-drives turned up similar rants. He’d even written a short story, describing almost an identical scenario, which when cross-referencing his final screams in the story and reality was scarily accurate,  especially given how he’d been separate from the computer for months.

The world was quite shaken. At least for a few months. Then it went back to caring about only the prettier endangered animals, and trivialities of celebrity. And its video games, of course.