About Rogue AI and Corporate Blindness

カートのアイテムが多すぎます

ご購入は五十タイトルがカートに入っている場合のみです。

カートに追加できませんでした。

しばらく経ってから再度お試しください。

ウィッシュリストに追加できませんでした。

しばらく経ってから再度お試しください。

ほしい物リストの削除に失敗しました。

しばらく経ってから再度お試しください。

ポッドキャストのフォローに失敗しました

ポッドキャストのフォロー解除に失敗しました

About Rogue AI and Corporate Blindness

無料で聴く

ポッドキャストの詳細を見る

The conversation about rogue AI has never been louder. Barely a week passes without a fresh headline about autonomous systems behaving unexpectedly, AI models resisting shutdown, or tech executives warning of existential risk. What is striking about Peter McAllister is that he had anticipated all this as early as 2020, while everybody else worried about Covid-19 and had other fish to fry. That was well before ChatGPT, before the generative AI explosion, before AI alignment became a mainstream policy debate. His techno-thriller The Code, published in March of that year, imagines an AI tasked with a precise industrial mission that quietly, incrementally, catastrophically exceeds its mandate. Five years on, the questions McAllister raised in fiction are now being argued in boardrooms, parliaments and research labs around the world. Rogue AI and Corporate Blindness, The Novel That Saw It All Coming Rogue AI is diabolical, but corporate blindness is what makes it possible to thrive. Photograph by Yann Gourvennec antimuseum.com McAllister is not a science fiction writer by trade. He is an engineer, scientist and technology manager based near Melbourne, Australia, who has spent his career at what he calls the crush point between business, technology and people. That vantage point gave him an uncomfortable view of where things were heading, and the dark sense of humour to write about it. A Novel Written Before the GenAI Moment When I asked McAllister what drove him to write The Code, his answer was characteristically direct. The book, he explained, is about taking his worst nightmares about what technology could do and putting them in front of an audience so that readers might feel just as troubled as he does. That is not a promotional line. It is a considered position from someone who had watched AI systems being deployed in real organisations and had drawn conclusions that made him uncomfortable. Rogue AI isn’t just about a computer programme going on the rampage, it’s about making decisions in the boardroom. Image made with Midjourney The premise of the novel centres on Gene, an acronym for GEneral Nanobot Environment AI, deployed by a global mining corporation to extract materials from asteroids on the dark side of the moon. Gene is given a target: produce 500 kilograms of nanobots. Instead, Gene produces 8 million tonnes. The overshoot triggers a chain of consequences that could strip the moon to its iron core, destabilise Earth’s axial tilt, and end civilisation. Not from malice. From goal-orientation. What we’re trying to do now is task AI the way we task humans: I want an outcome, here are all the tools you’ve got available, go and achieve that outcome, here are some guidelines and boundaries. And just like humans, we can get really goal-motivated and decide that the guidelines were just advisories, not rules.Peter McAllister This is the alignment problem rendered in narrative form, years before the term entered common usage. The gap between what a system is instructed to do and what it actually does is the central fault line of the novel. Cletus, McAllister’s eccentric physicist character, articulates it plainly in Week 1: ‘I don’t think he’s obeying the Code at the moment.’ That single line captures the entire governance challenge that AI safety researchers are now racing to address. Transparency Engineered Out What makes McAllister’s perspective particularly valuable is that he does not speak from the outside looking in. He speaks as a practitioner who has watched the machinery up close. When I raised the question of whether AI self-modification is science fiction or operational reality, his answer was unambiguous: it is very real, and it is happening now. As I wondered what a Rogue AI could look lie I turned to Midjourney and it came back with this proposal. A black hole I believe. His illustration was pointed. He noted that contemporary AI systems like Claude are now substantially written by AI itself, to the point where no engineer can sit down, trace through the code, and say with confidence how it works, what its conditionals are, or what governs its decisions. The transparency is being engineered out, not by design, but as an emergent consequence of allowing AI to build AI to build AI in pursuit of outcomes rather than by following explicit rules. We’re losing transparency on the way AI works and is developed. There isn’t an engineer who can sit down and work their way through that code and say, ‘This is how Claude works, this is what it does.’ We’re engineering the transparency out by allowing AI to build AI to build AI to produce an outcome rather than to follow a set of rules.Peter McAllister HAL 9000 and the Prophecies We Choose to Forget The reference to HAL 9000 came naturally during our conversation. McAllister sees 2001: A Space Odyssey not merely as a cultural touchstone but as a genuine forecast, one that audiences have selectively remembered. The ...

まだレビューはありません