AI Round-Up 03

George Walkley:

the Canadian ISBN Agency now requires a declaration that works were created by humans or with de minimis AI assistance. AI-generated works are not eligible for Canadian ISBNs.

This is a new layer in a stack of practical questions for AI-generated work: is it copyrightable, is it identifiable with an ISBN, does it comply with retailer policies, and how will readers react? Of course, other national ISBN agencies may take different approaches—a national library, as in the Canadian example, can gatekeep to an extent a commercial entity like Bowker or Nielsen might not—and many self-published authors don’t assign ISBNs at all, especially if they’re really only working within KDP. I suspect that, outside Canada, this is most interesting as a precedent and challenge for other bibliographic providers.

George writes the newsletter Outside Context and has a ton of info and insight into how AI is changing the publishing industry (and, by association, other media industries).


Cory Doctorow is a cranky Marxist, which makes him fun to read. He also occasionally makes good points:

Out of all the things I learned about publishing from Patrick, one truth stands out above all: publishing is “the act of making something public.” Publishing may or may not involve editing, layout, printing, binding, marketing or other activities that we associate with publishing houses, but publishing always involves identifying a work, identifying an audience, and finding ways to connect that work to that audience.

[…]

For another, this definition lets us understand that (for most of its existence) Google was the most successful publisher in human history. No other institution of any kind connected more works with more audiences. Lots of people have heard of Google’s wildly unsuccessful motto “Don’t be evil,” but far fewer are familiar with Google’s wildly successful mission statement, “organize the world’s information and make it universally accessible and useful.”

But all of that is in the past tense. With its AI projects – AI summaries, the Gemini chatbot – Google has abandoned the practice of “connecting works with audiences.” Today, Google’s function is to plunder all the works on the internet and chop them up and arrange them in an unimaginably vast, multidimensional array for the explicit purpose of preventing audiences from connecting with those works.


Also, Cory’s explanation of what happened during the Hugging Face attack is the most cogent I’ve read. If you have even the tiniest heart palpitation when you think about that event, take a look.

“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.


Benj Edwards on how AI makes it possible for everyone to have completely personalized entertainment:

I love my AI-translated songs, but my kids don’t. In fact, it’s unlikely anyone else on earth will ever want to listen to these recordings. Sure, I wrote the music behind them. But since they’ve been fed into the AI maw, it feels like they are missing a human touch or connection that might make them land with others. And yet I have had more fun listening to my own songs than I ever expected. I am an audience of one.

To be fair, I’ve been playing Benj’s game Violent Checkers for several weeks now and am still enjoying it.

AI Round-Up 02

The Auschwitz Memorial Museum posted a statement decrying the use of AI by a Facebook page called “90’s History” to create fictional images of Holocaust victims based on actual photographs. In an appeal to the page’s creators, the Museum’s staff said:

Stop this practice immediately. If your intent is to educate and honor the memory of Auschwitz victims, do so with integrity. Fabricating faces with AI does not preserve memory. It reshapes it, distorts it, and risks turning tragedy into aestheticized fiction. The tragedy of Auschwitz does not need to be made more visual, more “emotional,” or more “shareable.” It needs to be remembered truthfully.


Cory Doctorow insightfully ties the sociological concept of “placelessness” to AI model collapse:

“Placelessness” is the urbanist’s name for “when everywhere optimises toward the same template.” I think of it as Flinstones Syndrome, where the same background is looped behind Fred and Barney as they drive through Bedrock. In New York City, it’s Citibank-bodega-Chipotle-Walgreens; in the Chicago suburbs, it’s the strip malls with a Chili’s, a gas station, and a big box store.

Leek proposes that these are all expressions of the same underlying phenomenon, a failure mode of data science that takes a world of “granular personal data” and arrives at a world where “personalisation produc[es] more sameness.”


There’s always much to chew on in Robin Sloan’s newsletter (which waxes every 29 1/2 days, like the moon).


Meta announced that its Muse agent has 10 million users, of whom 25% are using it daily. Shopping is the biggest use case.

Source


Verity Harding comments on election results in her newsletter, AI Needs You:

It’s hard to separate whether opposition to data centres is about the buildout itself, or the fact that they are one of the few tangible things about AI that people feel they can control – and even stop. Even though it is widely used, the technology itself is hugely unpopular in the United States (especially with Gen Z), with voters citing fears that AI will undermine societal structures, replace human interaction, and take away their jobs. In his acceptance speech, Georgia senator (and potential 2028 Democratic hopeful) Jon Ossoff told his supporters: “I refuse to accept [that my daughters] will grow up with artificial friends and teachers”. A wider sense of AI as representative of what people dislike and fear about modern society has taken hold in a way that will be tough to reverse anytime soon.

AI Round-Up 01

I read a lot of articles about AI, some technical, but most in the realm of commentary. I’m going to use these Round-Ups to share what I’ve been reading. I might comment. I might not. Most of the time, the bolded text is my emphasis.


A fascinating article about the role of AI in mathematics. Essentially, AI reveals that mathematicians don’t study math to find answers. They study math to find the path to those answers. An AI that gives you the answer is taking away the key component of mathematical studies.

Arguably, theorem proving and problem solving aren’t even the most important goals for mathematicians. In a famous 1994 essay, the mathematician William Thurston argued that what mathematicians are doing “is finding ways for people to understand and think about mathematics,” especially as members of a social community. Thurston gave an example from his own life. Early in his career, he quickly proved a string of “dramatic theorems” in an area of mathematics called foliations. However, because he was so successful at proving the theorems — and significantly less successful at communicating the ideas behind his proofs — other mathematicians evacuated the field. The end result was that the social structure that had supported research into foliations collapsed and the subfield died. “I had the conception that what people wanted was to know the answers,” Thurston wrote. “That’s only one part of the story. More than the knowledge, people want personal understanding.”


From a Princeton study about the “delusion-like states” facilitated by AI, quoted here by Gary Marcus:

Unlike hallucinations, which introduce false-hoods, sycophancy is a bias in the selection of the data people see. When AI systems are trained to be helpful, they may inadvertently prioritize data that validates the user’s narrative over data that gets them closer to the truth.


Woodrow Hartzog and Jessica Silbey, quoted at length here by Gary Marcus:

In this Article, we hope to convince you of one simple and urgent point: the current design of artificial intelligence systems facilitates the degradation and destruction of our critical civic institutions. Even if predictive and generative AI systems are not directly used to eradicate these institutions, AI systems by their nature weaken the institutions to the point of enfeeblement. To clarify, we are not arguing that AI is a neutral or general purpose tool that can be used to destroy these institutions. Rather, we are arguing that AI’s current core functionality—that is, if it is used according to its design—will progressively exact a toll upon the institutions that support modern democratic life. The more AI is deployed in our existing economic and social systems, the more the institutions will become ossified and delegitimized. Regardless of whether tech companies intend this destruction, the key attributes of AI systems are anathema to the kind of cooperation, transparency, accountability, and evolution that give vital institutions their purpose and sustainability. In short, AI systems are a death sentence for civic institutions, and we should treat them as such.


Here’s Cory Doctorow on “cheap AI,” useful bits of tech that might be left after the AI bubble pops:

Here’s hardware hacker Pete Warden demoing a chatbot that you talk to and that talks back to you – and it’s running on Synaptics System-on-a-Chip (SoC) that costs “low single digit dollars”:

https://petewarden.com/2025/10/16/why-does-a-local-ai-voice-agent-running-on-a-super-cheap-soc-matter/

This is basically a little special-purpose Alexa, except it doesn’t connect to the internet at all (and therefore doesn’t leak any of your data). In Warden’s demo, the gadget is a button-sized voice assistant that is meant to be integrated into a dishwasher, which can interpret the dishwasher’s manual for you. If your dishes come out dirty or if the drain gets clogged, you press the button, describe your issue in pretty vague terms, and it instantly speaks aloud all the troubleshooting steps to deal with it. This privacy-preserving, cheap-like-borscht component adds a voice-activated, conversational assistant to a device, sipping power like the clock on your microwave, running on a processor that costs less than a pack of AA batteries. It’s seriously f****** cool. There’s going to be a lot of this AI, after the AI goes away – just like there was a lot of the web after the dotcom crash, when, overnight, San Francisco had infinity office-space, servers, and techies going begging.