AI Round-Up 03

George Walkley:

the Canadian ISBN Agency now requires a declaration that works were created by humans or with de minimis AI assistance. AI-generated works are not eligible for Canadian ISBNs.

This is a new layer in a stack of practical questions for AI-generated work: is it copyrightable, is it identifiable with an ISBN, does it comply with retailer policies, and how will readers react? Of course, other national ISBN agencies may take different approaches—a national library, as in the Canadian example, can gatekeep to an extent a commercial entity like Bowker or Nielsen might not—and many self-published authors don’t assign ISBNs at all, especially if they’re really only working within KDP. I suspect that, outside Canada, this is most interesting as a precedent and challenge for other bibliographic providers.

George writes the newsletter Outside Context and has a ton of info and insight into how AI is changing the publishing industry (and, by association, other media industries).

Cory Doctorow is a cranky Marxist, which makes him fun to read. He also occasionally makes good points:

Out of all the things I learned about publishing from Patrick, one truth stands out above all: publishing is “the act of making something public.” Publishing may or may not involve editing, layout, printing, binding, marketing or other activities that we associate with publishing houses, but publishing always involves identifying a work, identifying an audience, and finding ways to connect that work to that audience.

[…]

For another, this definition lets us understand that (for most of its existence) Google was the most successful publisher in human history. No other institution of any kind connected more works with more audiences. Lots of people have heard of Google’s wildly unsuccessful motto “Don’t be evil,” but far fewer are familiar with Google’s wildly successful mission statement, “organize the world’s information and make it universally accessible and useful.”

But all of that is in the past tense. With its AI projects – AI summaries, the Gemini chatbot – Google has abandoned the practice of “connecting works with audiences.” Today, Google’s function is to plunder all the works on the internet and chop them up and arrange them in an unimaginably vast, multidimensional array for the explicit purpose of preventing audiences from connecting with those works.

Also, Cory’s explanation of what happened during the Hugging Face attack is the most cogent I’ve read. If you have even the tiniest heart palpitation when you think about that event, take a look.

“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.

Benj Edwards on how AI makes it possible for everyone to have completely personalized entertainment:

I love my AI-translated songs, but my kids don’t. In fact, it’s unlikely anyone else on earth will ever want to listen to these recordings. Sure, I wrote the music behind them. But since they’ve been fed into the AI maw, it feels like they are missing a human touch or connection that might make them land with others. And yet I have had more fun listening to my own songs than I ever expected. I am an audience of one.

To be fair, I’ve been playing Benj’s game Violent Checkers for several weeks now and am still enjoying it.

AI Round-Up 02

The Auschwitz Memorial Museum posted a statement decrying the use of AI by a Facebook page called “90’s History” to create fictional images of Holocaust victims based on actual photographs. In an appeal to the page’s creators, the Museum’s staff said:

Stop this practice immediately. If your intent is to educate and honor the memory of Auschwitz victims, do so with integrity. Fabricating faces with AI does not preserve memory. It reshapes it, distorts it, and risks turning tragedy into aestheticized fiction. The tragedy of Auschwitz does not need to be made more visual, more “emotional,” or more “shareable.” It needs to be remembered truthfully.

Cory Doctorow insightfully ties the sociological concept of “placelessness” to AI model collapse:

“Placelessness” is the urbanist’s name for “when everywhere optimises toward the same template.” I think of it as Flinstones Syndrome, where the same background is looped behind Fred and Barney as they drive through Bedrock. In New York City, it’s Citibank-bodega-Chipotle-Walgreens; in the Chicago suburbs, it’s the strip malls with a Chili’s, a gas station, and a big box store.

Leek proposes that these are all expressions of the same underlying phenomenon, a failure mode of data science that takes a world of “granular personal data” and arrives at a world where “personalisation produc[es] more sameness.”

There’s always much to chew on in Robin Sloan’s newsletter (which waxes every 29 1/2 days, like the moon).

Meta announced that its Muse agent has 10 million users, of whom 25% are using it daily. Shopping is the biggest use case.

Source

Verity Harding comments on election results in her newsletter, AI Needs You:

It’s hard to separate whether opposition to data centres is about the buildout itself, or the fact that they are one of the few tangible things about AI that people feel they can control – and even stop. Even though it is widely used, the technology itself is hugely unpopular in the United States (especially with Gen Z), with voters citing fears that AI will undermine societal structures, replace human interaction, and take away their jobs. In his acceptance speech, Georgia senator (and potential 2028 Democratic hopeful) Jon Ossoff told his supporters: “I refuse to accept [that my daughters] will grow up with artificial friends and teachers”. A wider sense of AI as representative of what people dislike and fear about modern society has taken hold in a way that will be tough to reverse anytime soon.

AI Round-Up 01

I read a lot of articles about AI, some technical, but most in the realm of commentary. I’m going to use these Round-Ups to share what I’ve been reading. I might comment. I might not. Most of the time, the bolded text is my emphasis.

A fascinating article about the role of AI in mathematics. Essentially, AI reveals that mathematicians don’t study math to find answers. They study math to find the path to those answers. An AI that gives you the answer is taking away the key component of mathematical studies.

Arguably, theorem proving and problem solving aren’t even the most important goals for mathematicians. In a famous 1994 essay, the mathematician William Thurston argued that what mathematicians are doing “is finding ways for people to understand and think about mathematics,” especially as members of a social community. Thurston gave an example from his own life. Early in his career, he quickly proved a string of “dramatic theorems” in an area of mathematics called foliations. However, because he was so successful at proving the theorems — and significantly less successful at communicating the ideas behind his proofs — other mathematicians evacuated the field. The end result was that the social structure that had supported research into foliations collapsed and the subfield died. “I had the conception that what people wanted was to know the answers,” Thurston wrote. “That’s only one part of the story. More than the knowledge, people want personal understanding.”

From a Princeton study about the “delusion-like states” facilitated by AI, quoted here by Gary Marcus:

Unlike hallucinations, which introduce false-hoods, sycophancy is a bias in the selection of the data people see. When AI systems are trained to be helpful, they may inadvertently prioritize data that validates the user’s narrative over data that gets them closer to the truth.

Woodrow Hartzog and Jessica Silbey, quoted at length here by Gary Marcus:

In this Article, we hope to convince you of one simple and urgent point: the current design of artificial intelligence systems facilitates the degradation and destruction of our critical civic institutions. Even if predictive and generative AI systems are not directly used to eradicate these institutions, AI systems by their nature weaken the institutions to the point of enfeeblement. To clarify, we are not arguing that AI is a neutral or general purpose tool that can be used to destroy these institutions. Rather, we are arguing that AI’s current core functionality—that is, if it is used according to its design—will progressively exact a toll upon the institutions that support modern democratic life. The more AI is deployed in our existing economic and social systems, the more the institutions will become ossified and delegitimized. Regardless of whether tech companies intend this destruction, the key attributes of AI systems are anathema to the kind of cooperation, transparency, accountability, and evolution that give vital institutions their purpose and sustainability. In short, AI systems are a death sentence for civic institutions, and we should treat them as such.

Here’s Cory Doctorow on “cheap AI,” useful bits of tech that might be left after the AI bubble pops:

Here’s hardware hacker Pete Warden demoing a chatbot that you talk to and that talks back to you – and it’s running on Synaptics System-on-a-Chip (SoC) that costs “low single digit dollars”:

https://petewarden.com/2025/10/16/why-does-a-local-ai-voice-agent-running-on-a-super-cheap-soc-matter/

This is basically a little special-purpose Alexa, except it doesn’t connect to the internet at all (and therefore doesn’t leak any of your data). In Warden’s demo, the gadget is a button-sized voice assistant that is meant to be integrated into a dishwasher, which can interpret the dishwasher’s manual for you. If your dishes come out dirty or if the drain gets clogged, you press the button, describe your issue in pretty vague terms, and it instantly speaks aloud all the troubleshooting steps to deal with it. This privacy-preserving, cheap-like-borscht component adds a voice-activated, conversational assistant to a device, sipping power like the clock on your microwave, running on a processor that costs less than a pack of AA batteries. It’s seriously f****** cool. There’s going to be a lot of this AI, after the AI goes away – just like there was a lot of the web after the dotcom crash, when, overnight, San Francisco had infinity office-space, servers, and techies going begging.

The Internet of Fake

Blogger Kev Quirk shares an email he got from his son’s school:

In hard-copy or online school publications which are widely available, we will no longer use images in which our students can easily be identified.

To allow us to continue to give a visual flavour of school life, we may use AI-generated illustrative images featuring entirely fictional people. These images will not depict real students and are intended only to represent the type of activities, subjects and experiences available at[school name].

On the one hand, this seems like a good idea, protecting the privacy of the students in an age when anyone can do anything with any photo.

On the other hand, it’s an example of a problem that’s been around for at least ten years and is just going to get worse. We’ve told ourselves and our kids that “you can’t trust the internet” and “online is not real life.” The rise of generative AI underscores the truth of those statements.

But also note that the school will be using these fake images in “hard-copy” publications as well. Those mid-century media theorists that warned us about the dangers of photography, mechanical reproduction, and all the rest were right. Maybe AI will finally teach us the lesson.

Idle

I have read many interesting or insightful articles online. I have not read many that make me stop and think, “That was well put.” The work of Wright Thompson and Eva Holland comes to mind, as does a short story by Carmen Maria Machado that stopped me in my tracks when I came across it more than ten years ago. But the vast majority of blog posts and web articles are meant to be consumed quickly. The internet doesn’t lend itself to slow, patient reading. As for re-reading, the internet seems designed to actively discourage it.

You can imagine, then, how delighted I was to come across The Idle Gazette. The guy who writes under the pseudonym Leo Umilio has a casual, brief style that suits the internet, but he also has the ability to turn a phrase. In his writing, he comes across as someone who reads books more than he reads online, which I think is true.1

Here are a few tidbits, chosen for their quality as much as for their insight.

On the jar cellar:

The keeper of the cellar, my grandmother in the classic administration, would descend on some pretext and remain below suspiciously long, and what she was doing down there was auditing: touching lids to check the seal’s dimple, turning a doubtful jar to the light, aligning labels, moving last year’s survivors to the front the way librarians face out the neglected stock. And then, the audit done, simply standing, arms crossed, regarding the ranks. That regard is the dispatch’s whole subject. It was the look of a person counting wealth, and the wealth was real, but no part of it was money: it was proof, in glass, that the year had been lived attentively, that the summer had not escaped unrecorded, that the household would eat its own July in January. Economists have no column for this asset. The cellar keeps its own books.

On cats:

A cat is a live-in demonstration, renewed daily, that a life can be entirely without output and entirely without shame; she is the only ascetic who owns nothing, wants nothing, and has renounced nothing, since renunciation would imply the desire had existed. Every household with a cat contains one being who has solved the problem the rest of the household is working on.

On the man burning leaves:

He has a rake, and the rake matters enormously, being his license. Leaning on it, he presides over a low smoldering heap at the bottom of the garden, and he will preside there for two hours, occasionally turning something, mostly not, watching the smoke lean one way and then the other, and no passerby, spouse, or municipal authority of the old dispensation would ever have described him as idle. He is burning the leaves. It is a recognized activity with a name and a season and visible output. That the actual labor involved amounts to eight minutes of raking distributed across an entire afternoon; that the rest is standing, poking, and long unfocused looking into the slow combustion; that he could compost the lot in a corner in a quarter of the time; none of this has ever been held against him, because the fire cannot be left, can it, and everyone accepts that a fire cannot be left. Nobody has ever burned leaves efficiently. The fire enforces attendance, and the attendance was always the point.

On the quince:

Raw, the quince is a fraud: it advertises like a pear and delivers like a fence post, hard past biting, sour past joking, its flesh full of grit the botanists politely call stone cells.

On waiting to cut wood:

[W]ood that has just changed houses is still arguing with its new air, swelling here, easing there, settling its fibers into the humidity of the room, and a joint cut today, however cleverly, will have quietly walked out of square by winter. The boards are stacked with spacers between them so the air can circulate, which is the workshop’s way of saying that they are thinking. This is not a metaphor, or not only one. The moisture equalizes; the grain finds its final opinion. My intelligence can measure the plank today. Wisdom is not cutting it until it has finished changing its mind.

You really should follow all the links and read each essay in full.

  1. Umilio does not include hyperlinks, even when referencing his own website, which makes the posts self-contained and therefore more permanent than most online writing. ↩︎

Protect the Wildlife

The Sun is not known for journalistic excellence, but its reporters sure do know how to end an article:

In Skamania County, harming Bigfoot is punishable by a $1,000 fine and can include jail time.

This law is meant to protect the mysterious creature and prevent hunters with large beards from accidentally getting shot.

From Idaho B Roll

by Brian Blanchfield

In Garfield, Washington, the second of three 
speed-trap towns cutting over into 
Idaho on the way home from Spokane, 
there is a gray-going-white basketball 
furred from use and exposure, deflated 
only enough to discourage prolonged play, 
in the grass by the public court, beside 
the little park's restroom, the simplest soonest 
option en route. It pleases me again 
to spot it and, before returning to the car, 
to shoot two or three baskets. It must, 
with everything else, be buried under snow 
half of each winter. You lose the news, 
you shake the hour of seated transit off 
and stand quiet with whatever you've seen: 
a tractor waiting to pull the giant buck 
from the double yellow line, the pheasant vanishing 
in the bush, the long bright flowering wheat 
or waves of grain in the anthem area wind 
inspires. Bounce it two or three times and find, 
at the four finger pads of your right hand, 
a meridian bowed across the ball, the grace, 
remembered, by feel, of backspin. Unseen 
mark of experience, in a groove, at the line, 
clock stopped, to get it to roll back to you.

Originally published in the Atlantic