Apparently I really like pancakes
A bumpy ride through the current state of personalized AI
I love making breakfast for my kids. It’s the meal where indulgence and nutrition fight it out, and I’m always picking sides. A few months ago I stepped into the breakfast ring and decided to make pancakes. My wife took a stance on behalf of nutrition and we settled on making them oatmeal pancakes. Rather than battling it out with recipe websites that don’t show you the actual recipe until you’ve scrolled through 9 ads and six paragraphs of the cook’s deep thoughts on how to properly soak oatmeal, I decided to ask Gemini for a recipe. It delivered. They were delicious.
Without thinking twice I told Gemini “That was great. Remember that recipe.” From that point forward, Gemini has decided to bring up oatmeal pancakes with just about every response it provides. “Where can I recycle batteries nearby?” “Try Corbet’s hardware in Larkspur. Oh, and before you go let me know if you want to make your kids some of those oatmeal pancakes they love”. Or this one during a session about planning a vacation:
It continues to this day. Gemini’s desire to endear itself to me and give me a sense that it “knows me” manifests in the most ham-fisted way possible. The whole point of personalization is that it should be invisible until it matters. Gemini is getting that exactly backwards.
Here it is giving me suggestions about what to call this Substack newsletter:
I spent a decade shipping the productivity tools these companies are now trying to make personal — Outlook at Microsoft, Workspace at Google. Personalization is the problem we kept circling and never fully solved. Watching the AI labs run at it now is like watching a familiar movie with new actors. Personalized AI is one of the pieces of the AI puzzle that the big AI labs are all racing to solve and they are all coming up short.
Anthropic’s Claude Cowork is taking a different approach. They’re trying to convince us that the most effective way to represent memory is long form, structured documents in Markdown format starting with you CLAUDE.md file.
This is what Cowork knows about me. A directory. There's a file for my project context, a file for my voice, a file for my workflow, and a separate file devoted to the fact that I don't like em dashes. Someone, somewhere, decided em dashes warranted their own document. I can open each file and read it, but transparency isn't the same as knowing someone. The guy who cuts my hair knows me better than this and he doesn't carve me into seven markdown files.
The promise of Cowork is the ability to take information from a range of sources (yes, including Gmail and Outlook – the crown jewels of personal data) and put it to work to make you more productive. The results there are also hit or miss. It only consults those sources when explicitly directed by the user to do so (Gemini does this too). It stores and maintains markdown documents that it believes will be relevant for future sessions. How does it determine this? Magic. How does it maintain this? More magic. What’s in there? Go check. Bring Emacs.
Apparently we’ve settled on the filesystem as the closest representation of the complexity and mysteries of the human brain.
It may not be a black box, but a bunch of oddly formatted text files doesn’t add up to actual control either.
These approaches fail because they’re either trying to delight the user through emergent behavior (Gemini) or through declarative insights (Anthropic). Gemini over-applies personalization to try to endear itself to its users. Cowork asks you to do a bunch of upfront work and puts it on you to decide what is and isn’t relevant.
So what’s the bar? Start with a simple principle:
Personalization should be invisible until it’s useful.
And only the user can judge that.
The current systems get this backwards. Gemini decides pancakes are useful because that’s what it has and it thinks using that will make me trust it. The user is the last person actually consulted.
What do you know about me? Apparently I like making pancakes.
Personalization is one half of the problem. The harness — the actual interface you sit in — is the other. And on that front, AI is sleepwalking into every mistake email already made. More on that next time.
on’t show you the actual recipe until you’ve scrolled through 9 ads and six paragraphs of the cook’s deep thoughts on how to properly soak oatmeal, I decided to ask Gemini for a recipe. It delivered. They were delicious.
Without thinking twice I told Gemini “That was great. Remember that recipe.” From that point forward, Gemini has decided to bring up oatmeal pancakes with just about every response it provides. “Where can I recycle batteries nearby?” “Try Corbet’s hardware in Larkspur. Oh, and before you go let me know if you want to make your kids some of those oatmeal pancakes they love”. Or this one during a session about planning a vacation:
It continues to this day. Gemini’s desire to endear itself to me and give me a sense that it “knows me” manifests in the most ham-fisted way possible. The whole point of personalization is that it should be invisible until it matters. Gemini is getting that exactly backwards.
Here it is giving me suggestions about what to call this Substack newsletter:
I spent a decade shipping the productivity tools these companies are now trying to make personal — Outlook at Microsoft, Workspace at Google. Personalization is the problem we kept circling and never fully solved. Watching the AI labs run at it now is like watching a familiar movie with new actors. Personalized AI is one of the pieces of the AI puzzle that the big AI labs are all racing to solve and they are all coming up short.
Anthropic’s Claude Cowork is taking a different approach. They’re trying to convince us that the most effective way to represent memory is long form, structured documents in Markdown format starting with you CLAUDE.md file.
This is what Cowork knows about me. A directory. There's a file for my project context, a file for my voice, a file for my workflow, and a separate file devoted to the fact that I don't like em dashes. Someone, somewhere, decided em dashes warranted their own document. I can open each file and read it, but transparency isn't the same as knowing someone. The guy who cuts my hair knows me better than this and he doesn't carve me into seven markdown files.
The promise of Cowork is the ability to take information from a range of sources (yes, including Gmail and Outlook – the crown jewels of personal data) and put it to work to make you more productive. The results there are also hit or miss. It only consults those sources when explicitly directed by the user to do so (Gemini does this too). It stores and maintains markdown documents that it believes will be relevant for future sessions. How does it determine this? Magic. How does it maintain this? More magic. What’s in there? Go check. Bring Emacs.
Apparently we’ve settled on the filesystem as the closest representation of the complexity and mysteries of the human brain.
It may not be a black box, but a bunch of oddly formatted text files doesn’t add up to actual control either.
These approaches fail because they’re either trying to delight the user through emergent behavior (Gemini) or through declarative insights (Anthropic). Gemini over-applies personalization to try to endear itself to its users. Cowork asks you to do a bunch of upfront work and puts it on you to decide what is and isn’t relevant.
So what’s the bar? Start with a simple principle:
Personalization should be invisible until it’s useful.
And only the user can judge that.
The current systems get this backwards. Gemini decides pancakes are useful because that’s what it has and it thinks using that will make me trust it. The user is the last person actually consulted.
What do you know about me? Apparently I like making pancakes.
Personalization is one half of the problem. The harness — the actual interface you sit in — is the other. And on that front, AI is sleepwalking into every mistake email already made. More on that next time.