What's happening next to search
Searchable content is becoming more exclusive, and AI is having its SEO moment.
Google may still be the dominant way we search, but social media and LLMs are changing how we find information. We’ve gone from actively foraging for information to being passively fed it. With that shift, I’ve noticed two trends:
- Searchable content is becoming more exclusive
- AI is having its SEO moment
Background: How we got here
It feels like searching on Google isn't as good as it used to be. I'm not the first to say it, and this isn't the first time I'm saying it either.
- Search results have become overrun by SEO, making them less useful. (What I love most about that article is how it hit a nerve with people in the SEO community.) More recently, nefarious SEO tactics are used for purely monetary gains: “Forbes Marketplace is the single largest (and most successful) parasite SEO program of all time.”
- Vertical search engines, which focus on narrow topics, have grown in popularity. Platforms like Yelp for restaurants, Hotels.com for accommodation, and LinkedIn for professionals have become go-to sources for specific searches. While Google has incorporate similar features via search tabs and Google Maps, it didn't capture everything.
- More content moved to social media, leaving it hidden from search engines. This changed how people discover products: “Young shoppers don’t seek out brands; brands come to them” and fueled an appetite for bite-sized information.
- The rise of email newsletters and boom in paid subscriptions, driven in part by Substack, created a network of curators, often accessed through email inboxes or behind sign-in screens.
- Curating information became fashionable, thanks to the rise of email newsletters and boom in paid subscriptions. While Substack, in part, fueled this growth, this was a wider response to information overload.
- LLMs arrived like a breath of fresh air. Instead of traversing ten links, you get a single, direct answer. OpenAI, Bing, and Perplexity jumped at this opportunity, and Google followed suit with Gemini.
These shifts have resulted in a web where we expect information in the spaces we already occupy, along with a desire for more concise results.
1. Searchable content is becoming more exclusive
For years, social media content has been invisible to Google. The behavior of turning to social media platforms over Google for specific information is already well established. Google is no longer the single gateway to all the world’s information.
That’s why Google’s July 2024 content licensing deal with Reddit stands out to me. Google is now the only search engine that works on Reddit. Will content exclusivity licenses become the way platforms create moats around search behaviors?
Similarly, a recent NewsGuard analysis found 67% of top news sites block AI access. There are various reasons for this, including ongoing lawsuits, but what matters is LLMs don’t have the same access to news as Google search.
Knowing where to go is becoming more important than knowing what to search for.
2. AI is having its SEO moment
Earlier this year, a research paper titled GEO: Generative Engine Optimization, caught my attention. It proposed a novel way to “aid content creators in improving their content visibility in Generative Engine responses.” While promising, even the paper acknowledged it as just a first step.
A few months later, Kevin Roose of The New York Times reported on companies working to improve their reputation with LLMs, called “A.I.O. — A.I. optimization.” And there’s demand for it:
Companies are eager to insert themselves into chatbot responses, so that when a ChatGPT or Gemini user asks “What’s the best restaurant in Dallas?” or “Which S.U.V. should I buy?” the chatbot recommends their products.
SEO for generative AI went from research to product in less than a year. All it takes is editing the websites that are ingested by LLMs. With many major, reputable news sites being excluded from LLMs, it makes it easier to exploit LLMs to write favorable (for you) content.
How information is presented is critical. A search result with ten links doesn’t give you the answer—it requires users to interpret what they see. Even if all the links are wrong, the user still has to make a judgement. An LLM, on the other hand, provides a single, authoritative answer. Right or wrong, it removes almost all evaluation effort. This is a monumental shift, especially now that it’s becoming easier to shape what those LLM answers are.
What this all means
I’m not entirely sure what all of this means yet. Google is still an incredibly popular search engine, used billions of times a day. I don’t see that changing. At least, not anytime soon. However, we are witnessing a splintering in how we find and access information. We still turn to Google for trending news stories or recipes—perhaps that won’t be the case in the future.
Having exclusive data or better interfaces for accessing information may be useful advantages to build moats around behaviors related to searching, discovering, or finding information.
Spam Mail as an AI generated podcast
I added this week's newsletter into NotebookLM—Google's AI research assistant—and generated a podcast. It's MIND BLOWING.
I've tried as many AI summarizing tools as I can in search of my dream product: I want to point an LLM at my Google Drive and be able to ask it all the questions I have. NotebookLM doesn't quite do that, but its summarization is better than most.
The podcast generator was released a couple weeks ago and it's frightening how good it is. I gave it no additional prompts yet it feels real, from the tiny pauses in conversation to the inserting of personality. Wild.
💩 Cool shit
Dead-Internet – A Github project that lets you browse an entirely fake web where everything is AI generated.
Gummy Search – This is an interesting product. It's a social listening tool built for Reddit. Search for pain points that could become a new product idea. I'm intrigued but am becoming increasingly suspicious that much of Reddit is spam.
Cursor Museum – Do you want an ancient mouse cursor?
Open and Shut – Another wild Github project I love. Write in morse code by slamming and opening your laptop lid.
Consumed Today – I don't know who made this. It's both mad and impressive. It's a running journal of every piece of media this person has consumed. Included are some, uh, sound effects, and an interesting floating effect in the UI.
Is My Blue Your Blue – If, like me, you’ve ever wondered how our agreement on color is arbitrary, you’ll love this.
All Text in NYC – A search engine to look for text written around the streets of Brooklyn. It's deceptively simple but kind of mind blowing when you use it.
Websites Done Cheap – I have no idea if this is real or just a joke site. But, I love the UI.
Share this with a friend for your Legal Notice of Class Action Settlement: You May Be Entitled To Compensation.