OpenAI has announced SearchGPT, a prototype AI search engine that answers questions with up-to-date information from the web and links to the sources it used. If you run a website, the big question is simple: should you worry, and is there anything to do today?
My short answer: the SearchGPT prototype will not change your traffic this month, because only a small test group can use it. But it does come with one practical task, which is checking that your robots.txt file lets OpenAI’s new search crawler in. Below I explain what was announced, how it works, how it compares with Google, and what I recommend you do now.
What OpenAI Announced on 25 July 2024
On 25 July 2024, OpenAI introduced SearchGPT and called it a “temporary prototype” of new AI search features. It is not a finished product, and it is not yet part of ChatGPT.
Here are the main facts from the announcement:
- Who can use it: a small group of users and publishers at first. Everyone else can join a waitlist at chatgpt.com/search.
- What it does: it gives a direct answer to a question, pulled from current web information, with clear links to the sources.
- Follow-up questions: you can keep asking, and it remembers the context of the conversation, like a chat.
- Source links: answers show named, in-line links to the original sites, plus a sidebar with more links.
- Future plans: OpenAI says it will add the best parts of SearchGPT into ChatGPT later. It also wants to improve areas like local information and shopping.
- Publishers: OpenAI says it built the prototype with feedback from news publishers, and it quoted partners such as The Atlantic and News Corp.
OpenAI has not given a date for a full launch. It has not said how big the waitlist is or how fast people will get in.
How the SearchGPT Prototype Works
If you have used ChatGPT, SearchGPT will look familiar. There is a large search box. You type a question in normal language, and you get a written answer instead of a list of ten blue links.
The difference from plain ChatGPT is the live web. A normal chatbot answers from what it learned during training, which can be months out of date. SearchGPT looks things up on the web at the time you ask, then writes an answer from what it finds.
Answers With Named Sources
OpenAI put a lot of weight on attribution, which means showing clearly where the information came from. Answers include in-line links that name the source, and a sidebar lists more results you can click.
For site owners, this is the part that matters most. A source link is the only way an AI answer can send a visitor to you. The more visible those links are, the better the chance of a click.
Follow-Up Questions
Like ChatGPT, SearchGPT keeps the conversation going. You might ask “best hiking boots for wide feet” and then “which of those are waterproof?” without repeating yourself.
This changes how people search. Instead of five separate searches, a person can refine one long conversation. Each follow-up is another chance for a different set of sources to appear.
It Can Get Things Wrong
Even OpenAI’s own launch demo had a date error in one answer about an event. That is a good reminder that this is a prototype.
AI answers can mix up facts, especially dates, prices and opening times. I would not trust any AI search tool for details that matter without clicking through to the source.
SearchGPT vs Google, Bing and Perplexity
SearchGPT is not the first tool of its kind. It joins a busy field, and it helps to see where it fits.
| Tool | What it is | Who can use it now |
|---|---|---|
| SearchGPT | A separate AI search prototype from OpenAI with source links and follow-ups | A small test group plus a waitlist |
| Google AI Overviews | AI summaries shown above Google’s normal results for some queries | Searchers in the US since May 2024, with more countries planned |
| Microsoft Copilot (formerly Bing Chat) | AI chat answers built into Bing, with numbered source links | Widely available |
| Perplexity | An AI answer engine that cites web sources | Widely available |
Google’s version sits inside the search results page that billions of people already use. I covered how that rollout started in AI Overviews launching in the US. SearchGPT, for now, is a separate place people have to choose to visit.
Microsoft got there first with AI chat in Bing, which I wrote about when the new Bing with AI chat arrived in February 2023. Perplexity has also built a following with a similar answer-plus-sources format.
So what makes SearchGPT different? Mainly the name behind it. ChatGPT is one of the best-known AI products in the world. If OpenAI adds search into ChatGPT, it puts an AI search tool in front of a very large audience that already has the habit of asking it questions.
Will SearchGPT Replace Google?
Not any time soon, in my opinion. Google handles a huge share of searches, it is the default in most browsers and phones, and people have used it for decades. Habits like that change slowly.
There are also things a prototype will find hard at first. Local searches, shopping, fresh news and maps need a lot of data and trust. OpenAI itself says local information and commerce are areas it still wants to improve.
What I do expect is that search keeps splitting across more places. A few years ago, most of your search traffic came from Google. Now some people ask ChatGPT, some use Bing’s chat, some use Perplexity, and many still use Google. SearchGPT adds one more place where your content could appear, or not appear.
That is why I treat it as another channel to prepare for, not a threat that needs a panic plan.
What SearchGPT Means for SEO and Website Traffic
Let’s be practical. Today, the effect on your traffic is close to zero, because so few people can use it. The real question is what happens if the best parts move into ChatGPT, as OpenAI plans.
Fewer Clicks for Simple Questions
When an AI tool answers a simple fact question in full, many people will not click anything. That is the same worry SEOs have with Google’s AI Overviews and featured snippets.
Pages that only answer one short question, like “how many ounces in a cup”, are the most at risk. I’ve seen this pattern for years with featured snippets, and AI answers make it stronger.
Better Clicks for Deeper Content
On the other side, people who click a source link from an AI answer usually want more depth. They want the full guide, the product details, the tool or the expert view.
In my experience, content with real experience, original data, clear opinions and useful tools gets the clicks that are left. Thin pages that repeat what everyone else says will struggle in both Google and AI search.
Your Brand Matters More
When an AI tool names a source in its answer, a known brand gets more trust and more clicks. If people already recognise your name, they are more likely to pick your link from the sidebar.
This is one reason I tell site owners to think beyond rankings. Being mentioned and trusted across the web helps you in classic search and in AI answers at the same time.
OAI-SearchBot: The New Crawler You Should Know
This is the most useful part of the announcement for site owners. Along with SearchGPT, OpenAI introduced a new web crawler called OAI-SearchBot. A crawler is a program that visits web pages and reads them, the way Googlebot does for Google.
OpenAI says OAI-SearchBot is used to find and show websites in SearchGPT results. It also says this bot is not used to collect content for training its AI models.
OAI-SearchBot vs GPTBot
OpenAI now has two separate crawlers that site owners should understand:
- GPTBot: collects content that may be used to train OpenAI’s AI models. OpenAI introduced it in August 2023.
- OAI-SearchBot: finds and shows websites in SearchGPT search results. It is not used for training.
This split is good news. OpenAI says your site can still appear in SearchGPT even if you block your content from being used for AI training. You no longer have to choose between “allow everything” and “block everything”.
Google made a similar split last year with Google-Extended, which I explained in blocking Bard and Vertex AI training from your site. The idea is the same: one control for training, separate from being found in search.
What OpenAI Recommends
If you want your site to show up in SearchGPT, OpenAI recommends two things:
- Allow OAI-SearchBot in your robots.txt file (a plain text file at the root of your site that tells crawlers what they may visit).
- Allow requests from OpenAI’s published IP address ranges, so your firewall or security plugin does not block the bot.
You can find the user agent names and IP lists in OpenAI’s crawler documentation.
How to Check and Update Your robots.txt for SearchGPT
This is the one job I recommend doing this week. It takes a few minutes, and it means you are ready if SearchGPT grows.
- Open your robots.txt file. Go to yourdomain.com/robots.txt in a browser and read what is there.
- Look for blanket blocks. Check for
User-agent: *followed byDisallow: /. That blocks every crawler, including OAI-SearchBot. - Look for OpenAI rules. Search the file for “GPTBot”, “ChatGPT-User” and “OAI-SearchBot”. Many sites added a GPTBot block last year and never looked again.
- Decide your policy. Choose whether you want to appear in SearchGPT, and separately whether you want to allow AI training.
- Add the rules you need. Write a clear rule for each bot, then save and upload the file.
- Check your firewall and CDN. Make sure your security tools are not blocking OpenAI’s IP ranges, or your robots.txt rules won’t matter.
- Test the file. Reload the robots.txt URL and confirm the new rules show up exactly as written.
Here is an example for a site that wants to appear in SearchGPT but does not want its content used for training:
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /If you are happy with both, you don’t need any special rule. A crawler that is not blocked is allowed by default.
One caution: robots.txt controls crawling, not what happens to content already collected. Google’s own introduction to robots.txt is a good refresher on how these rules work and what they can’t do.
Mistakes I Usually See
- Copy-pasted AI block lists. Many sites pasted long lists of AI bots to block without reading them. Those lists may now block search, not just training.
- Security plugins with bot blocking turned on. These can block new crawlers by default. I always check the firewall settings after any robots.txt change.
- Typos in user agent names. A rule for “OAISearchBot” without the dash does nothing. Copy the exact name from OpenAI’s documentation.
- Conflicting rules. A blanket block for all bots above a specific allow can confuse people reading the file later. Keep the file short and clear.
How to Prepare Your Content for AI Search
Beyond robots.txt, there is no special “SearchGPT SEO” yet. OpenAI has not shared how it picks sources. So I’d focus on the things that already help in Google and in other AI tools.
Answer the Question Early and Clearly
AI tools pull out clear, direct answers. Put the main answer near the top of the page in plain words. Then add the depth, examples and detail below it.
Show Real Experience
Generic content is easy for an AI to replace. First-hand testing, real photos, clear opinions and original data are much harder to copy. Those are also the things that make someone want to click through.
Keep Facts Up to Date
SearchGPT’s selling point is fresh information from the web. Pages with old prices, past dates or outdated steps are less useful as a source. I review my most important pages on a schedule and update the facts, not just the date stamp.
Make Your Brand and Authors Clear
Use a clear site name, an About page, and author pages that explain who writes your content. When answers show named sources, a clear brand helps people trust the link and pick yours.
Keep the Basics Strong
Fast pages, clean HTML, clear headings and good internal links help every crawler, not just Googlebot. If a page is hard for Google to read, it is likely hard for other bots too.
How to Track Traffic From SearchGPT
Right now, the test group is small, so don’t expect to see much. Still, it is worth setting up a simple way to spot AI search visits early.
- Check referral traffic in GA4. Look at the session source report for referrals from chatgpt.com and other AI tools. Visits from AI answers often show up here.
- Watch your server logs. Look for OAI-SearchBot in the user agent field. That shows whether the bot is visiting and which pages it reads.
- Save a baseline. Note your current AI referral numbers so you can compare when SearchGPT features reach more people.
I’d also keep an eye on how often OpenAI’s bots hit your server. If crawling grows, you want to know before it affects site speed.
My Take on the SearchGPT Prototype
I think the SearchGPT prototype is an important signal, more than an important product today. OpenAI is telling the market that it wants a share of search, and that it plans to bring search into ChatGPT.
I like two things about the announcement. First, the focus on clear source links. Second, the split between the search crawler and the training crawler, which gives site owners a real choice.
What I’d skip is any rush to rewrite your content strategy around a waitlist product. Nobody outside OpenAI knows how it ranks sources, and it may look very different by the time it reaches more users.
Conclusion: What to Do About SearchGPT Now
The SearchGPT prototype is OpenAI’s first real step into web search. It gives direct answers with named source links and supports follow-up questions, but for now only a small test group can use it.
Your next step is small and practical: check your robots.txt and firewall today, decide how you want to treat OAI-SearchBot and GPTBot, and keep building content that is worth clicking. If you want the bigger picture on how AI answers are changing clicks, my guide to what AI Overviews mean for clicks is a good place to start.
