Google, Microsoft, OpenAI and Anthropic Adopt New AI Crawling Rules

On November 2, 2026, the search world got a jolt on multiple fronts. Google, Microsoft, OpenAI, and Anthropic all moved toward adopting a new AI training protocol that would give publishers cleaner, simpler control over how their content is used for AI training and search grounding. At the same time, Reddit's stock fell sharply after its own CEO confirmed what every SEO already suspected: Google Search traffic to the platform is declining. If you've been watching AI visibility tracking as a signal for where search is heading, this week made the picture a lot clearer.

Barry Schwartz at Search Engine Roundtable covered the full picture in his daily recap. The convergence of these stories, new AI crawl controls, declining Google traffic to Reddit, and ChatGPT's rapid ad expansion, tells a coherent story about where the search ecosystem is right now. Let's break it down.

What Google and Microsoft's New AI Training Controls Actually Say

According to the Search Engine Roundtable November 2 recap, Google is adopting new AI training controls currently under consideration by the IETF, the body that sets internet standards. The goal is to simplify and streamline how publishers signal their preferences around AI training. That could mean less guesswork about how your AI training preferences are interpreted.

Microsoft Bing went further with a formal announcement. Krishna Madhavan, Principal Product Manager at Microsoft Bing, described the new controls as offering "newer, simpler, and clearer preferences that allow you to make choices about training, grounding, or both while remaining eligible for Bing search results," according to the recap. That last part matters a lot: you can opt out of training while remaining eligible for Bing search results. That's a meaningful distinction from the blunt all-or-nothing tools most publishers have had until now.

The fact that Google, Microsoft, OpenAI, and Anthropic are all moving in the same direction at the same time points to industry-level coordination, likely driven by publisher pressure and regulatory momentum. The IETF angle is significant too: a standards-body framework means these controls could eventually become a baseline expectation across every major AI crawler, not just the big four.

What the Reddit Stock Drop Reveals About AI's Impact on Search Traffic

Reddit's stock dropped 36% over the past week after CEO Steve Huffman confirmed that Google Search traffic to the platform has been declining, according to the recap. That's not a rumor or an SEO's speculation anymore. It came straight from the top.

And yes, every working SEO saw this coming. Reddit had an enormous surge in Google visibility in recent years as Google pushed forum content and first-hand experience into results. If AI Overviews are surfacing Reddit discussions directly in the SERP, fewer users need to click through. Referral traffic can decline even as brand visibility stays high. That is the core tension of the AI search era: your content can feed the machine without your site seeing the benefit.

The recap also noted that on Reddit's Q3 earnings call, the company was asked about advertisers seeking to be included in LLM pre-training so they can show up in AI search. Reddit's response was pointed: "We're skeptical of GEO." That's a notable public pushback from a platform that has been one of the biggest beneficiaries and casualties of the AI search shift at the same time.

ChatGPT Ads Expansion and What It Means for Your Paid Search Strategy

OpenAI sent its weekly advertiser update confirming several ChatGPT Ads developments, all reported in the recap. ChatGPT Ads are now live in 26 more countries. A new wider standard ad unit format is available. Multi-image ads entered open beta for all advertisers. And Shopify integration is now available internationally.

The pace here is fast. Multi-image ads, wider formats, and international Shopify integration landing in the same week signals that OpenAI is treating its ad platform as a serious competitive product, not a side experiment. For agencies managing ecommerce clients, the Shopify integration going international is the one to watch closely.

Gary Illyes on Why Click Data Won't Come to Google Search Console AI Reports

Gary Illyes from Google implied that click data and query data are likely not coming to the AI performance report in Google Search Console, at least not anytime soon. According to the recap, Illyes wrote that the Search Console team is interested in bringing "data that makes sense" to the report but did not confirm that clicks or query data qualified.

This is frustrating for anyone trying to measure AI Overview performance in a meaningful way. Without click or query breakdowns in the AI performance report, it's hard to tie AI visibility to business outcomes. Agencies building AI visibility dashboards can hit a wall when a client asks, "okay, but is it driving traffic?" For now, the honest answer is: Google hasn't given us the data to know.

Google Ads AI-Generated Reps and the Trust Problem for Advertisers

The recap flagged something that's going to bother a lot of advertisers: Google Ads Strategists, part of Google's Extended Workforce, are reportedly using AI-generated profile photos and names when reaching out. The recap referenced an example of an email sent from someone named Kassy Garcia.

This is a trust and transparency issue. Advertisers managing significant budgets expect to know who they're talking to. If the person giving you account recommendations isn't a real human with a real name, it changes how you should evaluate that advice. It also raises broader questions about disclosure standards as AI-generated personas become more common in customer-facing roles.

What Agencies and Marketers Should Do in Response to These AI Search Shifts

1. Audit your AI crawler permissions now. The new IETF-aligned controls from Google and Microsoft are coming. Get ahead of it by reviewing your current robots.txt and any existing crawler directives. Decide deliberately whether you want your content used for AI training, search grounding, or neither, and make sure your settings reflect that choice.
2. Stop treating AI Overview appearances as a vanity metric. Until Google adds click and query data to the AI performance report in Search Console, track AI visibility alongside direct traffic and branded search volume. If your AI appearances are rising but organic clicks are flat or falling, you may be feeding the answer engine without getting the referral benefit.
3. Evaluate ChatGPT Ads if you have ecommerce clients. The Shopify integration going international and multi-image ads entering open beta are concrete features worth testing. Budget a small allocation and run it as an incremental channel alongside Google and Meta before committing.
4. Be skeptical of outreach from Google Ads reps you can't verify. If you get an email from a Google Ads Strategist, confirm their identity through your official Google Ads account portal before acting on any recommendations or sharing account access.
5. Watch the Reddit traffic story as a leading indicator. The Reddit CEO's confirmation of declining Google Search referrals is a real-world data point about how AI answers may be affecting publisher traffic at scale. If your content strategy still relies heavily on third-party platforms for organic reach, now is a good time to build owned content assets that are harder to disintermediate.

How the November 2026 Search News Fits the Broader AI Search Transition

None of this is happening in isolation. The IETF crawl control framework, Reddit's declining Google traffic, the ChatGPT Ads expansion: these are three angles on the same underlying shift. Search is becoming an AI answer layer, and the entire ecosystem, publishers, advertisers, and platforms alike, is still figuring out who captures value in that model.

Google's stock also took pressure after its Q3 earnings raised AI revenue concerns, according to Bloomberg reporting cited in the recap. That tells you the market is asking the same question publishers are: if Google answers queries inside the SERP without a click, where does the revenue come from? The answer, presumably, is ads. Which is why ChatGPT's ad expansion and Google's continued push into AI Overviews with ad placements are so closely linked.

For SEOs, the most useful framing right now is this: optimize for citation, not just ranking. Getting your content cited in an AI answer is the new version of ranking on page one. And the signals that drive citation, structured data, clear attribution, topical authority, direct and specific answers, are things you can control today.

If you want a cleaner way to track how your site shows up across AI search environments alongside traditional rank data, AI visibility tracking tools like the ones inside Aergos can help you see both signals in one place, so you're not flying blind when a client asks where their traffic went.