Webinar: Optimising for AI Agents: What Google's New Guidance Means for SEOs

Google has published formal guidance on how AI agents interact with websites: screenshots, the accessibility tree, semantic HTML, performance. For most SEOs, the accessibility tree alone opens up a whole set of questions that aren't in the usual audit checklist.

Tory Gray and Jessyca Frederick have been working through what this guidance actually means in practice. They joined Patrick Hathaway to discuss what's in the documentation, what it changes, and where there are still more questions than answers.

What we cover

• What Google's agent-friendly site guidance actually says, and what it leaves unspecified

• The accessibility tree: what it is, how agents use it, and why most audits don't touch it

• Whether this changes how you approach rendering, semantic HTML, and performance in a technical audit

• Screenshots: what's documented about how agents use them, and what isn't

• Practical first steps for making client sites more agent-ready

Webinar recording

By the way, you can also subscribe to the Sitebulb YouTube channel! And register here to be the first to find out about upcoming webinars. 

Webinar resources

• Google's original advice on AI agent site UX — reason we did the webinar

• Chrome Canary — for running Agentic Audits via Lighthouse (Google's auditing tool)

• IsItAgentReady.com — Cloudflare's agentic auditing tool

• ChatGPT Atlas — to explore agentic things in general and generate ideas about how your users might use AI on your site; my instructions here

• Full Accessibility Tree tool — to "see" the accessibility tree

• How AI Agents See Your Website: The Accessibility Tree Explained — excellent reading on:

  • what the accessibility tree is

  • token costs of different actions (screenshots, DOM tree, accessibility tree)

  • recommendations on what to fix for Agentic Optimisation

  • Aria labeling and how to avoid spamming it

• AI Bot Traffic — great data on AI bots (Kinsta)

• Discussion: hidden content for AI — is content in tabs less valuable to AI agents? An interesting rendering/accessibility edge case

• XKCD 927 — how Tory feels about the competing standards from IsItAgentReady and Google's new OKFs (my post on this)

• Open Knowledge Format announcement — Google's OKF blog post

• LLM AI Crawl Audit — covers the difference between live fetching bots and AI training collection bots (The Gray Company)

• OpenAI Bots documentation — note: does not include agentic bots

• A breakdown of the different types of bots and agents roaming free

• Different aspects of working with bots

The speakers

Tory Gray is CEO of Gray Dot Co, a technical SEO and research consultancy. She's been working through the agent accessibility question hands-on, including how it fits into the audit process at Gray Dot and what that workflow actually looks like.

Jessyca Frederick is a Product & SEO/AEO Lead working at the intersection of product thinking and search optimisation in the age of AI. Currently Director of Digital Product at Wine Enthusiast, and a freelance SEO Consultant.

Webinar transcript

Patrick Hathaway: Hi, everyone, and welcome to another Sitebulb webinar. Today we'll be talking about Google's new guidance on building agent-friendly websites, and what that actually means for the way we audit and optimize sites. I'm Patrick, CEO and co-founder of Sitebulb, and I will be your host today. Sitebulb is a website auditing tool that helps in-house and agency teams get to grips with technical SEO on sites of all shapes and sizes. And we have two amazing guests on our panel today, so please welcome Tory Gray and Jessyca Frederick.

Tory Gray: Hello?

Patrick Hathaway: Right. Little bit of housekeeping before we start. The session is being recorded. Anyone who signed up will get a recording emailed to them tomorrow, courtesy of Jojo, who you will see in the chat already. And then after the forty-five minutes or so of questions with me, we have fifteen minutes that we'll save for Q&A. So I'm sure you've got loads of burning questions for Tory and Jessyca, but the really important part about this is that if you have questions, please put them in the Q&A tab and not the chat box. Everybody always puts them in the chat box. So put them in the Q&A tab instead, which is on the left-hand side. And then the other thing you can do in there, if you don't have questions yourself, is to go and upvote other people's questions. So if we get lots of questions, we will work from the most votes down in democratic style. So thank you for joining, and let's go in and meet our guests.

Tory Gray is the founder and CEO of Gray Dot Company, a technical SEO and research consultancy. She's worked in partnership with Sitebulb on a number of projects, and I expect a lot of you will be familiar with her from her brilliant JavaScript SEO training series. Now Tory has been working through the agent accessibility question hands-on. Her team is actively building agentic auditing into Gray Dot's own audit and rendering workflow. So she's been living in this documentation the last few weeks. Thanks for joining us, Tory.

Tory Gray: Thanks for having me.

Patrick Hathaway: And please also welcome Jessyca Frederick, who is the director of digital product at Wine Enthusiast and a freelance SEO consultant. Which means she looks at this stuff through both the consultant's lens, working with clients, and through an in-house lens where she is responsible for the product vision and roadmap. Thank you also for joining us today, Jessyca.

Jessyca Frederick: Thank you for the invitation. Yeah. For sure. You know, I'm somebody who very much trusted everything Google had to say until we learned about Navboost in the trial from a few years ago.

Patrick Hathaway: So please give a warm welcome in the chat to Tory and Jessyca, and we'll move on to our questions. Right. So Google has now published formal guidance on how AI agents interact with websites. So let's get everyone up to speed. What's actually new here and what does it mean for SEOs? Tory, you've been digging into the docs. Help us out.

Tory Gray: So what's new here? I'd say, overall, we've not had any guidance from Google on how to treat agents. Right? Because agents are a new breed of bots. We've had a lot of guidance for many years on SEO-specific needs. We've had some guidance more lately in terms of other AI bots, and how Google treats, say, AI search for use in Gemini and AI Mode and AI Overviews and all these things, which is much more analogous to classical SEO. So this is specifically about agents. And so to me, this is pretty substantially different and new. It's a new toolset. It's new checks. It's new tech requirements. It's all just — it's all pretty new. It's an exciting kind of blue-ocean opportunity, for us SEOs figuring it out.

Patrick Hathaway: Okay. And Jessyca, any part of it that makes you want to take it with a bit of a pinch of salt?

Jessyca Frederick: I'm very skeptical, not so much that Google is telling us the truth, but of the full scope of what we're learning in this documentation. And I think one of the problems that I really kind of latched onto is this is Google's way of doing it, but it doesn't really tell us anything about what ChatGPT or Perplexity or Claude need from us and their agents. So, you know, as the whole space sort of shakes out and we see what percentage of users are in Gemini versus in the other tools, the value of this guidance may not be as strong as it was for SEO.

Patrick Hathaway: Yeah.

Tory Gray: That's a really excellent point. If I can just jump in and say, because Cloudflare also has a really great tool out, and they also have proposed standards that are very much competing with a lot of what Google is announcing. There's strong overlap, and the goals are similar, but they're different implementations. Different technical requirements. And Google is biased and wants you to do things the Google way. Right? So I think that's such an excellent point.

Patrick Hathaway: Yeah. 100%. Okay. So I want to talk about what the guidance says, because it says that agents read a page in three ways: with screenshots, by looking at the raw HTML, and also the accessibility tree. And that modern agents are kind of combining all three. So can we just try and break this down a little bit for our viewers? How does an agent actually see a site, and where does that differ from how Googlebot sees it? Jessyca, can you help kind of explain this one in plain English for us?

Jessyca Frederick: Sure. I think they see the site — like, they're making an impression of the site. Right? Like, they're using these different tools to triangulate the meaning of the items on the page, and they have what loads in the browser render tree. They have what loads in the accessibility tree. They have these screenshots, but, you know, one of the pieces of guidance buried in there is that, you know, if a button is in a different place on different PDPs, it may not be as useful. And that's something, like, practically impossible to control for if you have things above the button on the PDP, which most of us do for those situations. So I don't know what it sees. I know what it tells me it uses to see. And I think that's kind of my feeling about it at the moment.

Patrick Hathaway: Okay. That's interesting. And Tory, have you got an understanding about what — at least, what we've been told about how different agents view sites differently?

Tory Gray: I mean, they're different bots, and they have different capabilities. You know, I think semantic HTML has been around for a long time, but it's such a big, broad topic, really. I feel like this guidance gets us much more narrow on what they mean by semantic HTML, and it's largely, like, ARIA labels on things so we can understand what links do and what actions sites take. So I think the big difference here is that Googlebot and search engine bots in general — and even classic AI bots — aren't users, and they aren't interacting like users do on a website. So these new agentic bots are, and therefore, they need new capabilities and we need to enable that for them. So there's, you know, the three different very, very high-level tools. There's a lot more coming and a lot more that we're adding to our own agentic auditing process.

But I think one important thing to keep in mind is that these are three high-level tools and they have different costs associated with them. And by cost I mean, like, token usage. And are you a free user, or are you a paid user? What can you get out of the box? And screenshots, I think, are substantially new. Accessibility trees are pretty new, but they're much more analogous to what we have been doing in auditing. And it's kind of a browser-based tool. Right? So we can figure those things out. But, like, screenshots are pretty dang new to the world of SEO, and trying to make sure that those work and understanding, I don't know, the specs around what are those scores. So the big one there that they measure against is CLP — is it the content layout — nope, CLS. How much is the layout shifting? Is your button moving, because you have page speed and shifting issues?

I think of the screenshots as, like, an expensive workaround. When the other tooling doesn't work, when your raw HTML, when your ARIA labels aren't in place so that a bot can't figure out how to add to cart or, you know, if something goes wrong. It's sort of a workaround, but it costs the tool more, and they're passing that cost along to you as a user. In terms of, like, what can you accomplish in a given month. Right? Are you running out of tokens, or are you paying extra? There is an actual cost, and we're making that really tangible for the first time.

Patrick Hathaway: Yeah. That's a really, really interesting point about the cost as well. Particularly when we don't know at the moment if they're charging us the full cost or if that will be a bill to come due further down the line. Alright. So I just want to dig into, I guess, quite a lot of confusion that Google ended up causing a lot of people in the community. So I'll just kind of catch everyone up on this. So they published the agent-friendly guidance that we've just been talking about. They also published a guide on optimizing your website for generative AI, which includes specifically a myth-busting section in it which states that you don't need special files like llms.txt. Then, like, pretty much at the same time — all these things happen within a few weeks of one another — they also ship this Chrome Lighthouse agentic browsing audit, that literally checks for elements of text. So what the fuck's going on?

Tory Gray: What's the goal there? Good question. The difference is Google's guidance historically has been about SEO, and it's been about AI search. It has not been about agentic workflows and bots. It's different advice because it's a different bot with different implementation and different goals. llms.txt is part of Google's proposed support for how we enable an agentic web. They have explicitly stated that they're not using it for, you know, AI search and Gemini, and they're not using it for classic traditional technical SEO. So I get that there's a lot of confusion, but it's literally a whole different thing. And I think the confusion is just people lump all the AI bots together as a thing. And agentic bots are pretty substantially different.

Jessyca Frederick: I think it might also be a little bit of a hedge about whether or not the standard gets adopted. Tory and I had discussed previously that when I look in our bot mitigation software, I don't see anybody asking for it. Nobody's asking for llms.txt. And I think this morning, I saw somewhere come across my feed that someone just did a study about it, and it's, like, less than 6% of crawls are looking for llms.txt. And so I think maybe they're baking it into the Chrome audit in case it suddenly flares up as something that the bots are using and they're prepared for it. But I've never really understood the point of the llms.txt file. Other than the group that created it was trying to create a standard, like a robots.txt file. But given the abuse of robots.txt, I don't see why we would think it would be any different with an llms.txt.

Patrick Hathaway: Yeah. I think that's a good way to frame it. Like, we see Google in particular doing this all the time. Where they'll support a thing — like a structured data rich result, for example — they'll be, okay, we're gonna have this one. And then a few months later or a couple years later, it's just gone. You know? We're not supporting it anymore. And, like, the idea that probably they're just hedging — but I think then the conclusion is we should not use it as evidence that Google supports it. Right? It's just a thing that you could do if you want to, but there's not really been any evidence that it's gonna actually have any impact.

Tory Gray: I think it's also important to keep in mind that Google is not a monolith. It is different people and different teams that believe different things. So they want different things to be in the world, and they have different goals even. Right? So the SEO teams and the team supporting that are looking at entirely, substantially different things versus — I'm thinking, like, the head of AI search? It's Adi — I don't remember the last name — but they have had out their guidance on agentic stuff for months and months and months, and it's had recommendations around llms.txt for the agentic use case. For a long time. This is the first time it's been more official to my knowledge from Google in terms of incorporating it into the agentic test. Which is new. And so maybe it's the hedging, but I think it's also, like, different teams and different goals are pulling for different standards to reach adoption. For their different use cases that they care about. So when you think about it like that, maybe they're not hedging, maybe it's just different people have different goals and are trying for different things.

Patrick Hathaway: Yeah. Yeah. Chrome's team is not the same as the Gemini team, which is not the same as the search team. They're all different. Right?

Tory Gray: Exactly. Yeah.

Patrick Hathaway: Yeah. I think that's definitely worth us paying attention to as well. Okay. So let's just get into one of these things mentioned in the docs that perhaps not everybody watching might be that familiar with. That's the accessibility tree. I guess I'm looking for a bit more of an explanation about what it is, right, in plain terms. How is it different from the DOM? And, like, why has something that's been built for screen readers suddenly become central to SEO-type conversation. Jessyca, do you want to start us off with that one?

Jessyca Frederick: I'd love to start with the last part, which is — the thing that screen reader users and agents have in common is that they don't have vision. And we as more able people who can see well take advantage of, or take for granted, that our world is processed visually. Right? So for us, it's just second nature. And the screen reader technology has always been designed to compensate for people who don't have good vision, or can't see anything. And that's the world the bot lives in. Like, how do I interact with the web if I can't see something and point at it, or see something and touch it on my phone? And so I think that's really the underlying reason why all of a sudden it's so important, even though the accessibility tree has been available in the browser since 2021. Or possibly before then. That's sort of my first awareness of it.

Patrick Hathaway: Yeah. I mean, it's a really interesting part. Actually, I've got a question about that later, so we'll come specifically on to kind of why now and whether it's right that it's that way around. Tory, could you give us a bit more of the other part of the question, I guess — what is the accessibility tree, and how is it different from the DOM?

Tory Gray: Essentially, the DOM has a lot more code in it. And so, like, if you're familiar with just, like, classic HTML, right, it's gonna have a paragraph tag. An accessibility tree is gonna be narrowed down to the fields that matter the most. So it's gonna be your headings, it's gonna be your page copy, it's gonna be your links, it's gonna be your buttons, essentially. And then there are some interesting things, like, if you're using CSS to hide your heading, for example, it won't be in the accessibility tree. So it's kind of a smaller, trimmed-down version that's a simplification with less code — or just, like, the important code, the most critical content of the page. It's very much a summary, or I think of it like an outline. If you're old like me and you remember, like, overhead projectors, and your teacher had something and they'd write on the thing and project it on the wall? It's like the outline. And it's a very quick summary. So, certainly, agents can get to the DOM tree as well, but the accessibility tree is a trimmed-up, summarized, like, "this is the critical information that you're conveying to your user." And, therefore, what AI agents need to know as well.

Patrick Hathaway: Brilliant. Okay. Well, I also think now if people look in the chat, Jojo has dropped a link which is called the accessibility tree. If folks go on there, they can find basically, like, how to see it in Chrome. Right? So if you've never done this, I think it's definitely worth doing. Going and looking — it will show you how to get to the accessibility tree in dev tools, and you can find it and look for yourself. And you'll see, you know, what Tory's just explaining. And you can compare that to the DOM yourself and see what's going on. So definitely worth doing that with that link just above.

Okay. So let's say when you open the accessibility tree on a client site, what are you checking for? Like, what's good versus bad?

Tory Gray: Good. This is something Jessyca and I have been talking about lately. I think we're still pretty early in figuring out what that means. But one of the things that I do know is that quote about the CSS hiding your headings? That's gonna matter. You can also use ARIA, like, labels on, say, links to make them show up or hide them from the accessibility tree. So I'd say, like, are your important buttons, links, text, and headings on your page included in your tree or not — those are my biggest takeaways. Jessyca, what about you?

Jessyca Frederick: Yeah. You know, looking at the tree the first time, it was really interesting to see, like, there's, like, duplication. And there's duplication because a lot of times we'll have an image with a link on it, and we'll put the alt text in the image. And then right below, it will have the same text in a link. And so, like, there's the sort of exploring what should be in there and what shouldn't be in there, to your point, and what do you need to hide. And I've kind of been thinking about it, like, not just in the "I see that I need to do something different with this element on the page to either make it more prominent in the accessibility tree or remove it." But how do I operationalize that? I mean, I can't go manually hit every single page on every single website and clean up the tree. And so I'm starting to think about, like, how do we apply those ideas and the concepts that the tree is surfacing for us as a matter of product as well.

Tory Gray: I also — Jessyca, you brought up that the accessibility tree that we see in Chrome is rendered. Which is not the same thing, which I thought was, like, such a good point. Would you mind elaborating a little on that? Because I think it's critical.

Jessyca Frederick: Absolutely. So, I hope that most of the people here know what client-side rendering is. Client-side rendering is when we use JavaScript to bring content into the page after it loads the HTML payload. And, you know, we use it sort of innocuously. We don't even necessarily think about the fact that it's there because Google's bots render JavaScript. So we don't have to really play out whether or not content is available on the page. We're thinking more about it for agents in general, but —

Patrick Hathaway: Yeah. So everybody go see the JavaScript SEO training, which now applies to all of this stuff as well.

Jessyca Frederick: I was looking at a PLP, a product listing page, on our website, and we use Algolia to bring in those listings. And I see them in the accessibility tree, and I know they're not in the payload. And I'm like, wait a minute. If you're relying on the accessibility tree as your guidance for all agents, it may not work for all agents. It may only work for Google. And then you need to go back to considering whether or not client-side rendering is a solution that's gonna be agent-friendly.

Patrick Hathaway: Yeah. I know. I think it's a really good point. And possibly why they've got, like, different methods, because they can't rely on one thing or another. You know? And like you mentioned before, it might not just be tokens — they might have a preference of one versus another.

Okay. So, I mean, this sort of ties into something you were saying before, Jessyca, but, like, does this collapse the wall a little bit between SEO and accessibility disciplines? And kind of following on from that, I suppose, realistically, how much accessibility knowledge does a technical SEO need?

Jessyca Frederick: The answer to that last part is: a lot. And it's sort of something that I'm paying more attention to in my day-to-day. I've always thought about accessibility when I was building websites, like, twenty years ago when I was building them by hand. I used the guidelines and I tried to make everything accessible. And at some point, we stopped really doing that, I think, in practice. We just didn't focus — I mean, certain websites are required to. A lot of government agencies in the United States, for example, require that the sites are accessible. And so there's, like, a discipline of people who focus on that. But I think for the businesses where it's not a legal requirement, I think it's much more challenging for us to put this on anyone but SEO, because nobody else is looking under the hood the way the SEOs do. So I think it does become an SEO discipline because we're the ones who are gonna be crawling the site, looking for those tags, making sure that we're making things available. And in organizations where AEO is also an SEO discipline, you know, same answer. Like, it has to be part of the SEO practice. So I think all of us in SEO need to get really good at accessibility, and it's complicated. I mean, even just the fact that there are different levels of standards and you don't know which level of standard you're aiming for and which one the agent needs — that's, like, an additional question.

Patrick Hathaway: Yeah. I mean, like, the sort of sad reality, isn't it, that we have accessibility standards, and what SEOs took away from that was, like, image alt text. Cool. We're gonna do image alt text. That's basically the only thing that we care about out of that. And you can sort of almost see the same thing coming. Right? So Sitebulb's got an accessibility option within the crawl, so it can do, like, 150 different checks — like, a crazy amount of checks all on accessibility. The way I can see this going is that we hone in on the sort of 20 or 30 or whatever it is that the agents care about, and they become the things that people optimize for rather than going, "oh, actually, accessibility is worthwhile in and of itself, and let's do that." And the side benefit is agents. It's gonna be the other way around. Right? Sadly.

Tory Gray: I mean, I know I've ranted about this to you, Patrick, already, but, yeah, to me, this just feels like another opportunity for SEOs to spam things. And just like we have spammed alt text for too long, and we're doing it wrong, and we're actually, like, harming the user experience for people that need assistive devices for the internet. I worry the same thing will happen to the accessibility tree. So even — there was an interesting conversation. The head of ChatGPT was — or I don't remember who it was. It was an executive. Right? He was talking about, "hey, maybe you should go and add all these ARIA labels to things so we can find things, for the agentic use case. And let's add them to buttons, for example." And some other executive from the accessibility side said, "whoa, whoa, whoa. Excuse me. Buttons have implied ARIA labels. Buttons are a thing with an established standard on the internet. Don't be adding labels and, like, spamming us with your tags and your keywords where you don't want them, because, again, you're creating a worse experience for the users." So yes, it's, like, a new thing in the SEO world, and, yes, we need to get better at it. But, like, let's all try to not be terrible spammers of the internet and actually implement it correctly and not cause harm to real users who could drive real revenue for our businesses. We want to be selfish about it. You know?

Jessyca Frederick: I think there used to be a benefit behind spamming the alt text. Right? It was: get your images. I mean, that was how you got your images into image search. Right? That was the real benefit. It wasn't the explicit accessibility benefit. I love that WordPress, when you upload an image, it asks you to think about whether or not this image is purely decorative, and it tells you don't put alt text in here if this image is just here to be pretty. And I love that, and I think that sort of thinking is what really needs to happen as we build out the use of the accessibility tree. It's, like, is this thing actually necessary? If not, hide it. Or, you know, is this thing actually necessary? Make sure it's clearly labeled. But, yeah, what would be the benefit of spamming a button, for example? Like — and I know people will do it. Like, I've been in SEM for twenty years. I know people will do it, but, like, why? Why would you want to spam a button unless you don't understand how the underlying technology works?

Tory Gray: I think that's it. I think there's just gonna be people that don't know, or who see an agentic benefit and don't give a crap about the accessibility workflow, and that sucks.

Jessyca Frederick: Probably already in development.

Patrick Hathaway: We're gonna need a Yoast SEO WordPress plugin for accessibility tree — yeah. If not, it will be now. So, I guess I just want to know as well: how comfortable do we think the average SEO is in dev tools? Like, is this gonna be a skills gap? Do you guys agree?

Jessyca Frederick: No. This is gonna be a skills gap. I mean, I'm a technical SEO. I have a programming background. I approach all of these tools very differently than the average SEO would. And I think if you're especially a content-focused SEO without a development background or a technical background, how are you possibly gonna make use of this? Like, it really does require you to be able to break down how the thing got on the page, how the thing got into the accessibility tree. It's asking a lot. And so — but I also don't think the developers in most organizations have the bandwidth to pick this up as part of their discipline. So, you know, I think it's gonna keep falling through the cracks except for really agent-focused websites. I think SEO needs to own it.

Patrick Hathaway: No. I mean, all of the developers are now just — Claude Code's doing all of their work, so they don't have anything to do. So that'll be fine. Sorry. Sorry, developers.

Alright. Okay. Tory, you mentioned earlier about semantic SEO. And, like, essentially, none of the advice they're giving is new advice. But we know — you know this better than most, probably — that most of the web fails at it. Right? Like, there's just crap everywhere. So do you think that the agent visibility, carrot or stick, actually changes the conversation with clients? Or dev teams.

Tory Gray: I do, specifically because I think we're, again, a lot more narrowed in on which parts of semantic HTML matter. So I think semantic HTML has been of interest for SEOs for twenty years, and it used to be one of the checks in tools. And then Google came out and was like, "no, we can figure it out. It's all good. Don't worry about it." People break that all the time. I mean, a classic example is Facebook's tracking ID — it's an image pixel that they tell you to put in the head of a website, which breaks your HTML, like, your head section of your site, and it's not semantic HTML, and it does not meet that W3C test. Right? So there's a lot of different things that go into what makes it semantic. And so what I'm seeing from this guidance and from the agentic tool — and that report specifically in Lighthouse, but only in Google's Canary, so it's a whole different browser you have to download — it is focused. It looks a lot at ARIA labels. That semantic HTML overlaps, frankly, a lot with the accessibility tree from what I can tell in the way that they're giving guidance for it. And so, again, the scope is significantly on a lot of ARIA labels. And therefore, what shows up in the accessibility tree and what doesn't. So — I mean, anytime you make something concrete and smaller, that makes the tickets much more clearly defined. And now that we have sort of executive interest and possibly a revenue use case for it, I think it could get onto a lot more roadmaps in a way that it did not historically. Again, for ways good and bad, for the spamming.

Patrick Hathaway: And on that, Jessyca, like, do you see this being, like, a dev education conversation, or is this something SEO should own — like writing semantic HTML?

Jessyca Frederick: So I work with the producers on our website, and I'm always training them about, like, using the right H1s in the right places. And we have heading-helper classes in our CSS so that you can observe semantic HTML and still style things however you want them to look. And so that's a training level. I think that's usually less of a problem for the developers. But, like, one of the things with semantic HTML that this conversation is making me realize I need to go talk to my developers right away about is that a lot of semantic HTML elements have implied ARIA. Like, you don't need to add ARIA labels to heading tags. The accessibility tree knows what those are and what they're for. So, you know, where to use them and where not to use them. But I think if it's not in my specifications, that's not getting on the website. And that may be the way we need to think about it. Maybe there's, like, a kickoff training and, like, "hey, this is what we're gonna be doing going forward." But I'm still gonna have to write it into every spec or it isn't gonna get built.

Patrick Hathaway: Okay.

Tory Gray: 100%.

Patrick Hathaway: Awesome. Right. Okay. So you were both, like, doing quite a lot of prep for this webinar. I was in email threads with you both back and forth. And, Tory, you mentioned that you had pointed a browser agent at a page and literally asked it what useful actions it could take. So could you, like, talk us through that? And I'm also really curious to know if you see this being, like, a standard client deliverable that agencies or consultants will be doing moving forward?

Tory Gray: Potentially. Like, it's not a meaty deliverable. To me, it's almost like a pre-deliverable, because the challenge that I'm facing is — you know, Google dropped this guidance and almost overnight almost all of my clients said, "okay, are we agent ready? Go audit my website." And, like, I had to go figure out what the heck that meant for them, and, in purposes of prepping for this webinar. So it's been a busy few weeks. It's been a lot of fun. And oh, god. Like, where do we even start? I'm getting lost. I'm scrolling in my own head for putting together the agent audit. So the biggest question I'm asking them is, okay, what do you want agents to be able to do?

Jessyca Frederick: Want me to jump in?

Tory Gray: Like, it's a straight question — if you want to be agent ready, if you want to be SEO ready, there's a series of questions I need to know and that I need to understand in order to audit your site properly to see if it meets specs. I have to define what those specs are and what the goals are. And right now, I don't think just about anybody knows what those actions are. And I need those defined both in terms of what do we want agents to do, and also what do we not want agents to do. Because let's not kid ourselves. Again, agents are going to help grow spam, and they're gonna help grow security issues. Right? So one of the ways in which we're helping our clients to define this — we're using the Atlas, yet another browser. I've downloaded so many browsers lately, but this is ChatGPT's agentic-specific browser. And it looks a lot like Chrome, except there's a little button on the top right corner, and it says, "hey, you should ask ChatGPT." So you can go to various pages on our client's websites and say, like, "hey, help me brainstorm. What actions could we take?" And so it doesn't take a substantial amount of time, and that's why I'm like, I don't know if it's a whole deliverable, because it's ChatGPT giving me a list of 75 things that an agent could potentially do on a specific page. So on the home page versus a product collection / listing page versus an actual product detail page. You have to view the different pages on your site and get ideas for those. And then I supply those to my clients, like, a sort of trimmed-down "this is what's meaningful" list, so they can start to have good conversations with their legal team, with their security team, with their cloud team, about what do we want them doing. And it was interesting, because, as an example, I ran it on Wells Fargo. So I just, you know, picked a big bank. And what does it say agents can do? Oh, it can help me as a user remember my email address for my account, or my password. Okay. Well, think of the spamming and security implications if our agents are gonna start doing that, because is that a way to essentially run an attack on a website through a new means? Whether that's intentional malicious behavior, or whoopsie-daisy, a user that doesn't know what they're doing accidentally sets an agent and the agent goes nuts and creates load on a website, and suddenly they have to deal with that. So that's my rant on: let's make sure we know what these actions are and define those, and then everything kind of follows from there. There are certainly, like, global considerations for agentic auditing. But then there's a lot of, like, very specific action — but what action?

Jessyca Frederick: So I love the security aspect of that question. And it's definitely something that needs to be investigated at any organization. I think one of the things that I'm particularly excited about about this idea of asking what can happen on a page is — as in product, I'm always asking the other teams, what is the purpose of this page? Who are the users for this page? And I'm always talking about bots as users, in addition to humans as users, because they are. Agents are now a new form of user. But, like, I think that if you get an audit of all the things an agent can do on a page, you might clarify for the humans making decisions about what the purpose of the page is that they're asking too much of individual pages. And I think this actually could be a leap forward for better UX design, because we don't want a page to have 4,000 purposes. We want a page to have two or three purposes so that the action you intend to have taken actually gets taken. I have found that agent readiness is an organizational —

Patrick Hathaway: Which also aligns with everything that everyone's been saying, you know, for the last few years about pages matching search intent — you know, user intent when they come to the page. You don't want to have pages that are trying to serve too many users from that side as well. So, okay. Yeah. We've covered that one. So — yeah, I had a question, but you guys already covered it. How efficient we're being. Brilliant. Okay. So Jessyca, I've got one for you. Let's say we've got an in-house SEO watching. Right? And they have, as everyone has, a finite dev ticket budget for Q3. Does agent readiness jump the queue, or is this, you know, wait-and-see-type land?

Jessyca Frederick: It is a priority. It is not just a dev priority. It is not just an SEO priority. There are so many elements. So, again, Wine Enthusiast has both a media site and a commerce site, and typically, most of the conversations I'm having internally are on the commerce side of this. And, you know, product data needs to be aligned to be agent ready. It's not just the technical part. Like, there's: what are we serving to the agents once we make it friendly for them to access? And so everybody has to participate. What is marketing's role in this? Like — SEO doesn't always report to marketing. Marketing often knows what the user intents are sometimes better than SEO, sometimes better than the merchandising team. Right? Like — and how do you coalesce all that information into one place, get it on the page, and make it accessible, which is really the dev requirement? So that's kind of where I'm stuck in a loop is just trying to figure out how to raise it to the organizational priority level and then backfill the priority of each individual ticket from there. But the other thing that I do in my practice — and it's because I am both product and SEO — the SEO goes into every ticket. The SEO tickets aren't separate. They are a part of every single thing. And so for my producer, when he gets a new task, there's an SEO review before he's allowed to do the work, because otherwise, you know, people who don't know SEO are calling the shots about what ends up on the website. So I think that this applies here too, in that your SEO tickets should be baked into your regular tickets. SEO should be looking at every ticket to see how it impacts the website. Increasingly, the feeds too. Like, I oversee merchant center feeds. Right? Like, that's an important element, and these things all tie together, and it's very hard to tease them out. So, in some cases, I would say certainly agentic and a lot of technical SEO really needs to move out of being their own tickets and being a part of every ticket.

Patrick Hathaway: Yeah. No. I love that. Like, it has to be an organizational thing. But let's just dig into it as well, though. Is this something that people should start doing now?

Tory Gray: Loaded question. I'll jump in and say, like — the challenge is, first of all, they already are. Because there's executive interest. There's stakeholder interest. Right? If you're a public company, your stakeholders want to see you being cutting edge. So I think that's a lot of what's driving it. It's not necessarily, to put a point on it, consumer interest driving it. There are not a lot of consumers out there building their own agents and actively taking action today. There are some early adopters doing some things. I've got a client who — there was a particular agent that attempted to take 2,000 actions in a series of ten minutes. Right? Like, one agent, or, you know, agents from one source doing a lot all of a sudden. But does that mean it's a revenue-driving activity today? Not substantially. I'm sure there are certain niche use cases where, if your industry is very cutting-edge engineers, then there's probably revenue opportunities for you. But for the big enterprise companies in the world, I do think they're gonna put this on their roadmap because it makes executives feel smart and cutting edge, but I don't think it'll net revenue in the short term. I do have a bias to think that we should focus on more like the B2B customer first. If only because those are the use cases that will probably drive revenue first. Because that's what ChatGPT and the LLMs of the world are supporting. Right? They want to make money. Therefore, they're building tooling to support Claude Cowork. And all these different — businesses will pay for it. And the business executives will tell their team, "hey, you better go figure out this process. Go figure out how to do it." And then, therefore, the revenue opportunities for that B2B use case — it's a shorter map to actually driving revenue versus an end consumer.

Patrick Hathaway: Yeah. We're going back to your Wells Fargo. Right? You're not really gonna get people going, "submit my mortgage application, please, AI." Right?

Tory Gray: Maybe one day, but that seems way overwhelming for the majority of mortgage applicants for sure.

Patrick Hathaway: Hopefully not. Hopefully not.

Jessyca Frederick: I think one way to demonstrate that an organization is gathering steam on the agent world is to apply these techniques internally. Like, a lot of us have a desire to have agents that can read across our data systems and synthesize reports for us on demand. Right? And that requires something that can go out and read your BigQuery database, and it needs to go look at the reports that you've already built and how to connect all these things together and then give you an answer. Right? So I think that, for organizations that want to see that they're moving forward on developing skills in their teams that correlate to agents, looking internal isn't a bad idea. You know, that's not the SEO world, but that's sort of for the organization that needs to demonstrate that they are paying attention to what's happening with agents, but there isn't a revenue use case for them at the moment.

Patrick Hathaway: Yeah. Totally. Okay, guys. I've got one last question. So then we'll move on to audience Q&A, of which there are plenty. So I'll just remind everybody — last question, last chance to get your questions in. You can do the upvoting thing as well. Like, if you see something in there, you're like, "oh, I want to know as well." Right. So mine was a quick one, but also respond in the chat if you have feelings about this as well. So let's say you were product manager for Sitebulb. Would you be pushing for Sitebulb to include an agent-readiness audit? And what would you want this to look like? And would you prioritize it over other things? Like, go. Let's do it right now. Jessyca, go first. Would you do it?

Jessyca Frederick: Yes. I would suggest it. I think for no other reason that the market is so hungry for understanding anything related to AI that it's an opportunity for you to demonstrate that you are forward-thinking and can support wherever the agent world is going.

Tory Gray: Yeah. I think at the end of the day, you deliver a service to your customer based on their demand, and they're demanding the hot shiny new thing, and therefore, it's your job to deliver it. What would I want to see? I mean, I think it'd be fascinating if you could — I don't know of another tool that can tell me an accessibility tree pre-render. Right? Because the tools that I've seen have all been browser-rendered. So what is an actual version of that? That feels like a huge opportunity. You already have great tooling around accessibility in general and the ARIA labels. That's how I knew what the heck that was before this guidance came out a few months ago. Or a month ago. I mean, it's pretty recent. And so I think you already have an edge there. And I think adding that would be the priority that I'd put on your list, because I totally have a say in your product roadmap.

Jessyca Frederick: I think also the list of actions to be taken. I think that would be good — listing out those actions. Yeah. Both, and then maybe even the side-by-side of humans versus agents, if that's possible. I don't think I have looked carefully enough to know.

Patrick Hathaway: Well, yeah, I think — James, early in our chat, he's one of our developers, suggested the response versus render for the accessibility tree. So it's essentially that same thing. Okay. Right, guys. Let's move on to the audience questions. So as I mentioned before, these are ordered based on upvotes first. So let's show on stage, and push these questions to our panel. So: "If we conclude this webinar, what points should we focus on when optimizing the website specifically for AI agents?" Okay. You guys are gonna have to shout who wants to answer.

Tory Gray: Do you want to go first?

Jessyca Frederick: So go ahead, Tory.

Tory Gray: So my biased opinion here is that the first step is figuring out what the heck you want to do. The second step is starting to audit your sites and starting to implement solutions for sort of the obvious global considerations. So, there's, for example, robots.txt. There's a series of, like, three, four checks that are well within the world of technical SEO and using tooling we already have. So XML sitemaps, robots.txt, and potentially — if you want to call it — llms.txt and how we use that for an agentic use case. All of those things sort of already exist, and they're a little bit more well known. So I think that they're concrete enough that you can take actions on them now. What bots should you allow access to your site? Whether it's through, you know, robots.txt — and they can choose to follow or not — or are you doing, like, IP fanciness? Then, again, defining what you want to do. And then starting to audit less of the global things and more of the page-specific things. So if you want agents to be able to purchase things on your website, then you have to go, maybe, to your product detail page, and you need to make sure that your content layout shift isn't bad on that page. Does it hit the range that Google is recommending? Because if the button is shifting — the add-to-cart button — then an AI bot may not be able to find it. So improving performance, making sure specifically that metric is good, and starting to think about the detailed recommendations — but that's where things start to get murky, because that's where we have a lot of competing standards. That whole underneath layer, where is it the Google — OKDs? I keep thinking OKRs, and now I can't — why do they have to make it too similar? But versus, like, I've seen skills.md files in the well-known directory. Right? Like, there's different methods, and that stuff gets murky, but I think you can and should do the global stuff first in preparation for anything you could do in the future. And that's my biased take.

Jessyca Frederick: I think the only thing I'd add to that — aside from knowing why you're doing it, which, as Tory stated, that is the very first thing: why are we doing this? But I think also one of the things that's often overlooked is that agents don't have a long attention span, and your site needs to be fast. So I'd be looking at optimizing time to first byte as part of any agent-friendly optimization, making sure that you're returning your payload quickly, that your payload isn't too large. Just the basic stuff of getting in the door, much less can they execute on the site. They have to be able to get to the site.

Patrick Hathaway: Good for users. Good for Google as well. So the next one is: "What are your thoughts on all the other emerging tech / experimental standards floating around with regards to agent readiness? E.g. NLWeb, web MCP, [unclear: possibly "MCP-UI agents"?], etcetera." Obviously, we've been talking about this a little bit. But have you been testing or recommending any of these? Any thoughts on this one?

Jessyca Frederick: So I am having traumatic flashbacks to VHS versus Betamax. And, you know, I think that you can't make a bet on what's gonna be the standard. I tend to bet on Google because Google's so big and so entrenched, and there's, you know, user switching costs. They're just not likely to leave Google if they don't have to. So I would probably prioritize, you know, Google's standard over all the others for that reason. But if your audience is a ChatGPT audience or a Perplexity audience, you know, really looking at the standard that you're focused on. There's a lot of discussion around sort of that, like, every LLM is gonna return a completely different answer for every prompt.

Patrick Hathaway: Every user is — yeah.

Jessyca Frederick: Yeah. And contextually too. Like, if you change your mind about something, you're gonna get a different answer next week from your agent. So predicting which agents matter and where to focus on standards is really difficult. And maybe the best course of action is trying to understand what's in common between the standards and making sure you're covering the bases there.

Patrick Hathaway: Cool. Alright. We've got lots of questions, so let's crack through them. Now I feel like we kind of answered this one already, but let's just see if there's anything else anyone wants to add. So: "Many brands already have huge dev backlogs. How much should agent readiness be prioritized right now given the low levels of agent traffic? Any website types / industries where you think it's more of a priority than others?" Again, I think we kind of covered this with my last question already before. So, yep, scroll back in time in the video. You'll find that one.

Alright. Let's move on to the next one here. So this is about publications. So: "If you implement a paywall, how do you balance the needs of agent accessibility with the needs of your own subscriber business model?" So I'm actually gonna suggest, for this question, you go back and watch the public webinar we did last month, which asks a lot of these sort of questions, from Harry, who was working at the Telegraph at the time, and Jesse, who was working at the Guardian. So literal publisher SEOs. So Jojo, if you could find the link to that, because we've got that published on the website. I think we're gonna go and find the answer in that one as well instead. Because we've got a few more questions in here we'll try and get through.

Tory Gray: While you pull that up, I could also say — again, agents are not AI bots. And it does not mean AI visibility. So we need to disentangle those two things. Like, they're different bots with different capabilities. So you as an org, as a publisher, are gonna need to decide what you want. In terms of, like, your information and training models, in terms of live fetching, and in terms of the agentic use cases. There are three substantially different use cases from the publisher perspective specifically.

Patrick Hathaway: Yeah. And, again, like, just to reiterate for everyone watching — that's kind of how we've ended up with this guidance that everybody's sort of like, "well, hang on, why are you saying llms.txt over here, and why not over there?" It's because they're for these specifically different use cases. So, like, almost the first point for anybody trying to implement it in the business is: understand what the different things are doing and who they're for and why we even have them in the first place.

Okay. So this next one then. "Have you ever tried 'Is It Agent Ready'?" Right. So I think you mentioned the Cloudflare tools earlier. "It scans your website for agent optimization and provides what your website lacks. Curious about your opinion on this tool." Anyone got thoughts?

Tory Gray: I've played with this tool a lot. This is the Cloudflare tool that I was speaking to earlier. It provides, I'd say, the Cloudflare perspective, and the standards that they are putting forth. So I think they're probably the best tool out there right now. It's much more substantially involved than Google's tooling. The agentic-readiness report — Google's version in Lighthouse Canary — is very shallow. It's very high-level, and it's not on a stable browser. It does not tell you a substantial lot of information. Cloudflare's tells you more, but it's from the lens of what standards Cloudflare is supporting. In addition to, like, informational ones about, say, the WebM— you know, another Google-proposed standard. So, I mean, Google's OKFs only came out on Friday, so they're not incorporated, but it's just about everything else. I will say, like most starter tools, it only has a yes/no. It doesn't tell you if you did a good job at a thing. Did you address agents in your robots.txt file? Do you have a robots.txt file? Yes or no. Do you address agents? Yes or no. It can't tell you, does it meet your organizational goals or not. Right? You still have to synthesize. And then the last thing I'll say is, again, these are a lot of proposed standards. And many of those are competing standards. You're never gonna get a 100 on this test unless you implement all the standards, because you have a dev team that really just, you know, is going for broke and needs to be so cutting edge they're willing to put those resources forth. So, like, a score of 100 doesn't actually mean you're doing a good job. Nor is it reasonable to expect a score of 100. That said, I use it. I like it. It's the best thing out there today.

Jessyca Frederick: So — and to counterpoint that, I'm unable to use it at our business because we have bot mitigation, which doesn't recognize — Cloudflare doesn't tell us what IPs they're checking from. They don't provide us with a user agent for checking, so I have no way to whitelist it in my bot mitigation service. So I don't know what it offers.

Tory Gray: And it's not the Cloudflare bots. Like — and so it's funny, because the clients on Cloudflare are not able to use the tool. Or, like, it can use it for some use cases. But it can see the robots.txt file. I have one example, but it could not find that the XML sitemap was referenced in the robots.txt, because of the user agent. It's interesting, like, edge cases of what it thinks it can and can't do.

Patrick Hathaway: It is quite sort of enjoyably ironic. I think we have to sort of battle Cloudflare all the time inside Sitebulb. Okay. So, last few minutes, let's just see if we can get through the rest of the questions. So: "You've said about using ARIA selectively to hide or not announce actions to bots that you don't want agents performing. How can you use ARIA for traditional accessibility but also limit agent overreach?"

Tory Gray: I would not use ARIA for this use case. If you don't want agents doing things — I mean, certainly, not suggesting that they do it in the first place is one thing, but also controlling bot access through IPs and only allowing bots that you've determined to be good, I think, is a better way to control for that. Use the right tool for the job.

Jessyca Frederick: I think going back to what you were saying about kind of breaking out the three different user groups of, you know, regular users, AI bots, and then, you know, agents — and how they're all different and need different use cases for them. It's another good example of, like, what we use our bot mitigation service for. It provides a header that says "is bot," and we could then, you know, have the page respond to that header and just literally not render things for agents. If we know it's an agent.

Patrick Hathaway: Right tool for the job. Okay. The final one — I don't know if you guys know. Perhaps this is something we need to create on Sitebulb, Jojo. "Is there any article or resource about the different types of AI search and their use cases?" Again, I think even the question perhaps is kind of confusing the different things that we've talked about today. So I guess the question really is: does anyone have a good breakdown that they know about, on, you know, any other site that we can just drop in as a link to go and have a read about all this stuff? But it clearly shows, doesn't it, that there is a need for this sort of disambiguation, I suppose.

Tory Gray: Absolutely. So I can speak to two. One, I have an article on gray.co about agentic auditing. It predates this brand-new, month-old information on agents, so it's only really speaking to the live fetching and the training-model bot differentiation, and not agents specifically, since that's so new. But that's one potential resource. The other thing I'd say is, like, ChatGPT has a page where they list their bots and the purpose of the bot. And for the most part, all the other LLMs have a version of that. So there is a different user agent for the browser, ChatGPT Atlas, versus the live-fetching bot, versus the search-index — that bot is the most confusing of all the bots. But just reading through the different versions side by side, I think, might help.

Patrick Hathaway: Great. Anything final to add on that, Jessyca? That's the last question.

Jessyca Frederick: There's a link that I was trying to pull up. I can't find it. If I can send it to you later, maybe it can go out with the email. There was a website I found when Tory and I were working on our article together that was really helpful for this, but I just can't recall what it is. But I'll find it in the source.

Patrick Hathaway: Great. Yeah. Okay. Great. Feel free to do the same thing if you can share it with us.

Tory Gray: Yeah. I'll post on LinkedIn with all my resource links — because I have so many. So I'll make sure to add that one to it. Great call-out.

Patrick Hathaway: Good stuff. Okay. Well, we are at the end of the webinar today. That's what we've got time for. So thank you.

Tory Gray: Thank you.

Jessyca Frederick: Thank you.