October 13, 2026

AEO & GEO

The 3 Technical Fixes AI Search Needs: Schema, llms.txt and Crawler Access

The three technical fixes AI search needs are structured data (schema), an llms.txt file and open crawler access. Most small businesses have incomplete or missing structured data, and many have none at all. These fixes are the foundation, and they are cheaper than ads.

What is schema, and why does AI care?

Schema markup is code that labels the facts on your page so a machine can read them without guessing.

The most common format is JSON-LD, a block of structured text placed in your page. It says, in a language machines understand: this is a business, this is its name, these are its services, and these are its hours. Without it, an AI has to infer those facts from your paragraphs and sometimes gets them wrong.

Start with the types that match what buyers ask about.

  • Organization or LocalBusiness: your name, contact details, service area and links to your profiles elsewhere.
  • Service: each thing you sell, described the way a buyer would ask for it.
  • FAQPage: the questions and answers on your page, marked up so they can be lifted cleanly.

How do you know if your schema is incomplete?

Run your key pages through a schema validator and compare what it finds against what you actually offer.

Incomplete usually looks like this. The homepage has a business name but no services. Hours are missing or out of date. There are no links to your profiles on other sites. Or the markup says one thing and the page says another. Many businesses have no structured data at all, and many more have partial markup that leaves gaps.

What is an llms.txt file?

An llms.txt file is a plain text file at the root of your site that gives AI models a short summary of who you are and links to your most important pages.

Think of it as a cover page written for machines. It is a proposed convention, and engines vary in how they use it, so do not expect it to work alone. It is cheap to publish. It also forces you to write down your business in a clear, consistent form, which helps every other layer of the work.

Can AI crawlers actually reach your site?

Crawler access means AI systems are allowed to fetch your pages and can see your content once they do.

Three things block it. A robots.txt file that disallows AI crawlers such as GPTBot or PerplexityBot. A firewall or security setting that treats bots as threats. And heavy JavaScript that loads your content after the page opens, so a crawler sees an empty shell.

Check your own robots.txt by adding /robots.txt to your domain. Then view your page with JavaScript off and see what remains. If your services and answers vanish, so do you.

Why do these fixes come first?

They come first because they decide whether AI can read you at all, which is the Found stage of the Match Protocol Score™.

Found carries 20% of the score. Many businesses are readable. But the structured-data gap shows how much room there is to be read clearly rather than barely. These fixes are the Machine-Readable Architecture layer, the M in Match Protocol™.

Be clear about the limit. Technical fixes make you readable. They do not make you recommended. Most businesses score low on the Recommended stage, and most have no FAQ content answering buyer questions. That gap needs the Targeted Extraction layer.

What mistakes do owners make with technical fixes?

The biggest mistake is assuming the website platform handled it.

Some platforms add basic markup automatically, and it is often thin or wrong. Another common mistake is blocking all bots to stop spam, which also blocks the systems you want. A third is stuffing schema with claims the page does not support. Markup should match what a buyer can see. Mismatches make you look less reliable, not more.

Review these fixes whenever you redesign or change platforms, because a rebuild can quietly undo them.

What order should you do them in?

Do crawler access first, schema second and llms.txt third.

If crawlers are blocked, nothing else matters. If they can get in, schema gives them clean facts. Then llms.txt gives them a summary and a map. Terms like these are defined in the glossary, and the broader program is described on the AI search optimization page.

Frequently asked questions

Do I need a developer to add schema?

Not always. Many website platforms have plugins or settings for basic schema. Complete, accurate markup that matches your page content usually needs someone comfortable with JSON-LD, and it should be validated after it goes live.

Will adding schema guarantee that AI recommends me?

No. Schema makes your facts machine-readable. Recommendation also depends on matching buyer questions, having extractable answers and being verified across the web. It is the first layer of five, not the whole job.

Is llms.txt required for AI search?

No. It is a proposed convention, and engines differ in how they use it. It costs little to publish and clarifies how your business is described, so it is worth doing, but do not rely on it alone.

Next step

Start with a fast read on where AI stands today. Book Your Match Protocol™ Strategy Session or Get Your Match Protocol Score™.

Clint Rogers is the founder of Modern Asset Lab and the developer of Match Protocol™.

Want to know what AI says when a buyer asks about you?

Get your free Match Protocol Score™ in about 90 seconds, or book a Strategy Session and we will walk through it with you.

Top Recommended or You Don't Pay