How a crawler actually reads a page
When you visit a page, your browser does two jobs. First it downloads the raw HTML the server sends back, the plain text document with your tags and content in it. Then, if the page relies on JavaScript, the browser runs that code to build the rest of the page: pulling in data, inserting elements, sometimes assembling almost everything you end up seeing.
That second step is slow and expensive at scale. Google can do it, but on its own schedule, often with a delay of days while the page waits in a queue to be rendered. Many AI crawlers skip the step entirely. They read the HTML the server returns, take what is there, and move on. If something only appears after the JavaScript runs, they simply never see it.
What quietly goes missing
This is fine when the important parts are already in the HTML. It becomes a problem when they are injected on the client, after the page loads, by a script running in the browser. To a bot that does not render, the page arrives as a near empty shell and the script that would have filled it never runs.
The things most likely to be added this way are exactly the ones that decide how you show up:
- The title and meta description, set by a script instead of being in the document the server sends.
- The canonical tag (the line that tells search engines which address is the real one when a page can be reached at several), added late, so the bot cannot tell which version of the page is the real one.
- JSON-LD and other structured data (invisible code that tells a machine what the page is), painted in only after the page loads rather than written into the document from the start.
- In the worst case, your main content itself, loaded after the fact, leaving the crawler with a blank frame.
A 30 second check
You do not need any tools to find out which camp you are in. The trick is to look at the raw page, not the rendered one.
View the page source, with Ctrl+U on Windows or Cmd+Option+U on a Mac. That shows you the exact HTML the server sent, before any JavaScript ran. Do not use inspect element for this: inspect element shows the page after the scripts have done their work, which hides the very problem you are trying to spot. If your title, meta tags and schema are sitting there in view source, you are in good shape. If they are missing, a non rendering bot is likely missing them too.
If you are comfortable on the command line, fetching the page with curl shows you the same raw HTML a simple crawler gets. Same test, same answer.
The fix is to ship it in the HTML
The fix is not clever, which is the good news. Put the things that must be seen by a machine into the HTML the server returns, rather than having JavaScript paint them in later. That means your title, meta tags, canonical, JSON-LD and the key copy of the page are all present the moment the document arrives, no rendering required.
In plain terms, the page should arrive already built, not assemble itself in the visitor's browser. If you are on a no-code platform like Wix, Squarespace, Shopify or WordPress, this is mostly handled for you: the important tags are in the page by default, so you rarely have to think about it. If your site was custom built on a framework, it is a setting called server side rendering or static generation (serving the finished page from the server instead of building it in the browser), and it is worth asking whoever built the site to confirm it is on. The goal is simple: everything that matters should be readable without running a single line of JavaScript.
Why this matters more for AI than for Google
Google has spent years and a lot of money building the ability to render pages and wait for slow content to appear. It is not perfect, and the delay can hurt, but it usually gets there in the end. That history is why plenty of JavaScript heavy sites still rank.
Many AI crawlers do not have that budget or that patience. They take the HTML in front of them and leave. So serving your content in the source HTML matters even more if your goal is to be quoted by an AI assistant, not only ranked by a search engine. The cost of getting this wrong is higher in the place that is growing fastest.
Trust the HTML, not the JavaScript
The takeaway is short. When something has to be seen by a machine, trust what is in the HTML, not what JavaScript adds afterwards. A browser will run your scripts and reward you with a page that looks complete. A bot may not, and it judges you on what it can read in one pass.
This is one of the things we check when you scan a site with HowFindable, and the fixed files we generate are real files that live in your HTML, not a script that paints them in later. Put the important parts where every reader, human or machine, will find them, and the rest of your work finally gets the credit it deserves.