Website Management Tools
Search Engine Spider Simulator
A crawler first receives a page as HTML, without running its scripts or showing its design. Enter a page address and see exactly that: the response, title and description, the headings in order, the visible text, the links it can follow and the alt text of the images. It helps you spot content that a simple crawler would miss.
How to use it
- Enter the address of a public page.
- Press Simulate a crawler visit.
- Read the numbered sections from top to bottom and check that your important content, links and alt text are present.
Example
If the visible text section is nearly empty although the page looks full in your browser, the content is probably added by JavaScript.
The page is read from our server in one location. Some sites show different content to different visitors.
Why it matters
A crawler first receives the HTML, without running scripts or applying your design. The simulator shows the response, title, meta tags, headings and links in that raw form. If an important heading or link is missing here, search engines may not see it on the first pass, which is common when content is injected by JavaScript.
The page is requested once from our server with no cookies and no scripts run, much like a first, plain request. It does not reproduce how a particular search engine behaves.
Common mistakes
- Assuming Google never renders JavaScript. It can, but later, and it is safer for key content to be in the HTML.
- Missing a robots meta tag or canonical that changes how the page is treated.
Questions
Is this exactly how Google sees my page?
No. It is a plain HTML fetch from our server. Google can also run JavaScript and render pages, and it uses its own crawler. This tool shows the first, simple layer, which is still what most crawlers and many AI tools read.
Why is my text missing?
Usually because it is added by JavaScript after the page loads. Server-rendered content is easier for every crawler to read.
What do images look like to a crawler?
Mostly as their alt text and file name. That is why alt text matters for both accessibility and understanding.
Which links are listed?
Normal links to other addresses that a crawler can follow. JavaScript links, same-page links and email links are not followed and are left out of the list.
How is this different from the Page Snooper?
The Page Snooper lists everything it finds in detail. This tool shows a single text-only view of what a crawler would read, in the order it would read it.
Guides that go with this tool
Digital MarketingTechnical SEO Checklist: 14 Checks for a Healthy Website
Use this technical SEO checklist to check crawling, indexing, sitemaps, HTTPS, speed, mobile and structured data, based on Google's own guidance.
SEO GuidesCrawled, Currently Not Indexed: Diagnose a WordPress Page
Crawled, currently not indexed in Google? Check crawl dates, tested HTML and competing pages, then verify your WordPress page repair.
Get practical SEO tips by email
Occasional emails with new guides and tools. Confirm by email first; unsubscribe in one click.