Google-Inspection Tool: The New Google Crawler For Google Testing Tools

Google has released a new crawler, a new Googlebot termed a Google inspection tool. The new Google crawler test is going to specify how Google identifies a crawling activity used by Google’s search testing tools, like the rich test results and Google search console URL inspection tools.

The Google URL inspection tool is a diagnostic tool offered by the Google search console. You are allowed to submit individual pages to Google and stimulate how Google crawls and indexes a URL on their site.

The URL inspection tool enables you to view how Google is viewing your site live and monitor the technical changes at will. The first option with the Google URL inspection tool is to check individually if Google indexes the web pages. In the search console inspection tool, you simply enter the URL, and it will tell you the status of your page. Just proceed to the Google search console and type the URL that you want to inspect. Let us figure out all the available options with the URL inspection tool.

Check indexation status

If it says

• If the URL is on Google, then the page is indexed.
• If the URL is not on Google, then the page is not indexed.
• URL is not on Google (error status) The page is not indexed because of an issue.
• URL is on Google and has issues. page is valid and indexed but has a warning.

Test the live URL

The test live URL in a Google inspection tool is very useful to fetch the page live and see what Google does when they fetch it. It can be diagnosed to

• Diagnose the rendering issues, and there is a suggestion to use the rich testing tool in this regard.
• Check the indexability status of an URL at a given time.
• Check for page loading issues.

Viewing the crawled page

Using the Google crawler test, you will have an idea of how the page was crawled by Google. You may switch between HTML screenshots of the page and relevant information about the type of content, the HTTP response, and the page sources.

URL coverage

Right after the inspection results, there is a section referred to as coverage where you come across the status of the page. This plays an important role during the course of the Google crawler user agent process. It makes you aware of whether the pages are properly indexed, among a host of other things. It reports on

• Discovery: How the URL was discovered
• Crawl: The crawl status conveys the last time Googlebot crawled the page.
• Indexing: what is the canonical URL chosen for the URL?

Sitemaps

The section on sitemaps is a straightforward process. It simplifies and goes on to showcase which sitemap is linked to the URL that you searched for.

Referring page

The referring page section below sitemaps is a way to showcase how Google’s crawler user agent is crawling or indexing your site. The section goes on to convey how Google may have used this page to find the URL that you just searched. You could get a message saying that Google may have found your page from other sources, such as a referring page or sitemap, but is unable to display the information at this point in time.

Understanding errors and warnings

When you submit a page to the Google URL inspection tool, it will test the page for any warnings or errors you should know about.

• Warning: These statutes may require your attention, but it does not mean that your page is not indexed or found or that any action may be necessitated from your end.
• Error: Google does not index pages with errors. When there is an error related to an URL, you have to take action in order to correct it. This would increase the chances of it appearing in the search results. You need to be aware of the specific errors.

Requesting Google to index a URL

If you need Google to index a URL for the first time or if you have made changes to an indexed page, then you can request the Google URL inspection tool. If Google does not come across any errors, the live URL would be added to the priority-based crawl queue.

If you are a crawler junkie and you analyze the bot activity and crawler activity in your junk files, the Google URL inspection tool may show up. This is especially true if you are using the rich result test and URL inspection in the Google search console.

If you see issues with these tools doing their jobs, you may be blocking the Google crawler user agent tool from accessing your site. Ensure that the Google URL inspection tool crawls your site if there are strict rules in place.

For more such technical blogs, Stay connected with GTECH.