In the digital world, before ranking a website, search engines must know what your site and its contents are all about. This is where search engine crawlers do the job by navigating your site and indexing each page. Understanding how Google crawls, indexes, and ultimately sees your pages is crucial for online success.
For years, most SEO companies and professionals have typically relied on the Google Search Console as their primary interface for gaining these crucial insights. However, a new feature within the GSC, the URL Inspection tool, has emerged, offering an unprecedented level of granular detail about how Google’s crawlers interact with individual pages on your site. Let’s dive in and learn more about this feature in detail.
The Google Inspection Tool, officially known as the URL inspection tool, is a diagnostic tool offered by the Google Search Console. Using this feature, you can submit individual webpages to Google and simulate how the search engine crawls and indexes a URL on their site. This URL testing tool provides information about Google’s indexed version of a specific page, and also allows you to test whether a URL might be indexable. Information includes details about structured data, video, linked AMP, and indexing/indexability.
The Google URL inspection tool enables you to view how Google is viewing your site live and monitor the technical changes at will. The first option with the URL inspection tool is to check individually if Google indexes the web pages. In the search console inspection tool, you simply enter the URL, and it will tell you the status of your page. You can do an inspection check by opening the Google Search Console and entering the URL to be checked.
As explained in the introduction, search engines discover and organize website content through crawling and indexing. If any of your webpages aren’t indexed, they won’t appear in search results. The URL inspection tool gives a clear idea about any indexing issues that can prevent your site from ranking. It allows marketers and SEO specialists to uncover detailed insights into how search engines perceive webpages on a site. These insights can be missed if you opt for traditional SEO methods. Some of the reasons why this game-changing tool is important are as follows:
Now that you have an idea about the URL inspection tool, let’s understand how this tool works in the real world. If you are wondering how the Google URL Inspection Tool works, you have to first submit a specific URL from your verified GSC property into the tool. Upon hitting search, you will be presented with a similar image as shown.
You can see that the various checklists with a green tick. Let’s go through each of them.
The test live URL in a Google inspection tool is very useful to fetch the page live and see what Google does when they fetch it. It can be diagnosed to
When you submit a page to the Google URL inspection tool, it will test the page for any warnings or errors you should know about.
So, these are the things that you can expect when you submit and inspect your site using the URL inspection tool. Let’s now move on and explain each section in detail.
As highlighted in the screenshot, the purpose of this report is to determine whether the particular URL appears on Google search results. It will offer any one of the following results shown below:
This effectively means that the URL has been successfully crawled and indexed, and it appears on search results without issues. If your URL gets a status like this, no action is required. However, if the URL isn’t ranking, tweaking the content, adding more internal links, etc, helps. The URL can appear on search results unless a manual action or a website removal request has been submitted.
This means that the URL was crawled and indexed, but the search engine has found issues with its enhancements. It is important to identify any issues and fix them as needed. Running a Googlebot crawler test on the affected URL might provide more specific clues.
This situation occurs when google crawled but not indexed your site, and hence, won’t appear in search results. The reasons can be many, such as having a noindex directive, disallowing in your robots.txt file, etc. You have to find what is preventing search engines from completing the Google crawling test successfully and indexing your site. A thorough check using a URL testing tool can often reveal these issues.
When you submit your URL and get this, it tells that your website ran into indexing errors and therefore, wasn’t crawled and indexed by Google, hence it doesn’t appear in search results. Similar to “URL is not on Google” status, you have to investigate and find out what’s preventing search engines from crawling and indexing your site. A Google crawler checker can sometimes help identify common error patterns.
This status tells that the site URL you submitted for inspection is an alternate version of your mobile or desktop website. If your site is a mobile one and you submitted the desktop version of it or vice versa, you can encounter this status. It’s a good idea to check Googlebot crawling on both versions to ensure they are correctly inspected. Using a crawl checker helps confirm the right version is being indexed.
If you need Google to index a URL for the first time or if you have made changes to an indexed page, then you can request the Google URL inspection tool. If Google does not come across any errors, the live URL would be added to the priority-based crawl queue.
If you are a crawler junkie and you analyze the bot activity and crawler activity in your junk files, the Google URL inspection tool may show up. This is especially true if you are using the rich result test and URL inspection in the Google search console.
If you see issues with these tools doing their jobs, you may be blocking the Google crawler user agent tool from accessing your site. Ensure that the Google URL inspection tool crawls your site if there are strict rules in place.
After the indexation session, you will see a section named coverage. In our example screenshot, you can see that it says “Submitted and indexed”. This means that the website URL submitted for inspection was properly indexed. Upon clicking the coverage section, it shows how Google performed well in these three sections when inspecting the site URL.
This section shows how the URL was discovered by the search engines and consists of the following sub-fields:
The purpose of sitemaps is to showcase which XML sitemap is linked to the URL that you searched for.
The referring page section showcases how Google’s crawler user agent is crawling or indexing your site. It also tells how Google may have used this page to find the URL that you just searched.
This section describes the various parameters that occurred when Google crawled the URL the last time. It lists the following fields:
Using the Google crawler test, you will have an idea of how the page was crawled by Google. You may switch between HTML screenshots of the page and relevant information about the type of content, the HTTP response, and the page sources.
The purpose of the indexing section is to describe the canonical URL that Google found for the webpage and its outcome. It lists the following fields:
The canonical URL indicates the main URL among a group of similar URLs leading to the same content. In the inspection tool, this indicates the canonical URL that has been declared by the user for the URL.
This field specifies the URL that was selected by Google as canonical. In our above example, if the search engine has inspected the URL as canonical, the status shows as “Inspected URL”.
Within the URL inspection tool, after reviewing the coverage section, you’ll find the Enhancements section below it. The purpose of this is to report when Google has found any issues related to various factors such as AMP, Mobile Usability, Schema implementation, and whether improvements must be made or not.
These Enhancements will only appear for URLs that have been successfully indexed by Google; the findings are automatically updated after the search engine crawls and indexes the page as part of its ongoing Google crawler test.
If the inspected URL passes the mobile-friendly assessment during the website crawl test, the status will be shown, “Page is mobile-friendly.” This is especially important in the era of mobile first indexing, where Google primarily uses the mobile version of a site’s content for ranking and indexing.
The tool actively scans the URL for any implemented structured data. If a valid schema is found, it will be detailed in the report. Otherwise, if no enhancements are detected, the status will show as “URL has no enhancements,” acting as a quick Google crawler checker for structured data implementation.
The primary purpose of the URL inspection tool is to provide detailed information about a specific URL indexed by Google. You can see if the search engine has crawled and indexed your page, whether any indexing issues were identified, etc.
To check if Google can crawl my site, you can utilize a Google crawler test within the GSC.
This is a new feature within the Google Search Console. A URL testing tool allows you to analyze a specific page’s presence and potential issues in Google’s index.
You can verify a URL by performing a Google crawling test using Google Search Console.
Even though Chrome DevTools offers technical insights, the dedicated Google URL inspection tool is necessary for a true Googlebot test to view the search engine’s perspective. This acts as a crucial website crawl test and seo crawl test. It’s the best way to check Googlebot crawling, essentially a Google crawler tool and a vital crawl checker or Google spider test. This process helps you understand what is the purpose of the URL inspection tool in detail and serves as a Google crawler checker.
For more such technical blogs, Stay connected with GTECH.
Every aspect of digital marketing is reigned by any kind of content as you need…
Blogs and product pages are part of every business, and yours is no exception. To…
If you are a businessman who spends lots of effort and money on digital marketing…
B2B platforms and B2C platforms are closely related, but there is a substantial difference in…
Being humans, we love any sort of visual stimuli and respond to it compared to…
Content marketing captures your target audience’s attention, and having more mere content does not benefit…