Google generates organic search listings from a database of hundreds of billions of web pages. Adding pages to its database is Google’s first step in determining rankings.
In Google terms, “indexing” means putting URLs into a database along with important information (on page (headings, body text, meta tags) and off (internal and external links, text surrounding those links, author information)). means added. Google uses that information in its search results and updates the index regularly.
Companies can block pages from Google’s index using robots.txt commands, but this is not guaranteed and Google can use off-page information to index the page.
Given the size of the index, it can take several days for Google to discover and add new pages. Also, new pages are indexed faster than updates.
How to index a page?
Google typically finds and quickly indexes pages on websites with less than a few thousand URLs, provided there are internal links to each page. Similar to submitting an XML sitemap in Search Console, external links speed up discovery.
Once indexed, Google will periodically visit your site to check for changes.
Google allows requests to recrawl your site. However, frequent requests often indicate a technical problem. So instead of re-crawling, audit your site to make sure your internal links are valid and easily crawlable. Third-party crawling tools such as Screaming Frog can help.
XML sitemaps are especially useful for large database-driven sites with thousands of product pages and hundreds of categories. Sitemaps allow Google to easily access deeper pages and report structural or indexing issues (via Search Console).
How to check index?
There are two ways to check if Google has indexed your page:
- Search on Google Site: Full URL — For example, site:practicalecommerce.com. If indexed, the URL will appear in organic search results.
- Check the “Pages” report in Search Console.
Google can index your pages, but they cannot cache them.
For indexing glitches, keep an eye on the Pages tab. For a list of pages not indexed by Google and their causes, see[クロール済み – 現在インデックスに登録されていません]Scroll down to Reports. A report usually contains some pages that are not indexed. However, if the number is increasing or exceeding a few, check the site for underlying issues.
Browsing through that report, I often find URLs that are actually indexed. This indicates that the report is outdated.
Nonetheless, unindexed URLs can indicate a wider problem. So eliminate that possibility before requesting a re-crawl.
The “Crawl – Not Currently Indexed” report lists pages that are not in Google’s index and why. Click image to enlarge.
How to speed up indexing?
The most common (and justifiable) reason to speed up indexing is to validate fixes. In other words, Google reported the error via Search Console and you fixed it. If so, request indexing with the URL Inspection tool in Search Console.
Google will send you an email once your fix has been verified and your page has been indexed. It usually takes several days.
Use this tool only occasionally, not every day. If you are re-platforming your entire site to a new domain or content management system, do not use this tool to re-index page by page. Instead, submit your updated sitemap in Search Console. This allows Google to index your site quickly.
After fixing errors on your page, you can use the URL Inspection tool to speed up indexing. Click image to enlarge.
