Do you want to index website fast on Google? Here is a list of actions to follow for your website to be crawled and indexed as quickly as possible.
Table of Contents:
How your content end up on Google search
Here is a simple explanation for you to understand what happens behind the scenes. There is a 3 step process until your content appears along with other search results:
- Crawling- Google bots visits your website
- Indexing- website/URLs added to Google’s database
- Ranking- your content appears on Google search results
All websites go through the same process and after that Google has hundreds of ranking factors/criterias on which position every piece of content will appear in SERP (search engine results page).
Check your Robots txt file
Make sure your robots txt file does not block any links you want to index. Only this little mistake can prevent Google bots from visiting and crawling your website. Usually you can access robots.txt file via browser: yourdomain.com/robots.txt
If you ever see
User-agent: * disallow: /
Means you have made a rule for all crawlers user-agent with asterisk symbol, to block entire site starting from root domain with the forward slash symbol disallow: /
If you see something like this:
User-agent: * Disallow: Sitemap: https://www.domain.com/sitemap.xml
You’re all good. Oftentimes the link to sitemap is included in robots.txt file in order to easily reach all website links.
Add your website to Google search console
Google search console is specifically created for webmasters to add new websites,monitor results and also identify problems (if any). You may not use it and expect to pick up your content from other sources like backlinks from other sites linking to yours or social media. However I don’t see any reason why anyone would like to slow down the indexing process.
Is your content unique and helpful
During years Google made a lot of updates on how content is ranked. If you have a piece of content, which satisfies the specific query in the best possible way compared to others, then your content will appear on search results higher because it is valuable. The main mission for Google is to show the most relevant, best answer possible for the users.
Basic technical content optimization
Make sure your content is optimised:
- Title- use 50-60 characters to name your creative work. Longer will be cut off and may make no sense. This short text is important and helps people decide: click on your website link or scroll and find more relevant.
- Meta description- describe your content and use up to 155-160 characters. Often times Google will rewrite it to look more relevant.
- Images with relevant names and alt tag.
- SEO friendly URL- descriptive slug with a keyword.
- Schema markup- describes all entities on a specific page for Google bots.
Check out my guide: How to add Structured data
Check for orphaned pages- these are pages separated from others there are no internal links from those or to them from other relevant content on your site. In this case, if Google bot will land on such a page, it has nowhere to go, just crawl existing content and leave.
Internal links to index website fast
It is best practice to link every piece of content to another relevant content on your website. Great example is Wikipedia: some pages have hundreds of internal links back to another piece of content expanding depth of topic.
P.S. Make sure all internal links are using do follow attribute. If not,- crawler may ignore it. You can quickly check that: open any of your pages, right mouse click and pick “inspect”. Find an arrow on the new dialog window and click on it, then point it on the link you want to check on your website.
Request indexing in Google search console
I assume you have done all of the above and some time passed, still some content is not indexed. Or another situation is when you update your old content and want to index it faster. You can use Google search console built- in feature request indexing. Just open your GSC dashboard and find the “URL inspection” tab. Paste the link and see what it shows. If the crawler hasn’t visited your specific URL, you can click on request indexing and add it to the priority queue.
RankMath instant indexing plugin for WordPress
If your website is made using WordPress content management system you can use a great plugin Instant indexing for Google by RankMath. This plugin uses Google instant indexing API, meaning every time you publish new content it will be added to the priority crawl queue.
Allow at least a week for new content to be discovered and if it is still not there use Request indexing feature to fix the issue.
This may happen for a variety of reasons:
Your website might be hacked and you need to clean malicious code/content.
Your website got a penalty for link spamming
Your website removed for low quality/duplicate content
Your website does not show up because your not following Google quality guidelines
Open your favorite browser and go to Google type in the search
You will see if your website appears in results and how many of your pages are in search.
There are nearly 2 billion websites worldwide and Google might find it difficult to index all of them fast. Lately, one more big problem is AI generated content. People try to exploit/manipulate SERP by creating loads of content using this technique making hard times for Google to crawl and index websites faster.