In the following article we explain why the Ryte check for home page indexed may not be correct, and how you can check if your site is indexed by Google. With the recent update of Yoast SEO (version 7.3+) you’ll notice in your dashboard a homepage indexing check from Ryte.
This check will assess if your home page is indexed by search engines or not. If all is ok, you’ll see a green dot beside the index-ability check by Ryte, you can skip this article, unless you’re interested in knowing more about how to make sure that Google and other search engines can index your website.
What if it says my site isn’t indexed?
We’ve had a number of users experiencing an error stating that their homepage is not indexed, indicated with the red dot, or possibly you’re seeing a grey symbol which indicates that Ryte isn’t sure about your page indexing status. Don’t panic if that is you, it’s very possible its an error on Rytes behalf. In the next section we explain how you can check if your site is indexed by search engines.
How Can I Check if my Site is Indexed?
There are several ways to check if your site is indexed by Google. In this section we show 3 ways in which you can check if the site is indexable.
1. Using a Google Search
Google offers its own solutions to check for site indexing. One of the simplest ways to check if a site is indexed is by using Google Search itself. To do this follow these steps:
- Go to google.com
- Type site:example.com
- Replace example.com with your website link (you can also do per page)
- Do not include HTTP or www.
- Check results.
If your site is showing up here (you’ll want to look for the home page specifically) then it’s been indexed by Google.
2. Using Google Search Console (Webmaster Tools)
If you have Google search console set up, you will be able to check which pages have been indexed from Google. There are 2 methods, one with the classic version and one with the new version.
Classic Search Console
With the old version of search console, you can view pages that are receiving traffic, this should be a strong indicator that the page is indexed. To view if your home page is getting traffic (or other pages) do the following:
- Go to Google Search Console
- Navigate to Search Traffic > Search analytics
- Set the top bar to “impressions” that is impressions from search results (note there is a delay of 3 days for results, so if you recently installed Google Search Console, wait for at least 3 days before checking)
- Click on pages, to view the impressions per page:
- Find the page “/” this will be your home page
- If your page is getting impressions, then it will be ranking for some specific keywords, and should be indexed by Google
New Search Console
You’ll want to use the new version of Google Search Console for this option.
Follow the steps below to view your indexed pages according to Google Search Console:
- To browse the new version of Google Search Console, click top left, “Try the new search console”
- Navigate to Status > Index Coverage > Click on “Valid”
- Scroll down and click on any of the valid links
- Then scroll down to see example pages that are indexed. Note that not all pages may show up here, especially most recently indexed pages, as Google Search Console takes some time to show newly indexed results.
Find out more from Googles Index Status Report tutorial.
3. Index Checkers
You can check out the various index checker tools that will verify if your site is indexed by Google. You can try the following resources, simply paste your website URL(s) and run the test.
Using the Greenlane Marketing tool you can check pages that are indexed with the Google API, if you’d like a more comprehensive check, use this tool:
What to do if your site isn’t indexed
If tried to tools above, your site is not showing up in search results, it’s possible that your site has not been indexed by Google. There are a few things you should check with WordPress to make sure that you aren’t blocking the pages from being indexed.
1. WordPress Settings
With WordPress you have the ability to block search bots to your site via the settings. It’s most likely you won’t want this setting turned on unless you’re making a private site. To make sure bots can crawl your site do the following:
- To to Settings > Reading
- Make sure checkbox for “Search Engine Visibility” is NOT checked.
- Update Settings
Your site should now be crawl-able. Check steps 2 and 3 to make sure there are no other reasons that your site could be blocked.
2. Check .htaccess File
Check your .htaccess file to make sure it’s not blocking Google bots. With Yoast you have the possibility to check your .htaccess file without going to your hosting panel. To do this go to SEO (Yoast) > Tools > File Editor (alternatively use the link below and replace example.com with your own website).
If you find anything related to “Googlebot” it’s likely that’s the cause of your problem.
Generally speaking most .htaccess files will be pretty basic. If you’re not sure about the configuration, speak with a professional or contact your hosting.
3. Check Robots.txt File
Incorrect usage of the Robots.txt file can also cause issues with your site. Using the same steps as above, you’ll also see a section for robots.txt, it’s possible that you don’t have one created, if so then this is not the problem.
If you do have one, check for the following code:
If either of the above codes are present in your robots.txt file then google will be blocked from crawling and indexing your site. Remove these from the robots.txt file and update.
Note! The presence of “Disallow” is normal in a robots.txt file, however the presence of the forward slash / is what will prevent the home page from being crawled.
A regular robots.txt file looks like the following for most users:
If you’d like to learn more about creating a robots.txt file, you can read the following Ultimate Guide to Robot.txt resource by Yoast.
You can also test if your Robots.txt file is blocking the home page of your site by going to the following link:
To test if the home page is blocked, leave the field at the bottom block and click “test”.
If all is ok you’ll see the button turn to “ALLOWED” in green, this means the page can be crawled and isn’t blocked by the robots.txt file.
4. Yoast SEO Index and Meta Settings
Yoast has a built in setting allowing you to prevent bots from crawling specific pages, posts, categories etc on your site. In most cases you won’t require this feature, unless specifically hiding content, for example pricing, or client specific content from your site.
To make sure your homepage (or any page / post) is not blocked in this manner, do the following:
- Go to the page in question
- Scroll down to the Yoast SEO Section
- Click the Cog tab “Advanced”
- Allow search engines to to show this page in search results “Yes”
- Should search engines follow links on this page “Yes” (unless you have specific reasons to block outbound links)
- Meta Robots advanced can be left blank (unless you have specific requirements for meta bots)
Learn more about advanced meta robot tags from Yoast here.
Now that you have cleaned up the site you’ll want to request that Google checks and indexes your site. Do to this you’ll want to submit your sitemap to Google for crawling. This can be done very easily with Yoast.
Submitting a Sitemap via Google Search Console
If you aren’t using Google Analytics and Google Search Console at this point, stop what you’re doing and sign up. With Google Analytics you’re able to track the traffic and how users use your site. With Google Search Console you can see what has been indexed by Google and any site indexing errors.
You’ll want to make sure that you’ve submitted your Sitemap to Google to increase exposure of the site. This will notify Google spiders and bots to crawl the site and index it.
Note the difference between indexing and crawling. A site can be crawled by Google, but not indexed, and vice versa. For example, you can add a noindex meta tag to a page, and it won’t be shown in Google search results however it is still able to be crawled by Google. Also a page may already be indexed by Google, but a tag is set for it to no longer be crawled.
Learn more about Meta tags in Moz’s Meta Data Article
To submit a sitemap do the following:
- Login to Google Search Console
- Choose the site property
- In the left sidebar, click Crawl and then Sitemaps.
- Remove any outdated or invalid sitemaps like sitemap.xml
- Click the Add/Test Sitemap button in the top right.
- Enter sitemap_index.xml into the text box that appears.
For full documentation check out the tutorial from Yoast. Make sure that your site does not have more than one Sitemap submitted as this can confuse Google bots.
Thats it. Once you’ve submitted your sitemap to Google, they will be notified to crawl your site.
You can expect them to index your site within 2-3 weeks (it may be quicker).
You can then start tracking your sites index status.
To check your indexing results do the following:
- Login to Google Search Console
- Go to “Google Index”
- Then Click Index Status
Here you’ll see the number of pages indexed by Google.
That’s it folks. Your site should be indexed by Google and working well.
If you’re still having troubles get in touch to see if we can help.