How to Make Google Crawl Inner Pages

When Google crawls the web, it indexes each page that it links to. During this indexing, the page is judged for things like its link profile, content, and value to searchers. It’s one of the steps in taking your page and giving it its PageRank. But not every page on the web is indexed, which can create a problem for your business.

How Does Google Index Pages?

Google uses a program called Googlebot to examine pages and determine what value they offer and what order they should be presented in for each search. Googlebot uses links to move from one page to the next, as well as Sitemap data from each site. New links, dead links, and updated links are adjusted on Google each time Googlebot crawls a page and updates the content on it.

What Happens to Pages that Aren’t Indexed?

A page that isn’t indexed doesn’t show up in search results. It still exists on the web, but it loses one of the best ways to get organic visitors.

Why Are Some Pages Not Indexed?

There are a number of reasons a page won’t be indexed. It could be a technical issue, quality issue or problem with the link structure of your site. The only way to find out is to check each issue off one by one, and fix the problems when they appear.

Which Pages Are Most Likely to be Indexed?

Pages that are near the top of your domain, like the homepage and pages linked to the homepage, are most likely to be indexed quickly. One of the reasons for this is that Google’s crawlers use links as gateways to index each page. A crawler follows a certain number of links on each webpage it crawls, most of the time.

How Can I Tell If My Pages Are Indexed?

Type a specific search query into the Google search bar to see whether your site is indexed. First, use “site:domain.com”. If you were checking for indexed pages on Facebook, for example, you’d type “site:facebook.com” and run the search. It will show some—but not necessarily all—pages indexed from your site. You can also use “site:domain inurl:<slug> to show whether a specific page is indexed or “site:domain filetype:<filetype>” to see whether a specific file type has been indexed.

If you use the Google Search Console, check the “Index Status” to see how many pages are indexed, and which URLs are blocked or removed. Check the graph to see how your site has fared over time. Google Search Console also has “Sitemaps” that shows how many pages in the XML sitemap were submitted and indexed.

How to Make Google Index My Pages

There are a number of reasons why your pages may not be being indexed. Check each possibility against your site. Some changes are quick and painless, while others may take a little more effort. Either way, it’s worth doing the work so you can start getting those organic visits to those pages from Google searches.

Length of Time

Sometimes the issue is simply that your website hasn’t been up long enough. New pages aren’t indexed the second they appear on the web. Expect it to take a little while for Google to pick it up. The more you link build, the more quickly this is likely to happen, which is one of the reasons why a good link profile is so important early on.

Check Your Response Codes

If your site is producing anything other than a 200 (OK) server response code, you need to make adjustments. Errors, redirections, and dead links won’t be indexed by Google. You can use an HTTP status checker to check your pages and see what codes they’re producing if you’re not sure. If there are problems, resolve them.

Duplicate Content

Duplicate content is a bad idea for your SEO. Google may choose not to index pages with duplicate content and even if it is indexed, it may have a negative impact on your PageRank. There are tools you can use to see whether you have too much duplicate content. Make adjustments to your content if it’s too similar and repetitive, and you may see an increase in the number of pages that are indexed. It can also have a positive effect on your SEO.

Internal duplicate content isn’t the only issue, however. If you have content that’s too similar to that of other sites, your indexing may suffer. Check sections of your text to see how many other sites have the exact same phrasing. One issue may be that you’re pulling too many quotes from other sites. Another may be that your wording is just too similar. When these problems occur, work to make your content stand out and vary more from content on the same topics that other sites publish.

Page Quality

Sometimes a page is of such poor quality that Google simply won’t index it. Consider this: Google is creating a list of search results that offer value to its users. If the algorithm it uses to determine page quality shows that there isn’t enough, it may choose not to offer that page in search results at all. If you feel quality may be your issue, improve your content, link up with reputable sites, and consider getting expert advice on improving your content and SEO.

Another page quality issue to consider is the length of time it takes your site to load. Sites that spend too much time loading cause visitors to navigate away, which can negatively affect your PageRank. Eventually, it may lead to your page being removed from Google completely. Google’s PageSpeed Insights Tool can give you help to determine whether your site has loading issues and a guide on how to resolve them.

Internal Linking Issues

Sometimes you create pages on your site, but don’t link to them from other pages. These orphaned pages can negatively impact your rankings and may not be indexed. Crawlers follow links to find new pages. It’s impossible for them to find pages with no links pointing to them. If you aren’t sure whether your pages are linked correctly, use a site crawler to figure out whether your internal linking is done well.

Updating your site so that Google indexes more pages will have a positive impact on your representation on the search engine results page. As you produce new content, keep the suggestions for making your site get indexed in mind. That way you can produce content that is SEO-friendly, useful for viewers, and likely to get indexed as soon as possible so you can start getting organic views from search results.