Site Maps: Let Search Engines Find Your Pages

With 40 million websites in existence, and more than 3 billion web pages indexed by Google at the time of this writing (July 2003), it's no wonder that more and more people are relying on search engines to find their way through the unruly world that the web has become.

Nowadays, it is crucial to get your pages indexed by the most important search engines. To maximize traffic to your site, you must make sure that all your internal pages are indexed, not just your main page (homepage).

Fortunately, you don't need to submit each of your pages manually. The most efficient way is to create a Site Map (a list of links to all the pages in your site) and link to it directly from your homepage.

How Will A Site Map Help Me?

Search engines find pages by "crawling" the web. They go through the code of all the pages in their database (also called index), following links to other pages and adding them to the database (in fact, more pages are added this way than by manual submission).

However, search engines have trouble following links from pages buried too deep within the directory structure of a site. A Site Map solves this problem by giving the engines access to the links to all your pages once they follow the Site Map link in your homepage. For more effectiveness, place your Site Map in your root directory (where your index page is).

Site Maps: Not Just for Search Engines

While some web users will find their way through your site by following navigation links or by using the search box, others will turn to your Site Map. If you design your Site Map carefully, it will not only be useful to the search engines, but to your human visitors as well.

Here are some pointers:

  • The Site Map should act pretty much like the table of contents of a book.

  • The Site Map must clearly show all the sections of your site, and the information contained in each of those sections.

  • Every item in your Site Map must be hyperlinked to its URL.

  • If it's not too long or cumbersome, use each page's TITLE as the link text, since this tends to increase the relevance of your site. Otherwise, use the word or the short phrase that best describes the content of the page.

  • Make sure that you place the link to your Site Map at a visible location in your homepage (users shouldn't need a map to find your Site Map!).

  • Don't get creative: simply call the link "Site Map".

  • Make your Site Map a simple text link. If you use javascript the search engines will ignore it.

How can I check if my pages have been indexed?

Once you have created and uploaded your Site Map and placed a link to it in your homepage, submit both your homepage and your Site Map page to the search engines. You will then have to wait until the search engines do a web crawl. In the case of Google, the largest search engine, this happens approximately once a month.

To check if a page on your site has been picked-up and indexed by Google's, go to www.google.com and use the "allinurl" command in the search box:

allinurl:yourdomain.com/yourpage.html

Where "yourdomain.com/yourpage.html" is the URL of the page you want to check.

To get a list of all the pages in your domain that have been indexed by Google, you'll have to use the "site" command, followed by your domain name plus a word (or group of words) that you know appear in all your pages (for example, a copyright statement or some footer text):

site:yourdomain.com commonword

If after typing this command you get a list of all your pages (or at least a significant number of pages that weren't in the index before), this will be a strong indication that your Site Map has been successful.

-----

You can freely reprint this article. Just include the following resource box at the end:

About The Author

Mario Sanchez publishes The Internet Digest ( http://www.theinternetdigest.net ), an online collection of web design and Internet marketing articles and resources. You can freely reprint his weekly articles in your website, ezine, newsletter or ebook.

In The News:

Top Tips to Improve Your Website SEO  Business 2 Community
What Is Enterprise SEO?  Search Engine Journal
How to Show the Value of Local SEO  Search Engine Journal
Sherry Bonelli  Search Engine Journal
SEO in 2020: Going Beyond Google  Search Engine Journal
SEO How-to, Part 1: Why Use It?  Practical Ecommerce
5 Ways SEO & Web Design Go Together  Search Engine Journal
Google's Advice on How to Hire an SEO  Search Engine Journal
Can SEO Be Made Predictable?  Search Engine Journal

Absolute & Relative Links How Do They Rank?

The question for this article is whether or not you... Read More

Google Page Rank Is Dead - Part III

HELP! My PR page rank is grey, call the development... Read More

Tops In Toolbars?

Most internet marketers are aware of, and probably use, the... Read More

The Value of Search Engine Marketing

Search engine marketing vs traditional offline marketing. Most often, traditional... Read More

Keyword Selection- The Dark Horse of Search Engines Optimization

Below are what I call the "10 Commandments" for Keywords.To... Read More

Link Popularity: Distribute Content, Not Just Links

You've spent many hours trying to increase your online traffic... Read More

Paris Hiltons Reputation is SHOT! Hows Yours?

With apologies for the cheap trick of mentioning Paris Hilton... Read More

The Search Engine Secret That Is No Secret At All

It's common knowledge - we all know that it is... Read More

How to Choose Keywords to Theme Your Pages and Boost Your Traffic

One of the most frequent questions I get asked is... Read More

10 Basic Rules for Where to Place Your Keywords

First of all, Google and most other search engines do... Read More

Why Fear? Now Change Your Filenames Without Disturbing Search Engines/Visitors And Rank Well

By now, you know that to rank well in the... Read More

Why SEO Will Make or Break You, Part 1

Today's article is about the wonders of SEO. SEO is... Read More

Adding Content To Your Site Every Day

If you have been a webmaster for more than a... Read More

Selecting The Title Of Your Link Exchanges

I want to touch base with selecting the title of... Read More

Getting Honest With The Search Engines

Getting Honest With The Search EnginesI spend a lot of... Read More

Increase Page Rank with Search Engine Optimization

Utilizing effective search engine optimization techniques will improve the page... Read More

Search Engine Optimization Lies & Misconceptions

In a perfect world, everyone would be honest.In a perfect... Read More

2 Lesser Known Ways to Brainstorm for Internet Home Business Keywords

Search Engine Optimization (SEO) doctrine states that you should always... Read More

Find Best Keywords For Your Site

Keyword optimisation is probably the most important thing that you... Read More

eMarketing Sabotage - Top 10 Steps To Kill Your Search Engine Marketing Practices

We at America Web Works find ourselves amazed at the... Read More

Search Engine Optimization - Free Tips and Help - Part One - The Title Tag

Search Engine Optimization is a widely misunderstood industry. Many webmasters,... Read More

The Myth of Guaranteed #1 Ranking in Search Engine Marketing

You've seen the ads: Guaranteed #1 Ranking! There are no... Read More

Yahoo Listing Still Worth It?

In October 2002, the Yahoo! portal changed the way it... Read More

Why Pay-Per-Inclusion Search Engines are Dying

A Pay-Per-Inclusion search engine is a service in which a... Read More

The Importance of Search Engine Optimization

Search Engine Optimization is a key to any successful internet... Read More