Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future: use mysqli or PDO instead in /home/a26f9f83/public_html/articles/includes/config.php on line 159
Search Engine Robots - How They Work, What They Do (Part II) > NetSparsh - Viral Content you Love & Share

Search Engine Robots - How They Work, What They Do (Part II)

If your site isn't found in the search engines, it is probably because the robots couldn't deal with it. It could be something as simple as not being able to find the site, or it may be more complicated issues involving the robot's not being able to crawl the site or figure out what your pages are all about.

Submitting your site to the major search engines: that will help with the "can't find it" problem. Even having links pointing back to your site can be enough to attract the search engine robots. Google, for example, suggests that you may not have to submit your pages; they will find your site if you have a link pointing back to it from at least one other site on the web.

If the robots can find your site but can't make sense of it, then you may need to look at the content and technology used on your pages. Frames, Flash, dynamically generated pages, and invalid HTML source code can cause problems when the search engine robot tries to access your web pages. While some search engines are beginning to be able to index dynamically generated pages and Flash (e.g. Google and AllTheWeb), use of some of these technologies can hinder your ability to be indexed by the search engine robots.

Text in images cannot be read by the search engine robots. Using ALT image text is an important way to help the robots "read" your images. Websites with extensive images rely heavily on ALT text to present their content. How Do I Get The Most Out Of Indexing?

If you know what to "feed" the spidering robots you will help yourself with search engine ranking.

Having a website full of good content is the major factor. Search engines exist to serve their visitors, not to rank your website. You need to be sure to present yourself in your site in the way that will be most useful to the search engine visitor. Each search engine has its own idea of what is important in a page, but they all value text highly. Making sure that the text on your pages includes your most important keyword phrases will help the search engine evaluate the content of those pages.

Making sure that you have good title and meta tags will further assist the search engines in understanding what your page is about. If the text on the page is about widgets, the title is about widgets, and the meta tags are about widgets, the search engine will have a pretty good idea that you are all about widgets. When their visitors search for widgets, the search engines know to list your site in the results.

A sitemap page is a very good way of giving the search engine robot every opportunity to reach your website pages. Since robots click through the links of your web pages, make sure that at least your most important pages are included in the sitemap; you may even want to include all your pages there, depending on the size of your site. Be sure to add a link to the sitemap page from each page on your site.

Another important consideration is that of keeping all of your pages within a small number of "clicks" from your top page. Many robots will not follow links more than two or three levels deep, so if your "widgets" page can only be reached from your home page by following multiple links (e.g. home page >> about us page >> products page >> widgets page), the robot may not crawl deep enough to get to the widgets page. Testing Your Website For Search Engine Robot Accessibility

To get an idea just what the search engine robot "sees" on your page, you can look at the Sim Spider tool. You may be surprised at how different your site looks to the robot. You can find this tool at http://www.searchengineworld.com/cgi-bin/sim_spider.cgi

You will see text and ALT image text show up in the results. If your entire website is built in Flash, you will see nothing at all because robots don't understand Flash movies. The Bottom Line

When it comes to search engine robots, think simply. Lots of good content and text, hyperlinks the robots can follow, optimization of your pages, topical links pointing back to your site and a sitemap will help insure the best results when the robots come visiting.

Resources

*SpiderSpotting - Search Engine Watch http://searchenginewatch.com/webmasters/spiders.html

*Robotstxt.org List of robots and protocols for setting up a robots.txt file. http://www.robotstxt.org/

*Spider-Food Tutorials, forums and articles about Search Engine spiders and Search Engine Marketing. http://spider-food.net/

*Spiderhunter.com Articles and resources about tracking Search Engine spiders. http://www.spiderhunter.com/

*Sim Spider Search Engine Robot Simulator Search Engine World has a spider that simulates what the Search Engine robots read from your website. http://www.searchengineworld.com/cgi-bin/sim_spider.cgi

Daria Goetsch is the founder and Search Engine Marketing Consultant for Search Innovation Marketing, a Search Engine Optimization company serving small businesses. She has specialized in Search Engine Promotion since 1998, including three years as the Search Engine Specialist for O'Reilly Media, Inc., a technical book publishing company.

Copyright © 2002-2005 Search Innovation Marketing. http://www.searchinnovation.com All Rights Reserved.

Permission to reprint this article is granted if the article is reproduced in its entirety, without editing, including the bio information. Please include a hyperlink to http://www.searchinnovation.com when using this article in newsletters or online.

In The News:

This RSS feed URL is deprecated, please update. New URLs can be found in the footers at https://news.google.com/news

Fstoppers

Better SEO: Finding Keywords for Your Photography Website
Fstoppers
If you know anything about SEO (search engine optimization), then you know that finding good keywords is, well, key. Here are a few ways to find new keyword ideas and how to implement them on your website. Very little is known about Google's algorithm, ...
How to Balance SEO Strategy with Brand StorytellingThe Content Standard by Skyword
A Complete SEO Checklist for Website OwnersSearch Engine Journal
How to Optimize Content Efforts for SEOMarTech Advisor
Bankless Times -PR News - For Smart Communicators (blog) -O'Dwyer's PR News
all 8 news articles »

Search Engine Land

How to project SEO traffic levels and avoid saying, 'It depends'
Search Engine Land
Have you ever had this conversation with someone from upper management? Boss: “If we implement your recommendations, how will this impact traffic?” You: “It depends.” Boss: “If we create a new content project, will it generate a lot of traffic? You ...


Search Engine Land

SearchCap: Google Assistant ups its game, SMX East, SEO traffic projections & more
Search Engine Land
There is no such thing as being negative SEO-proof, says contributor Joe Sinkwitz. All you can do is take steps to lessen the probability of becoming a victim. Here's how to reduce attack vectors and protect your site. Ask an SMXpert – Google Tag Manager
How Do SEO and Google AdWords Work Together?Business 2 Community

all 13 news articles »

Forbes

Amazon SEO: Seven Factors For Ranking Highly On Amazon
Forbes
Amazon can be a highly lucrative yet competitive way of selling your products. You're competing with other sellers for visibility and high rankings in the search results pages. A placing on the first page is important as 70% of Amazon shoppers (paywall ...


Forbes

Google's Top Ranking Factors For Local SEO
Forbes
Relevance: First On The Local SEO Checklist. Relevance is a staple of Google's algorithm and attorneys, business owners and marketers need to make their GMB profile as relevant as possible to what searchers are using to find them. Here's how you can do ...
Infographic: 72 Stats to understand SEO in 2018AMEinfo

all 17 news articles »

Business.com

Should Your Business Hire an SEO Expert?
Business.com
The rudiments of good SEO are relatively easy to grasp, but it takes a lot of experience and expertise to be a master. There are plenty of resources to support your SEO efforts, though, including courses and tools, many of them free. Or you can hire a ...

and more »

Forbes

6 Reasons Why SEO Might Not Be Right For Your Business
Forbes
A lot of people think that search engine optimization (SEO) is a business staple, like email, desks, and, well … staples. But in reality, SEO is not something every business needs, especially small businesses with modest marketing budgets. If any of ...
Small Business SEO Checklist: 11 Ways to Improve Rankings TodaySearch Engine Journal

all 2 news articles »

Search Engine Journal

7 Signs Your Law Firm May Need a New SEO Partner
Search Engine Journal
The legal industry has a serious problem. Law firms and attorneys around the country continue to get taken advantage of by “churn and burn” SEO agencies. These agencies, which are run by self-proclaimed SEO experts, in most cases, promise big results ...


AI & SEO: Keeping Up with Google Rankbrain in 2018 and Beyond
MarTech Advisor
Over the years, SEO has gone through several transformations. Starting from simple keyword targeting through to complex website optimization processes, to the ever-dynamic tactics to optimize content across multiple devices, SEO rules are ever changing.


Search Engine Journal

Video SEO for Universal Search: Tips, Tools & Techniques to Get Found
Search Engine Journal
This trend may surprise SEO and online marketing professionals who haven't been able to escape proclamations of Direct Answers being the great new opportunity for boosting free, organic traffic. What the trend shows is that users weren't always happy ...

and more »
Google News

Soliciting Search Engines

As your guide operator through the web, search engines are... Read More

SEO for CEOs ? Search Engine Optimization Unmasked for CEOs

If you're like most other CEOs, the term "search engine... Read More

Developing A List Of Keywords For Marketing

Keywords aren't just some words that allow search engines, like... Read More

SEO - Are You Making The Search Engines Mad?

If you've been involved in SEO (search engine optimization) for... Read More

Link Building in Light of Vision-based Page Segmentation

The days of basing a successful link building strategy on... Read More

Google Sitemaps Just Got Better

Having a Google Sitemap just got better! Not only does... Read More

Maximize Your Search Engine Traffic - 13 Ways to Pull in More Visitors From the Search Engines

Maximizing traffic from the search engines to your web site... Read More

How and When Should I Submit My Website to Google?

As soon as you register your domain name, submit it... Read More

Design A Spider Friendly Site

To be successful in the search engines it's important to... Read More

Sitemaps 101 - Back to SEO School

Sitemaps are without doubt one of the most often ignored... Read More

Keywords, Ranking, & Search Engine Optimization Fun

I am a Search Engine Optimization newbie. I have read... Read More

Picking Keywords for SEO ? A Different View

The first step to developing any search engine optimization effort... Read More

What are My Chances to Get the First Place in Search Engine Listings?

You must have heard the stories how people became rich... Read More

Finding Profitable Keyword Phrases

So you have a site concept developed that you are... Read More

How and Why to Avoid the SEO Mania

The WhyBut what is the reality of reaching a number... Read More

Linking for Fun and Profit

Well actually, linking isn't fun at all. In fact, it's... Read More

Easy And Simple Steps To Get Listed In Search Engines

Search engines are one of THE best resource for free... Read More

Google Rankings ? Achieving a Top 10 Position in Google ? Part 1

Achieving a top ranking position in Google is every webmasters... Read More

Link Horse Trading For The PR Challenged

After 105 days Google finally updated PR. And it's about... Read More

Linking Strategies to Skyrocket to the Top of Google

If you don't know already, one of the key success... Read More

Paris Hiltons Reputation is SHOT! Hows Yours?

With apologies for the cheap trick of mentioning Paris Hilton... Read More

Advertise Locally Using Search Engines

While search engine advertising has been a great advertising medium... Read More

Google Page Rank Is Dead - Or Is It? - Part I

For a long time now, marketing gurus all over the... Read More

One Way Linking Campaigns II

There is a way to generate links with the content... Read More

Writing SEO Copy ? 8 Steps to Success

We all know that the lion's share of web traffic... Read More