Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future: use mysqli or PDO instead in /home/a26f9f83/public_html/articles/includes/config.php on line 159
Effectively Using Robots Meta Tags > NetSparsh - Viral Content you Love & Share

Effectively Using Robots Meta Tags

The "robots" meta tag, when used properly, will tell the search engine spiders whether or not to index and follow a particular page. For the purposes of this article, we will be using the "( )" symbols to represent the "< >" in html coding.

Some examples of robot usage are as follows:

(meta name="robots" content="index,follow")

(meta name="robots" content="noindex,follow")

(meta name="robots" content="index,nofollow")

(meta name="robots" content="noindex,nofollow")

Let us first examine what these terms mean before we explain the usage for each one:

"index"- This directive tells the search engine robots (or spiders) that it is okay to index the page. Another words, you are allowing the search engine to include your page within their search directory.

"noindex"- Using this tag, you are informing the robots that this page should not be indexed. Simply put, this page will not appear in their search directory.

"follow"- When you use this tag, you are telling the search engines that you want their robot to follow any links that are found on that page.

"nofollow"- The opposite of the above definition, this directive will tell the robots not to follow any links on your page.

Putting It All Together:

With the robots tags explained, let's examine the usage for each one.

1. (meta name="robots" content="index,follow")

This tag will be used when you want the search engine spiders to index the page and follow the links to other pages. Most search engines use this setting as a "default" setting. It is possible that you may not even need to use this tag if you want the search engines to follow and index the page. However, an article at Search Engine World (searchengineworld.com/metatag/robots.htm) suggests that Inktomi does not use this as their default setting. Instead, they use the "index, nofollow" tag.

Better safe than sorry!

There has been much debate over whether or not it is necessary to use this tag. If there is even a slight possibility that some search engines do not use this as the default setting, then it would only make sense to include this tag if you want your page included in their search directory AND your links to be followed. Do the research and decide for yourself.

2. (meta name="robots" content="noindex,follow")

This tag can be used to tell the search engines that you do not want the page included in their directory, but you DO want them to follow the links that lead to other pages. A good example of its usage would be your disclaimer or privacy policy pages. You may not want these pages to show up in the search engines if they are only important to your actual visitors. However, if the links on these pages point to other pages that you want the search engines to find, then you would still want the spiders to "follow" those links.

3. (meta name="robots" content="index,nofollow")

This tag will allow your page to be indexed in the search engines, but any links on that page will not be followed.

4. (meta name="robots" content="noindex,nofollow")

When using this tag, the search engine spiders will not include this page in their directory and will not follow any links on the page either.

Where does the "robots" tag belong?

The "robots" meta tag should be used within the (head) and (/head) tags of your page. These tags are located at the top of the html coding. It will look something like this:

(html)
(head)
(title)Title of your page goes here(/title)
(meta name="keywords" content="word1,word2,word3,word4")
(meta name="description" content="A brief description of the content of this page.")
(meta name="robots" content="index,follow")
(/head)
(body)
Your webpage information here.
(/body)
(/html)

More Robots Tags

Google automatically archives a page as it crawls it. This is called a "cached" version of the page. Visitors can retrieve the archived version of the page by clicking on the "cached" link within Google's search results. If you do not want your content to be archived, you can use the following tag:

(meta name="robots" content="noarchive")

*This will only prevent your page from being "cached". If you do not want your page to be indexed at all, you will still need to include the "noindex" tag.

Another alternative to the above tag is the tag that specifically addresses Google only. If you want other search engine robots to archive your site, but you would like to prevent Google from doing so, then you can use the following tag:

(meta name="googlebot" content="noarchive")

The Misuse of Robots Tags

Something that has been popping up on websites everywhere is the Google indexing tag. This is a silly little tag that is not necessary. Some people think this tag helps Google to spider your site, but this simply isn't true. The tag looks like this: (meta name="googlebot" content="index,follow"). Some website owners believe that by specifying "googlebot" that their site has the advantage of being spidered faster and listed by Google. According to Google's web crawler information, you may use the noindex, nofollow, or noarchive tags when you DO NOT want Google to cache, index, or follow that page. Google's default setting is to index and follow the links on the page, so this "so called" googlebot index/follow tag that some site owner made up one day is completely unnecessary.

Another silly little tag--- The "Revisit-After" Tag

(meta name="revisit-after" content="90 days")
(meta name="revisit-after" content="15 days")

I'm not sure where this myth was started. Today, you will find this tag all over the Internet. Webmasters have even promoted it, claiming that it actually works. Are we so naive to believe the search engine spiders need to know when to come back? I have never used this tag, and my site has no problem with being crawled on a regular basis. Even some SEO (search engine optimization) sites are claiming its value. This comes back to the importance of researching the topic thoroughly first. My research came from some interesting information on WebmasterWorld.

It is important to examine the correct usage of the "robots" tag before applying it to your website. Incorrect usage of tags could result in spidering errors that cause robots to completely ignore the page all together. If you are interested in learning more about web robots, this great little site will provide you with the information you need to use them effectively: www.robotstxt.org/wc/robots.html

Stephani Richardson is a work at home mother of 4 who has been actively involved with affiliate marketing and home business opportunities since December 1999. She owns and operates several business related websites including 1 Work At Home Dot Com.

In The News:

This RSS feed URL is deprecated, please update. New URLs can be found in the footers at https://news.google.com/news

Techaeris

Web development services: What business opportunities do they provide?
Techaeris
Web development services are becoming more crucial with each passing day. Simple site builders can provide some semblance of a digital presence albeit a temporary one. A higher level of site quality is becoming a deciding factor that can make or break ...


Customer Think

Know about the current trends in web development
Customer Think
In case of web development, you may have probably heard that the only constant is the change. It is an industry-defining statement as web development keeps on changing. User expectations are evolving rapidly, and it is quite essential than ever before ...


Business 2 Community

4 Ways to Ensure Effective Web Development
Business 2 Community
When it comes to building a new website, many businesses choose to focus predominantly on the design aspect of the project when approaching agencies to carry out the work. Whilst design is a vital part to any web development project, there are a vast ...


Project6 Receives 2018 WebAward for Outstanding Achievement in Web Design and Development for CrossFit Games
Markets Insider
EMERYVILLE, Calif., Oct. 18, 2018 /PRNewswire-PRWeb/ -- San Francisco Bay Areawebsite design and development company Project6 is proud to announce that it has been awarded the 2018 WebAward for Outstanding Achievement in Web Design and ...

and more »

AroundtheO

New Portland program focuses on coding and web development
AroundtheO
In an effort to meet regional demand for web developers, the UO will offer a new 24-week coding course in Portland starting this January. The University of Oregon Coding Boot Camp is a part-time program is geared toward adult learners and working ...

and more »

Coinpedia (press release) (blog)

Top 10 Crypto Web Development Company
Coinpedia (press release) (blog)
After the emergence of the first cryptocurrency in 2009, a chain of reactions started leading to the growth and development of the cryptocurrency industry. Here we are listing the Top ten companies that are leading in cryptocurrency web and application ...


Neowin

Pay What You Want for this Full Stack Web Development Bundle
Neowin
Today's highlighted deal comes via our Online Courses section of the Neowin Deals store, where you can Pay What You Want for the Full Stack Web Development Bundle. The world's best developers are Full Stack developers. Here's 122+ hours of ...


International Association of Athletics Federations

Job vacancy: Marketing and web development project manager
International Association of Athletics Federations
The IAAF is seeking a marketing and web development project manager at its Monaco headquarters. Reporting to the chief information officer, the marketing and web development project manager will oversee several projects and ensure that information ...


Terrakan.com Announces Launch of New Web Service
Business Wire (press release)
LOS ANGELES--(BUSINESS WIRE)--Terrakan.com is a brand new web platform for real estate developers that was just released to the public this month. The trailblazing new service offers property reports that can tell you the development value of any ...

and more »

Code Staff: Disrupting a 40 Billion Dollar Web Development Industry with Its Launch Success and Continued Popularity
Digital Journal
For clients and businesses, it still is hard to find quality and reliable experts in web design, web development or software development. While this is the dilemma to the client side, there are more than hundreds of thousands of tech professionals ...

and more »
Google News

7 Killer Ways To Increase Your Online Sales

Every online entrepreneur I talk to is asking the same... Read More

Beautiful Web Sites Seldom Make More Sales

Designing an E-commerce Web site is not as simple as... Read More

Product Promoting: Getting The Response

When you publish a content site there are times when... Read More

Web Accessibility: The Basics

What is web accessibility & why is it important?Web accessibility... Read More

The Topic Of Your Website

The first thing you will have to deal building your... Read More

4 Steps to Selling More With Your Web Site

Most web sites are more like stage sets than real... Read More

Web Coach Tip: What You Should Know About DIY Web Sites

Recently, a friend asked "What's the deal with those DIY... Read More

Have You Got Your Ear To The Ground Or Your Head In The Sand?

What happened on your website yesterday? What about last week... Read More

The Power Is In The Pipes: How To Get Maximum Leverage From Your Website

What is the most important part of your online business?Many... Read More

Why You Need A Web Site

"I have a small business with just a couple of... Read More

7 Reasons Why Your Small Business Needs A Web Site

Many small businesses have the misconception that their business can... Read More

Web accessibility for screen magnifier users

The needs of screen magnifier users are overlooked when implementing... Read More

Website Value - Whats Your Business Website Worth?

If you were asked to put a value on your... Read More

Get a Website!, Register Your Domain!: To Website or Not to Website?

If you're fairly new to the Internet scene, you may... Read More

Maintaining Your Business Website

QUESTION: Should I build and maintain my business Web... Read More

Thinking Of Taking Money Online? Whats Best For Merchants? PayPal, WorldPay or Something Else?

It's now easier than ever to take payments on the... Read More

Improve Conversion Rates ? Effective Content

Your site is fast and getting traffic, but conversion rates... Read More

Why to Have a Website for Your Company

After having decided to start a business, the next thing... Read More

From Search to Sale

The Internet car buyer is an elusive animal. Trying to... Read More

Create Huge Income From Your Web Site - 10 Easy Ways

The cost of setting up a web site is dirt... Read More

Learn to Build a Better Online Business Website Using Keywords and Content

Are you aware that 90% of the websites on the... Read More

Why Every Small Business Owner Needs Two Websites

Every small business owner knows that they need a website,... Read More

How To Start Your Own Website

I'm one of those people that always wanted a website,... Read More

Direct Sales and Your Corporate Website - A Creative Marketing Plan that Works!

Creatively marketing your corporate site takes time in the set... Read More

Up The Sandbox!

Go to any internet marketing forum you want these days... Read More