Deprecated: mysql_connect(): The mysql extension is deprecated and will be removed in the future: use mysqli or PDO instead in /home/a26f9f83/public_html/articles/includes/config.php on line 159
The Role of the Robots.txt File to Improve Site Ranking! > NetSparsh - Viral Content you Love & Share

The Role of the Robots.txt File to Improve Site Ranking!

Not many web master take the time to use a robots.txt file for their website. For search engine spiders that use the robots.txt to see what directories to search through, the robots.txt file can be very helpful in keeping the spiders indexing your actual pages and not other information, such as looking through your stats!

The robots.txt file is useful in keeping your spiders from accessing parts folders and files in your hosting directory that are totally unrelated to your actual web site content. You can choose to have the spiders kept out of areas that contain programming that search engines cannot parse properly, and to keep them out of the web stats portion of your site.

Many search engines cannot view dynamically generated content properly, mainly created by programming languages, such as PHP or ASP. If you have an online store programmed in your hosting account, and it is in a seperate directory, you would be wise to block out the spiders from this directory so it only finds relevant information.

The robots.txt file should be placed in the directory where your main files for your hosting are located. So you would be advised to create a blank text file, and save it as robots.txt, and then upload it to your web hosting to the same directory your index.htm file is located.

Here is examples of the use of the robots.txt file:

To block out a directory in a robots.txt file, such as a subdirectory for your online store called /store/ you would do the following: Disallow: /store/

Another example to block out your stats directory: Disallow: /stats/

You may also want to disallow individual files that you do not want searched by the search engines. For example you dont want search.php to be parsed by the Search Engines. To do this you type in the following on its own line:

Disallow: /search.php

Following the rules outlined and creating the robots.txt file, you will keep search engine spiders out of unwanted files and directories, and letting them go through the important files to see what your web site is all about!

Michael Kralj is owner of Emenki Web Solutions and Domains at Retail. Emenki Web Solutions are web site designers and programmers based in Hamilton, Ontario, providing businesses with an informative and strategic approach to establishing an online presence on the web.

Please visit Emenki Web Solutions & Web Design in Hamilton, Ontario on the web http://www.emenki.com

Please visit Domains at Retail - Cheap Domain Name Registration on the web: http://www.domainsatretail.com

In The News:

This RSS feed URL is deprecated, please update. New URLs can be found in the footers at https://news.google.com/news

Search Engine Land

SearchCap: Google Assistant ups its game, SMX East, SEO traffic projections & more
Search Engine Land
There is no such thing as being negative SEO-proof, says contributor Joe Sinkwitz. All you can do is take steps to lessen the probability of becoming a victim. Here's how to reduce attack vectors and protect your site. Ask an SMXpert – Google Tag Manager
SEO 101 – How It Can Help Your Business? [Infographic]Business 2 Community
SEO Done Right: 5 Ways to Effectively Optimize your WebsiteBdaily
Rev Up That (Search) Engine: Three Ways To Drive Online And In ...Forbes
thebusinesstactics.com -Search Engine Journal -CUinsight.com (press release)
all 13 news articles »

Search Engine Land

How to project SEO traffic levels and avoid saying, 'It depends'
Search Engine Land
Have you ever had this conversation with someone from upper management? Boss: “If we implement your recommendations, how will this impact traffic?” You: “It depends.” Boss: “If we create a new content project, will it generate a lot of traffic? You ...


Search Engine Journal

Video SEO for Universal Search: How to Get Found in All the Right ...
Search Engine Journal
Here are the best video SEO best practices to get your YouTube videos found in both Google Universal Search and YouTube search results – and why you ...

and more »

Business.com

Should Your Business Hire an SEO Expert?
Business.com
The rudiments of good SEO are relatively easy to grasp, but it takes a lot of experience and expertise to be a master. There are plenty of resources to support your SEO efforts, though, including courses and tools, many of them free. Or you can hire a ...

and more »

Forbes

Google's Top Ranking Factors For Local SEO
Forbes
Relevance: First On The Local SEO Checklist. Relevance is a staple of Google's algorithm and attorneys, business owners and marketers need to make their GMB profile as relevant as possible to what searchers are using to find them. Here's how you can do ...
Infographic: 72 Stats to understand SEO in 2018AMEinfo
What Are Pillar Pages & How They Impact Content Marketing & SEOBusiness 2 Community

all 17 news articles »

Search Engine Journal

How Important Is an H1 Tag for SEO?
Search Engine Journal
Editor's note: “Ask an SEO” is a weekly column by technical SEO expert Jenny Halasz. Come up with your hardest SEO question and fill out our form. You might see your answer in the next #AskanSEO post!


Search Engine Journal

7 Signs Your Law Firm May Need a New SEO Partner
Search Engine Journal
The legal industry has a serious problem. Law firms and attorneys around the country continue to get taken advantage of by “churn and burn” SEO agencies. These agencies, which are run by self-proclaimed SEO experts, in most cases, promise big results ...


Forbes

Amazon SEO: Seven Factors For Ranking Highly On Amazon
Forbes
Amazon can be a highly lucrative yet competitive way of selling your products. You're competing with other sellers for visibility and high rankings in the search results pages. A placing on the first page is important as 70% of Amazon shoppers (paywall ...


Search Engine Journal

YouTube SEO from Basic to Advanced: How to Optimize Your Videos
Search Engine Journal
Video optimization, or YouTube SEO as it is becoming more commonly called, gives good video content the ability to have greater visibility against other videos – not just in YouTube search, but also in Google search and other search engines like Bing.

and more »

Entrepreneur

5 SEO Mistakes that People are Still Doing in 2018
Entrepreneur
Google has been updating its algorithms with some strict rules and keeping SEO professionals on their toes. Panda, Hummingbird, and Penguin are all the different names given to updates done in the search algorithms by Google. This made SEO serious ...
How to be proactive and prevent a negative SEO campaign — Part 3Search Engine Land
A Beginner's Guide to SEO in a Machine Learning WorldSearch Engine Journal
The core pillars of technical international SEOMarketing Tech
MediaPost Communications
all 14 news articles »
Google News

Common Sense Approach to Search Engine Optimization

Most web surfers start their sessions at a search engine... Read More

Press Release + PRWeb = Top Google Rank... True or False?

I've always been a big time press release aficionado.And up... Read More

Googles Good-Writing Filter

I was recently struck by the fact that the top-ranking... Read More

Taking Advantage of Googles Sandbox Effect

Most new sites submitted to Google (at least within the... Read More

Do Not Drop Your Web Site Off the Search Engine Cliff

If you've been feeling like Tom Cruise climbing up the... Read More

Martial Arts Webmasters: Time to Optimize Your Site!

A few months ago I was looking through the search... Read More

Search Engine Submission - What to Look for when Selecting an Australian SEO Company

Every business owner I've met has been encouraged at some... Read More

Valid HTML code is crucial to Search Engine Optimization

Why valid HTML code is crucial to your web site's... Read More

A Simple White Hat Technique To Get Indexed By Google

Everybody knows that getting indexed in Google is getting more... Read More

The Hottest Tip On Getting A Web Site Listed Quickly by Yahoo, Inktomi, MSN, FAST and AltaVista

Every webmaster knows that the free search engines are a... Read More

The Myth of Rankings - Beyond Search Engine Optimization

What follows is a condensed version of a conversation that... Read More

How to Top Google by Writing Articles

Search engines determine their rankings based on two things: ... Read More

Is Reciprocal Linking Dead?

I just read an article at SitePro News that really... Read More

Search Term Suggestion Tool

Overture.com offers a cool function to assist you on your... Read More

Link Popularity: Distribute Content, Not Just Links

You've spent many hours trying to increase your online traffic... Read More

Search Engine Optimization for Beginners

If you are confused about terms like "search engine optimization"... Read More

The First Search Engine Marketing Method: Content That Meets Customers Needs

For your business web site, good search engine rankings and... Read More

How To Find Good Keywords

Good keywords are frequently searched for (high demand) but not... Read More

The Search Doesnt End at Your Homepage

In a recent report for a new client, I wrote:... Read More

Search Engine Optimization Techniques

Search engine optimization is the process of increasing the amount... Read More

7 Search Engine Optimization Mistakes and Solutions

To many websites, webmasters discover that major sources of website... Read More

Your Website Title Could Be Costing You Money

Nothing could be simpler than the title you give to... Read More

Driving Your Website through Google Sandbox

What is Google Sandbox?Google Sandbox is applied on new websites,... Read More

Googles Trap, DMOZs Nap, And Yahoo!s Crap

On November 16th, 2003, Google commenced an update (the Florida... Read More

The Best and Easiest Google-Friendly Change to Your Web Site

No matter who you are or how much you pay... Read More