The Role of the Robots.txt File to Improve Site Ranking!

Not many web master take the time to use a robots.txt file for their website. For search engine spiders that use the robots.txt to see what directories to search through, the robots.txt file can be very helpful in keeping the spiders indexing your actual pages and not other information, such as looking through your stats!

The robots.txt file is useful in keeping your spiders from accessing parts folders and files in your hosting directory that are totally unrelated to your actual web site content. You can choose to have the spiders kept out of areas that contain programming that search engines cannot parse properly, and to keep them out of the web stats portion of your site.

Many search engines cannot view dynamically generated content properly, mainly created by programming languages, such as PHP or ASP. If you have an online store programmed in your hosting account, and it is in a seperate directory, you would be wise to block out the spiders from this directory so it only finds relevant information.

The robots.txt file should be placed in the directory where your main files for your hosting are located. So you would be advised to create a blank text file, and save it as robots.txt, and then upload it to your web hosting to the same directory your index.htm file is located.

Here is examples of the use of the robots.txt file:

To block out a directory in a robots.txt file, such as a subdirectory for your online store called /store/ you would do the following: Disallow: /store/

Another example to block out your stats directory: Disallow: /stats/

You may also want to disallow individual files that you do not want searched by the search engines. For example you dont want search.php to be parsed by the Search Engines. To do this you type in the following on its own line:

Disallow: /search.php

Following the rules outlined and creating the robots.txt file, you will keep search engine spiders out of unwanted files and directories, and letting them go through the important files to see what your web site is all about!

Michael Kralj is owner of Emenki Web Solutions and Domains at Retail. Emenki Web Solutions are web site designers and programmers based in Hamilton, Ontario, providing businesses with an informative and strategic approach to establishing an online presence on the web.

Please visit Emenki Web Solutions & Web Design in Hamilton, Ontario on the web http://www.emenki.com

Please visit Domains at Retail - Cheap Domain Name Registration on the web: http://www.domainsatretail.com

In The News:

What Is Dwell Time & Why It Matters for SEO  Search Engine JournalHere's what you really need to know about dwell time, how search engines use it, how it impacts rankings, and how to influence it.
5 Growing SEO Methods to Optimize your Keywords in 2019  YourStory.comWhile everything else stops to run in a proper manner, or when you have to do it, tweaking your keyword lists is similar to that of your regular car maintenance.
Publishing Thousands of URLs at Once – Is It Risky for SEO?  Search Engine JournalFind out whether publishing thousands, or hundreds of thousands, of new pages at the same time could have any SEO risks or unintended consequences.
The future of SEO for online businesses  Business MattersBusiness MattersSEO has changed radically in the last decade. Search results have become more accurate to ensure web users get their desired search results.
Importance of Local SEO Ranking for Hoteliers  Hotel News ResourceArticle - Importance of Local SEO Ranking for Hoteliers - Any business that has an actual physical location needs to put time into their local SEO. It is imperative ...
What is HTML, cache and SEO? What are cookies and how do algorithms work? Digital language explained  Evening StandardTech speak can give us all a headache sometimes. Every day in the digital age, it seems we have a new phrase to contend with that describes the intricate ...
Vietnam coach Park Hang-seo confirms desire to give up coaching multiple teams  FOX Sports AsiaVietnam coach Park Hang-seo claims that he still plans to concentrate on only one of the two national teams for the country.
SEO Manager job, Johannesburg  Bizcommunity.comOur client a leading and renowned agency, seeks to hire an experienced SEO manager. Develop and implement SEO and link building strategiesUser ...
[Video] WebTextTool Review & Walkthrough: Level Up Your SEO in 2019  99Signals - Tech News, Tech Hacks, & MoreHere's an in-depth video review & walkthrough of WebTextTool — an SEO tool that can help you optimize your blog posts & improve your *content's* readability.
College students learn digital marketing, SEO during seminar  Daily PioneerThe college students learnt about Digital Marketing and SEO during a daylong seminar. A seminar on Digital Marketing and SEO was held at Oriental Institute of ...
7 Best SEO Tools to Help You Rank Higher in Google  EntrepreneurThe key to building and sustaining organic traffic is knowing which are the best SEO tools to use. Today, there are so many SEO checkers on the market that it be ...
The Most Effective SEO Strategies in 2019  QriusNew year, new you, new SEO strategies. With search engines like Google looking ahead to improve their services, it's vital to keep up and prepare for the.
Online Marketing Referrals For SEO Campaigns  CEOWORLD magazineIf you believe you want to learn more about internet marketing and SEO concepts take BrightEdge digital advertising quiz and SEO quiz. Finally, you've got to ...
3 Benefits to investing in SEO now  TechGenYZAs an online business or company, you want your products or services to reach the most number of internet users in order to make more sales. If you decide to ...
Introducing SEO for developers. It’s time.  Search Engine LandComputer programming is making enormous innovative leaps on the web, evolving from crusty old-school AJAX, to jQuery, to reactive views that synchronize ...
SEO Optimizers free digital marketing training classes in Los Angeles - Press Release  Digital JournalWe are a full *service* digital marketing agency, specializing in search engine optimization, website design, social media, and paid advertising. Since their ...
Exploring the Role of Content Groups & Search Intent in SEO  Search Engine JournalHere's what SEO professionals need to know about a Google patent "Optimized Web Domains Classification Based on Progressive Crawling with Clustering."
Vietnam coach Park Hang-seo begins planning for AFC U-23 Championship  FOX Sports AsiaSouth Korean Park Hang-seo joined as the head coach of the Vietnam national team back in 2017. His reign saw the Golden Dragons go on one of their more ...
SEO Tidbits From Google Fights Disinformation White Paper  Search Engine RoundtableOn Friday, Google published on their blog about fighting disinformation and posted a 30 page white paper. Truth is, there is nothing really new in this white ...
Is It Time To Outsource SEO? Here's What You Need To Consider  ForbesEveryone and their mother is chasing after the holy grail of internet marketing: organic search traffic.
Stay on Top of Your Site's SEO Ranking for Less Than $30  EntrepreneurKeeping up with the Kardashians is one thing, but keeping up with Google's ever-changing search engine ranking algorithm is a whole 'nother issue.
SEO vs SEM – What’s the Difference and Why it Matters to Your Business  G2 CrowdLearn the key difference between SEO and SEM and how they greatly increase the visibility of your site on search engines like Google, Yahoo, Bing, and more.
SEO Guide 2019: Your Strategy for This Year  Search Engine JournalAre you ready to become up-to-date with the world of search engine optimization? Read WebCEO's guide to learn the top SEO tactics you need in 2019.
Blockchain Consulting, SEO Marketing and Cryptocurrency Coaching Guide  Bitcoin Exchange GuideThe cryptocurrency and blockchain industry has exploded with growth over the last few years. Billions of dollars is pouring into the industry. Blockchain and ...
Updated for 2019! ‘Enterprise SEO Platforms: A Marketer’s Guide’  Search Engine LandMarTech Today's “ Enterprise SEO Platforms: A Marketer's Guide” has been updated for 2019. Compiled from the latest research, this 61-page report is your ...
5 SEO Career Trends for 2019 [New Research]  Search Engine JournalWhile the idea of SEO being dead is nothing more than worn out, industry fodder, SEO as a job title and role have been legitimately questioned. Not so long ago, ...
4 demanding areas of SEO to focus this year  TechGenYZThere's so much more to understand in SEO which is must to know to drive traffic to your website and rank higher. Know more here.
6 Ways SEO Tools Can Help You Run Your Business  Search Engine JournalTools make a marketer's life more productive. This post will tell you why and how SEO tools can help you better run your business.
Context in Outbound Links for High Ranking SEO  Search Engine JournalLinking out to other sites is important. The sites you choose to link to can cause your site to lose rankings. This article discusses outbound links for SEO.
SEO For Plumbers: Grow Your Plumbing Business With Search Engine Optimization  ForbesSearch engine optimization (SEO) offers a unique advantage for local *service* businesses. Just like the phone book of yesteryear, SEO allows the small business ...
Sharpen your SEO & SEM skills. See the SMX Advanced agenda!  Marketing LandFor nearly 15 years, SMX® Advanced has been the event expert search marketers attend to learn performance-enhancing tactics, make game-changing ...
New Study Shows How Much SEO Professionals Are Earning Per Year  Search Engine JournalA recent study on the local search industry reveals how much the average SEO professional earns in a year, and what they bill per hour. Data is based on ...
How to Set & Achieve Realistic SEO Goals for 2019  Search Engine JournalWhat milestones does your company need to reach in 2019? As marketers and SEO professionals, charting the path toward our goals is one of the exercises we ...
What Is an SEO Professional in 2019? An Identity Crisis Averted  Search Engine JournalPicture this: You're an SEO professional, and someone asks you what exactly it is that you do. What do you say? I'm a: Search analytics guru; *Content* strategist ...
Wix ‘SEO Wiz’ Stars in Big Super Bowl Ad, Fumbles with SEO Community  Search Engine JournalDid you hear that sound last night during the Super Bowl? It was the collective SEO community hitting their heads against a proverbial desk. Wix's foray into the ...
5 Things I Wish I Knew Before Hiring an SEO Vendor  Search Engine JournalLooking to oursource your SEO needs? Here are a few important things you should know before hiring your first SEO vendor.
SEO: Free Keyword Research in 3 Steps  Practical EcommerceThere are many expensive tools that claim to simplify and speed up the keyword research process. If you don't have thousands of dollars in your budget, ...
Angular Universal: What you need to know for SEO  Search Engine LandIf you use Angular to power your website, then you have an extra hurdle to cross when it comes to SEO. Fortunately, Angular Universal makes it easy to leap ...
How to keep your SEO team (and yourself) relevant  Search Engine LandWeekly rank tracking. Optimization meetings with the media, copy, UX and dev teams. Monthly analytics reporting with the client. And decks. So many decks.
Stirling 2019: Soon-Seok Seo seeks revenge  International Paralympic CommitteeSouth Korea's Soon-Seok Seo wants to take care of some unfinished business from a year ago at the PyeongChang 2018 Paralympic Winter Games. His team ...
SearchCap: Google updates, SEO education & A/B testing  Search Engine LandMueller previews Google image search changes, old Search Console sunsetting at NYC meetup. Jan 25, 2019 by Barry Schwartz. Here are the highlights from a ...
SEO Evolution: What’s next for SEO?  Fourth SourceThe year is almost at an end and with the dawn of a new one comes the prospect of Google's raft of SEO changes that the company is ushering for 2019.
SEO: Monitoring Traffic from Domain Mergers, with Google Data Studio  Practical EcommerceA common practice among retailers is creating microsites to target different audiences. The strategy can produce strong conversions because the user ...
MarTech Today Research: A marketer’s guide to enterprise SEO platforms  Search Engine LandEnterprise SEO continues to grow more complex as the technology and algorithms powering search engines evolve. Comprehensive enterprise SEO platforms ...
How to Use Python to Analyze SEO Data: A Reference Guide  Search Engine JournalDo you find yourself doing the same repetitive SEO tasks each day or facing challenges where there are not tools that can help you? If so, it might be time for you ...
60+ SEO Stats to Help You Rank Better in 2019 [Infographic]  Social Media TodayAre you looking to maximize your search performance in 2019? Need to know the stats and facts to help you understand the SEO process? The team from Red ...
Super Bowl Sunday SEO: The importance of search during the Big Game  ClickZSEO is crucial during the Super Bowl, both for $5 million-dollar advertisers and companies hoping to appear in local search results.
11 SEO Tips & Tricks to Improve Indexation  Search Engine JournalHere are 11 tips and tricks to ensure that your website gets crawled and indexed properly – and help your webpages rank higher in search.
4 Link Building Tips for International SEO  Search Engine JournalDiscover four link building tips for global websites that can be easily implemented even by companies with limited resources.
Targeted SEO: Here’s how to grow a page from 200 visits per month to 30,000  Search Engine LandSpoiler alert: SEO works. Strategic, targeted SEO can drive huge results for the right pages. Of course, the catch here is that you need the “right” pages to see the ...
SearchCap: Bing Maps API, Google and YouTube ads and targeted SEO  Search Engine LandNow your Google text ads can show on YouTube search results, too. Jan 30, 2019 by Ginny Marvin. Google says early tests have shown similar performance ...
Search Engine Land adds technical, in-house SEO experts to team  Search Engine LandSearch Engine Land is thrilled to announce that the team of experts who help us bring daily search marketing news and insights to thousands just got a bit ...
How to redesign your website without effective your SEO  Customer ThinkWhen you are planning to have a website redesign, it is vital that you take your time doing it and be careful, as if it is not done right, it could bring chaos to your ...
Reason why search engine optimisation will give you leading results  The Munich EyeSearch Engine Optimisation (SEO) is undoubtedly the top technique to use in the marketing industry. Imagine how many people advertise the same product that ...
SEO Keywords Best Practices for 2019 (for the Win!)  Business 2 CommunityCome the new year, you might be wondering if the keyword SEO whose rules you've been adhering to until now will still make sense with the frequent updates ...
Battle of the SERP: Combining SEO & PPC Forces to Win  Business 2 CommunityLegend has it there's a castle deep in the heart of the second page of Google. Guarding a castle is a monster named Bowser. Every so often, Bowser will creep ...
SEO for B2B Businesses: Is It Really a Necessity?  Thrive GlobalEffective SEO (search engine optimization) is a large portion of the entire mission of a B2B *service* company, as if you don't have stellar SEO, people aren't ...
Why Prioritize Local SEO?  Vertical MeasuresLocal SEO consists of many different local citation components, but is absolutely vital to the success of any organization aiming to reach to a nearby audience.
SearchCap: Local SEO survey, Moz domain authority and GOOG earnings  Search Engine Land5 takeaways from Google's Q4 2018 earnings for search marketers. Feb 5, 2019 by Ginny Marvin. Google executives discussed search experience, campaign, ...
Get your site to the top of any search with this SEO tool  Boing BoingThere's a lot of competition on the internet, no matter what your hustle. The important thing is that when people search for what you've got, they find you - and ...
Rise of Mobile Users Alters SEO Tactics  Practical EcommerceThe percentage of U.S. searchers clicking on advertisements in Google search results is growing. Likewise, the percentage of visitors that click nothing is also ...
SEO Specialist - Aceable  Built In AustinAceable is hiring for a SEO Specialist in Austin. Find more details about the job and how to apply at Built In Austin.
Save 98% off a lifetime subscription to RankActive SEO Toolkit  NeowinToday's highlighted deal comes via our Apps + Software section of the Neowin Deals store, where you can save 98% off a lifetime subscription to RankActive ...
Google Search Console sending alerts for big ranking, traffic drops  Search Engine LandGoogle is now sending alerts and notifications from Google Search Console when it detects a “substantial drop” in clicks to your website from the Google search ...
BVM Now Providing SEO Services to Businesses in El Paso, TX  Markets InsiderHOUSTON, Feb. 15, 2019 /PRNewswire-PRWeb/ -- BVM (https://www.brazosvalleymarketing.com), an UpCity certified partner, is now offering SEO services in El ...
Tangle Teezer appoints Avenue Digital for international SEO and paid search  The DrumBritish hairbrush brand Tangle Teezer has appointed London search agency Avenue Digital to handle its SEO and PPC activity following a competitive pitch.
Top 10 mistakes companies are making with local SEO  ZDNetAlmost two out of five business leaders do not know if their website is optimised for local SEO. So, what should they look out for?
'Very concerning': Senate watchdog hasn't finished an ethics probe since 2017  CBC NewsThe Senate Ethics Officer didn't publish a single inquiry report last year, prompting some senators to wonder exactly what the office (which has a $1.2 million ...
Google’s Gary Illyes Urges SEOs to Stick to the Basics  Search Engine JournalMore than once during his recent AMA on Reddit, Google's Gary Illyes advised SEOs to stick to the basics.
ILS at forefront of monitoring for climate change impacts: John Seo  Artemis.bmThe impacts of climate change present challenges for all industries and markets, but over the last two decades, the insurance-linked securities (ILS)
Sugar Land Google Ranking Expert SEO Services Launched  NewswireHouston, Texas online marketing agency Impactnext announced that its SEO and online marketing services are now available to local businesses in the Sugar ...
Yoast SEO to add “live indexing” feature for Bing  OnMSFTFollowing Bing's announcement that they are hoping to change the way search engines discover new and updated *content*, one of WordPress's largest SEO ...
The 12 Areas of SEO Knowledge You Must Master to be Successful  Search Engine JournalThe role of an SEO requires mastery of certain skills and knowledge. Here's what SEOs need to master for career advancement and campaign success.
HempLife Today Gives Shareholder Update on Search Engine Optimization Progress and the 500% Increase in Its Home Page Trust Score  NasdaqDenver, CO, Jan. 31, 2019-- via NEWMEDIAWIRE-- Ubiquitech Software Corp., through its subsidiary HempLife Today™, is giving shareholders an update.
40 Major SEO Mistakes You Must Avoid in 2019 [Infographic]  Social Media TodayAre you struggling to get your website to rank on the first page of Google? Want to see if you're making any SEO mistakes that could be holding your website ...
DigitalBerge, The #1 SEO Company in Delhi NCR, Launches its Personal Branding Services  Business StandardRead more about DigitalBerge, The #1 SEO Company in Delhi NCR, Launches its Personal Branding Services on Business Standard. / -- With the launch of its ...
Mansi Dhorda  Search Engine JournalMansi Dhorda works at E2M Solutions Inc. - a Digital Marketing Agency. She is well-versed in all things *Content* Marketing and Branding. During her free time, ...
SEO 101 Tutorial Videos  Vertical MeasuresHere are 6 SEO tutorial videos from our SEO 101 training series on improving your search engine ranking through proper onsite SEO tips and techniques.
Park Eun Bin, Seo Ji Hoon, And BTOB's Yook Sungjae In Talks For New tvN Drama  soompiMore actors are in talks to star in tvN's upcoming drama “Man Who Bakes Bread” (literal title). “Man Who Bakes Bread” is a romance drama based on a Naver ...
5 Critical SEO Considerations When Optimizing News Websites  Search Engine JournalA few rules of SEO, like publishing great *content* and providing a good user experience, are universal. But not all websites thrive by using the exact same set of ...
SE Ranking | The Best SEO Tool To Outperform Your Competitors  StartupWorldSE Ranking website audit tool not only checks out the main parameters of your websites but also shows all SEO errors, technical issues.
jeonghwa seo crafts brutalist concrete interior for etcetera cafe in seoul  Designboomin the design of etcetera cafe seoul, jeonghwa seo emphasizes the material contrast between concrete and glass, and a visual continuity toward the street.
12 Reasons Why Your Business Absolutely Needs SEO  Search Engine JournalSEO is important for greater searchability and visibility, but offers more real value than that. Here are 12 reasons why businesses need SEO to take their brand to ...
Is SEO the future of successful apps?  Geospatial WorldGoogle recently updated their mobile search algorithm to show the results of not only websites but also apps. Plus, the app results come with an install button.
Omaha SEO-Optimized Copywriting Services Announced  NewswireDeb Griffith Copywriting has announced it can provide clients with expert, professional web *content*. She writes high quality copy with a focus on getting clients ...
Google: SEOs Don't Manipulate Google They Manipulate Web Sites  Search Engine RoundtableSometimes Google's John Mueller has these one liners that I just want to share with the SEO community. He said on Twitter "SEOs don't manipulate Google, ...
Watch: Jin Goo And Seo Eun Soo Are Headstrong Lawyers In Highlight Video For “Legal High”  soompiLegal High” has released a new highlight video showing a preview of the upcoming drama! JTBC's upcoming Friday-Saturday drama “Legal High” is about two ...
SEO: 6 Things to Do When You Have Nothing to Do  Practical EcommerceSearch engine optimization is self-directed. Unless you're being managed by another practitioner who tells you what to do and when to do it, SEO can be difficult ...
47 Experts on the 2019 SEO Trends That Really Matter [Ebook]  Search Engine JournalIn 47 Experts on the 2019 SEO Trends That Really Matter, learn how the following SEO outlooks will influence the way we do SEO in 2019: Understand Your ...
SEO still leaves most marketers puzzled. With this $30 training package, you won’t be.  The Next WebDo you know the most searched keywords around your business or website? Do you use those keywords in your titles and tags? Do you know your optimal ...
Yoast To Add Live Indexing With Bing & Google  Search Engine RoundtableJoost de Valk announced at YoastCon that the popular WordPress plugin, Yoast SEO, will provide live indexing with both Google and Bing starting next month.
Long Island SEO Company, fishbat, Shares 4 Tips For Promoting Holidays on Social Media to Stand Apart from the Crowd  PRNewswireNEW YORK, Feb. 13, 2019 /PRNewswire/ -- As part of their continuing effort to teach companies how to make the most of their social media marketing, Long ...
Why Prioritization Is Critical to SEO Success  Search Engine JournalHere's why prioritizing on-site and technical optimization, *content* creation and repurposing, and link building will set your SEO campaign up for success.
12 SEO Goals for 2019: Ideas That Will Inspire You  Search Engine JournalIt's time to set your SEO goals for 2019. Need some inspiration? Read this roundup of New Year's goals from the SEO community.
Is There Any New Innovation In SEO?  Forbes NowSearch engine optimization (SEO) has been a field of its own in the marketing community for the past two decades. But in a world where most industries are ...
SEO drives search results. Control those rankings with RankActive SEO for $25  The Next WebSEO is like water. You can hit it pretty consistently and even capture some now and again. But trying to firmly grip it is darn near impossible. That's because the ...
5 SEO trends that will matter most in 2019  Search Engine LandTo be atop the waves, think about your SEO strategy in advance. A shortcut to success: get to know the upcoming trends and work out an action plan for each.
15 Common SEO Mistakes To Avoid  ForbesUnderstanding SEO is the ultimate marketing discipline, especially in today's highly digital world. If you want to drive traffic to your site and increase sales, you ...
A checklist: Important SEO points to cover in a content campaign  Search Engine LandAfter you've chosen the right execution for your link-building campaign and you're getting ready to launch it, you can't forget about search engine optimization ...
SEO Is A Must For Small Businesses: Here's How To Get Started  ForbesYou are starting a business and have a checklist of the things you need to get your business going. You have the idea for a product or *service* -- check.

Microsofts New Search Engine

I tried out Microsoft's new search engine (beta version) the... Read More

Top Ten Listing? Hmmmmmmmm

This is my question...I spend a fortune on SEO and... Read More

Mindset from Yahoo!- Changing the SEO Game

Yahoo! has a developed a new concept in search engine... Read More

The Smart Way To Get Listed On Google!

So you want to get listed on Google? And you... Read More

Link Reputation - How to Improve Search Engine Rankings

One of the most overlooked strategies to improve search engine... Read More

An SEO Checklist

Search engine optimization is on every webmaster's mind these days.... Read More

Search Engine Optimization Explained

Search Engine Listing And OptimizationIn the world of website marketing,... Read More

Creating a Google SiteMap For Your Work At Home Business Web Page

Search engine traffic is the best traffic You can get... Read More

Using Blogs for SEO

Why Start A Blog?I knew about blogging and blogs for... Read More

Keywords ? Key For All Doors

Keyword research is certainly the most important phase in Search... Read More

SEO Expert Guide - Page Optimization (part 5/10)

In parts 1 - 4 you learnt how to develop... Read More

Is Reciprocal Linking Dead?

I just read an article at SitePro News that really... Read More

Good Content: The Key To Search Engine Ranking

Good Content: The Key To Search Engine RankingWhat drives visitors... Read More

Submitting Your Site To The Open Web Directory: Some Dos And Don?ts

One of the most important steps in any site's publicity... Read More

A Three Day Marketing Plan for Better Google Rankings

If you're reading this article, you've probably discovered that simply... Read More

60 Day Sandbox for Google & AskJeeves; MSN Indexes Quickest, Yahoo Next

Search engine listing delays have come to be called the... Read More

2 Powerful Ways To Capitalize on Your Search Engine Traffic

Many marketers know that search engine marketing is among one... Read More

Search Engine Marketing: 20 Nitty-Gritty Strategies To Compel People To Link To Your Website

You don't have to be a rocket scientist to know... Read More

The Hottest Tip On Getting A Web Site Listed Quickly by Yahoo, Inktomi, MSN, FAST and AltaVista

Every webmaster knows that the free search engines are a... Read More

5 Qualities to Look for in a Search Engine Optimization Program for Your Website

These days with millions of websites out there, you need... Read More

Optimum SEO Keyword Density ? A Real-Life Case Study

So you've built your website, you know what keywords you... Read More

Googles New SEO Rules

Google has recently made some pretty significant changes in its... Read More

Simple Way to Get High Pagerank

Everyone knows that if your website has a high Google... Read More

Google Page Rank - Important Or Just Another Number?

In my last newsletter I wrote about how your websites... Read More

Search Engine Optimization Techniques

Search engine optimization is the process of increasing the amount... Read More