Blocking external links in Robots.txt - need advice on Best Practice
-
I look after an affiliate site that is doing quite well in the search rankings. We've been doing a review of our practices and one thing that has cropped up is our robots.txt.
In it, we disallow Google from crawling external links. This used to be best practice in the affiliate industry a couple of years ago, but I wanted to know if this is still the case, and what the implications are if we were to:
a/ leave it as is?
b/ allow crawling?
Thanks in advance.
-
I'm not sure how you're blocking Google from crawling external links in the robots.txt file--typically you only block them from crawling internal pages on your site.
If you're using a script, though, to track the clicks on external links and that script is running your site (and you're blocking that script in robot.txt), then that still should be fine. You may want to add a "nofollow" tag on those links, though, so you don't end up passing link "credit" or "link juice" to those affiliates (unless you want to do that?).
As far as external links go, though, it's typically okay (and expected) that you link out to other sites.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I easily no-follow my affiliate links?
Hi, I am in the process of setting up an affiliate marketing campaign for my site. I have done a bit of research and understand that it is safest in terms of SEO to add no-follow to the links from the affiliate sites. What is the easiest and simplest way to implement this?
Affiliate Marketing | | whiteonlySEO2 -
NoFollow Affiliate links from (very) shady websites will effect rankings?
Should I be worried , and if so what to do, if affiliates are linking from shady / gaming/adult websites? or dost the fact that the links are nofollow solve this issue completely ?
Affiliate Marketing | | KSafer0 -
What's the best way to go about making Duplicate Content Pages on my domain for affiliates?
Hello, I would like to make a bunch of duplicate pages of my site's Home Page, that way affiliates of mine can have a page of their own with links specialized to themselves littered throughout the page. What's the best practice going about this without jeopardizing domain authority from tons of duplicate content signals firing off?
Affiliate Marketing | | Benavest0 -
Need help about facebook fanepage!
I have a fanepage on facebook , some time have other fanpage go to my fanepage and spam..., now i want protect for they can't look at my fanpage :(( Who can help me thank very much! [link removed by staff]
Affiliate Marketing | | ITVSEO.COM0 -
I need help setting a campain
I am setting a campaign and I put the first page as a main route. Sample www.sample.com On this page people need click on a enter button because it is and adult page. After that everything redirects to www.sample.com/home.php Where you find all the links on the web page such as about, blogs, contact, promotions and so on. The problem that I see it's that I already set up the campaign but detects only one page. How can I see the errors of missing meta tags of the other inside pages. Should I set up the campaign and in my domain shall need to be www.sample.com/home.php instead of www.sample.com any wise advise on this?
Affiliate Marketing | | lnietob0 -
Where to look for relevant links
I and many others are targeting keywords such as "free iphone" or "free ps3" as part of incentive marketing schemes that offer rewards for referring new users to businesses. These often seem too good to be true but actually do work (if you don't get caught up in the spam sites). The thing is, what sort of link building should I be looking at once I have my site showing the users how the opportunity works and that it is a legitimate site. I don't want to have free iphone links placed where people may think I'm scamming them, so should I look at press releases, creating infographics or what? I look forward to hearing some of your ideas...
Affiliate Marketing | | GrassRootsSEO0 -
How much link juice passes through urls with affiliate id's?
hi we can get a valuable link with the desired anchor text from a news site. the destination url would be something like www.site.com/product. but in order to track conversions, our sales team would like to add an affiliate id to the url, so that it would look like this: www.site.com/product?sess_affiliate=ta how much link juice would a link to this affiliate url pass? would we be shooting our wad by linking to the ?sess-URL instead of the original URL?
Affiliate Marketing | | zeepartner0 -
Hiding Affiliate Links
Hello, Currently I'm developing a website that will contain a lot of affiliate links (5000). This is because I'm going to compare the prices of a lot of products. The visitor can go to all websites (my advertisers) that sell these products via affiliate links. My question is how to hide these affiliate links. I've read lots of solutions, like: cloaking, rel=nofollow, using JavaScript, 301 redirects. I would like to know what the best solution is to deal with these affiliate links for SEO. Regards, Martijn
Affiliate Marketing | | MartijnWijbenga290