Controlling Your Referer, and Hiding Your Traffic Sources
|
| |
 | |
How Do I Keep Affiliate Programs From Knowing My Traffic Source?
This question comes up a lot, so I thought I’d handle it now. While truly everyone has a good reason to hide their referrer(outranking their affiliate program for key terms, blackhat traffic, fear someone else will clone your site/copy your traffic source, or general paranoia). I’m going to approach this from a blackhat angle though.
It’s no secret many affiliate programs do not like blackhat traffic. This is especially true for affiliate NETWORKS like commission junction. So how can we keep them from identifying our traffic source? We definitely do not want them to see our blackhat URL. But we also do not want them to see a user coming directly from the search engine. So how can we properly redirect someone, so we can control what the aff program sees as our referer? How can we make it survive a possible hand check?
While this is a complicated topic(since it IS the browser’s choice on who to pass), I’ll do my best to test out some sample setups.
Note: If you don’t want to read the various tests, that’s fine. I bring it all together at the bottom.
Note2: I’ve heard of a lot of blackhats using methods of redirect who say their results contradict the ones I got from this experiment. If any would care to chime in, I’d be more than interested.
The Basics: Our goal is to take our blackhat traffic, bounce it off a nice whitehat site, then from there send that to the affiliate program. This 2 way redirect, when implemented properly, can hide the traffic source decently. The trick is having the whitehat site know when to redirect, and when not to.
- Our Test File Structure
- myStartPage.php : This is taking the place of the Google search result page in our experiment. Passing this variable to the affiliate program tells them that the user was either redirected, or came straight from the search engine result page(hereafter referred to as a “SERP”)
- myBlackhatPage.php: The second stop in our redirection, this page is our scraped/cloaked/whatever’d site. Whatever site we’re actually using them to promote, but don’t want them to see.
- myWhitehatPage.php: What we WANT the affiliate program to see as our referrer. This page[optimally] has 2 modes: whitehat quality content mode, and blackhat redirection mode. This page is not actually promoted at all, even whitehat. So we don’t care if it ranks. It may eventually get banned, but we don’t care.
- myAffiliate.php: The end page of our redirection algo. We’re hoping that this page will show myWhitehatPage.php as the referrer.
Non-Changing Variables
myBlackhatPage.php will ALWAYS be a 301/302 redirect to myWhitehatPage.php. Although we may want to alert the whitehat page to this fact later on, for right now we’re just researching what passes from page to page.
- Experiment 1: 302/301 Redirect
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage>myAffiliate.
- Code of Whitehat page page consists of
header(”HTTP/1.1 302 Temporarily Moved”);
header(”Location: http://localhost/referrerCheck/myAffiliate.php”); - Result: Passed myStartPage.php: Failure
- Experiment 2: Meta Refresh (6 Second Refresh)
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage(refresh)>myAffiliate.
Code of Whitehat page consists of
<meta http-equiv=”refresh” content=”6;url=http://localhost/referrerCheck/myAffiliate.php”> - Result: Passed a blank referrer: Semi-Pass
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage(refresh)>myAffiliate.
- Experiment 3: Meta Refresh (Immediate Refresh)
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage(refresh)>myAffiliate.
- Code of Whiotehat page consists of
<meta http-equiv=”refresh” content=”6;url=http://localhost/referrerCheck/myAffiliate.php”> - Result: Passed a blank referrer: Semi-Pass
- Experiment 4: Javascript Redirect (Window.location)
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage(JS redirect)>MyAffiliate
- Code of Whitehat Page consists of
<script type=”text/javascript”>
<!–
window.location = “http://localhost/referrerCheck/myAffiliate.php”
//–>
</script> - Result: Passed blank referrer: Semi-Pass
- Experiment 5: Frameset
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage(Frameset)>MyAffiliate
- Code of Whitehat page page consists of
<frameset><frame src=”http://localhost/referrerCheck/myAffiliate.php”></frameset> - Result: Successfully Passes myWhitehat.php as the referring file!
- Note: This is less than optimal. Some affiliate programs are set to jump out of a frame, and also we don’t want the user overly aware of where they were bounced off of.
- Experiment 6: IFrame
- Path for this is myStartPage>myBlackhatPage>myWhitehatPage(IFrame)>MyAffiliate
- Code of Whitehat page page consists of
<frameset><frame src=”http://localhost/referrerCheck/myAffiliate.php”></frameset> - Result: Successfully Passes myWhitehat.php as the referring file!
- Note: Suffers from the same problems as the previous (frameset) redirection method. Damn. However, with this, we can stop internet explorer only from breaking out of the iframe by adding security=restricted into the iframe tag, which [I believe] disallows the use of Javascript for iframes. But without firefox, this is not too useful.
Bringing it all Together
Now, we have no perfect example here. One might exist, one might not. My research says no. But we’re going to do the best we can. Here’s a decent start point for you all. The desired affect here is to get as close as possible to a perfect redirect in terms of not getting caught. It is not the most practical approach.
- Telling the Whitehat Site to Redirect, and Not Display it’s Content
- IFrame the whitehat site/page itself, from the blackhat site/page.
- On the whitehat page, check to see if their IP is in the database of people who have come in via the blackhat site previously. Or check to see if the cookie you [may] have dropped in step 4(yeah, confusing, I know) is still there. If they or the cookie is found execute a refresh to the affiliate program (if they’re already been here once, they will not pass a referrer. You’ll understand that better as you read on
- On the whitehat page, have php search for the blackhat referrer:
if(stristr($_SERVER[’HTTP_REFERER’],”myBlackhatdomain.com”)!==FALSE) //we have a BH redirect
{
//Output the Javascript in Step 5
}
else
{
//Output pretty whitehat content that will satisfy an affiliate program
} - Insert the user’s IP address into a database OR drop a cookie. Either one of these we can use to identify them as a user that got here via a blackhat source in the future.
- Use Javascript(this may not work in Opera, but will in Internet Explorer/Firefox) to break out of said iFrame.
<script type=”text/javascript”>
if (top.location!= self.location) {
window.location = “http://www.mywhitehatdomain.com/myWhitehatPage.php”
}
</script>
Note: In case Javascript is turned off, you may want to throw a meta-refresh in there to pick up the slack. Give it a couple second delay to give the JS a chance to work. If the meta refresh is forced to execute though, we will still be iFramed in the blackhat URL. But the referrer should be the whitehat site. So in 99% of cases we’ll be ok. - Now, the command above SHOULD have made the page reload, with no referrer(since they were already in an iframe of http://www.mywhitehatdomain.com/myWhitehatPage.php, and Javascript redirects do not pass a referrer.) So, once again, as in step 2, we check to see if their IP is in the database, or the cookie is still present. If it is, that means….yup! We can now iframe the real affiliate offer, with the blackhat site nowhere in existance. Not in the URL bar, not in the referrer, nowhere! And the whole thing looks like it happened immediately to the user.
XMCP, Why the Hell Does that Have To be So Complicated?
Because we now have a way to set a whitehat domain as the referrer. But even if the affiliate program comes to check the URL, nothing appears to be odd about it! They can’t find the iframe unless it was initially accessed via the blackhat site, which they have no way to find. So it appears to be a static link.
There might be a simpler way to achieve that, but hell if I can figure out how.
January 10th, 2008 at 3:58 pm
Is there a way to force a click based on referrer?
Meaning, if the referrer is a SE, the buy now button on your WH site gets clicked sending them through to the merchant.
That would be slick.
Nice article.
January 10th, 2008 at 4:37 pm
@XLSpecial:Thanks!
The only reason I dont do what you’re talking about, is I can never find an affiliate program that will pick up the affiliate ID off the “purchase” post. But if you’re running your own site…then heh that could work awesome.
And I can’t think of a proper way to forge a click. Although you CAN force a “submit” of a form
January 10th, 2008 at 5:23 pm
I’m looking at setting up some WH sites so I will definitely be looking into this.
Drop me an email sometime if you could. I have some questions I wouldn’t like posted here.
Thanks again XMCP. Learn more here than about anywhere else.
January 10th, 2008 at 6:03 pm
Glad to hear it!
XL, you never use your e-mail address when posting though, so I don’t have it!
January 10th, 2008 at 8:52 pm
Whitehat SEO with affiliate wannabe ambitions (for personal stuff, no blackhat client work) here. lol …
Anyways, how about just doing what money launderers do and send the person through a whole bunch of referrers? Set up a series of instant or near-instant meta-refreshes to kill the blackhat referrer, then drop them on your whitehat page, which has something like ‘this page has moved, you are now being redirected’ and sends them to the merch. Chances are the merchant/network, even if they do check the page, just figure you’re doing longtail ppc. Or you could not even let them review it and cloak the review page with a 404 that you serve to any people not coming through your own set of approved referrers. Or better than a 404, they get a cloaked, legit page with some copy, pretty pictures and calls to action. Then even if they share that with someone else, you don’t care if that’s copied since they don’t know your real traffic source nor the real copy.
Am I way off or missing something? Or is that a smart way to do it?
January 10th, 2008 at 10:11 pm
@Gab:It’s not a terrible way, although 1 meta refresh would do enough to clear the referrer. The thing is, the goal here is pretty much to make sure our traffic does not stand out in any way. That means a referrer should be present, but not a blackhat one. This appears to be the only way to do this while making sure that the affiliate program cannot come back and detect what you’re doing.
January 11th, 2008 at 3:48 am
So does sending them through the whitehat site and cloaking for reviewers work?
January 11th, 2008 at 8:16 am
Excellent article, XMCP! Like XL says, you totally rule. (even if not verbatim ;))
So, anyway, I’m going to try this technique on a couple of sites, although I’m having a bit of trouble understanding the entire process. I already tried some of it and the “breaking out of the iFrames thing didn’t work both in FireFox and in Opera for me. IE was kinder but what the heck; I’ll just keep trying
Thanks again for the great info, XMCP!
January 11th, 2008 at 9:28 am
Another method is to force a click with javascript rather than using window.location. I’m not sure what the referer result would be in this instance however, but its certainly a feasible technique.
January 11th, 2008 at 10:27 am
And for any n00bs listening, don’t just set it up and forget it. Make sure you’re checking exactly what header information is passing on different browsers (each one behaves differently). Live HTTP headers for firefox is a decent start.
Nice educational material my friend
January 11th, 2008 at 11:17 am
Break it down into 2 groups, those that break out of frames and those that don’t. Those that don’t can be had with the straight up iframe solution.
Those that do still present the same problems but we know something about these guys. If they have js code to bust out of frames, what are they going to see in their logs every time that happens? a hit with a referrer followed by a hit with no referrer. So then you have to ask if that makes you stand out.
What if you use your same iframe trick here but make it look like you WH page has a frameset with a tiny innocent looking nav bar across the top when there is no BH referrer. So when the logs look like they always have to bust out of your frame, they come and take a look and you have a frame but when you click the aff link it busts out. Bingo passes human check. They might not be in love with the fact that you’re trying to frame them, but they implemented a technical solution, and if that solution is working for them they may be satisfied.
I understand it’s not about “they *may* be satisfied”. It’s about never generating any suspicion in the first place.
My point is that you can have a factual (automated) answer to the following questions: does the aff link break out of frames, what is the browser version of the current user, how exactly does that browser handle redirects and frames. And that going down this road might lead to some solutions that are cleaner than a one size fits all solution.
And you could also try things like how different browsers handle other redirect codes like 303 and how they handle shifting from https to http on a redirect.
January 11th, 2008 at 12:46 pm
Yes there is a way to forge a click. That is using this technique described above (just toss an iframe on the page to the path you want to click to IF the user is who you want it to be: referred from SE) also, a lot of people don’t know about internet explorer’s wicked click() function.. mwhahahahah
January 11th, 2008 at 2:57 pm
@MadHat: Good advice, and thank you!
Although typically, I don’t trust the advanced JS functions, as they’re handled differently by so many browsers.
@PR: They won’t see two hits, since it’s your WHITEHAT site breaking out of your BLACKHAT site’s iframe. After that, it knows the person has been there once before, so it simply iframes the offer itself. That would be only one hit the affiliate program sees.
@DangerMouse: I will look into that further
@Tob:Is it internet explorer only? If so, it could be decent still with some browser detection in place…could also detect people faking their browser(although there would be false positives for JS turned off)
January 11th, 2008 at 6:34 pm
I do this.
Visitors arrive to verybadpage.html
Javascript autoclick to myownwhitehatpage.html
Referrer cloaking script - if visitors come from verybadpage.html autoclick to http://www.affoffer.com, else display a clean and good looking page with a static link.
Very simple.
Note. I use the javascript autoclick script because with a PHP redirect I had problems parsing the referrer.
http://www.perkiset.org/forum/php/failing_to_parse_the_referer-t171.0.html
This is the autoclick script:
_
isIE = ( ( document.all ) && ( document.getElementById ) );
if( isIE )
document.getElementById( “targetlink” ).click();
else
document.getElementById( “targetlink” ).onclick();
January 11th, 2008 at 6:36 pm
Admin the blog is stripping out a bit of the code.
January 11th, 2008 at 8:22 pm
Yeah, sorry doc. Not too much I can do. Try using the [code] thing, it stripped it out of the comment all together, so I cant rebuild it for ya.
Nice to see you here btw
January 12th, 2008 at 2:24 am
why not use that php method only? php being server side programming will not output the whitehatpage till it gets the referrer.
i think php method works 100% without any flaw
January 12th, 2008 at 3:19 am
@Ashkay: Because that will pass the URL of the blackhat site we’re really getting our traffic from, which we REALLY want to avoid.
January 14th, 2008 at 1:52 am
@ shady Your working too hard. Just cloak the redirect.
January 14th, 2008 at 11:39 am
@roguespammer: It all depends on your goal. Here, we were trying to pass a WH referrer to the affiliate program. Although “cloak” is a broad term, I’ll assume you mean JS/meta refresh redicection, which leaves NO referrer. But yes, it generally works.
January 15th, 2008 at 10:25 pm
Does anyone want to share maybe a working example?
January 17th, 2008 at 1:18 pm
Has anyone attempted to use these auto-click methods to cloak?
January 17th, 2008 at 3:17 pm
Wow. You’re selling out humanity’s best chance of a truly balanced social frontier, and bringing the greed-spawned corruption to the internet! It’s exactly what we all need. Well done.
January 17th, 2008 at 8:58 pm
@DP: Care to elaborate on that little gem? Not quite sure I get you…
January 24th, 2008 at 12:36 am
@admin
not sure why u guys want to make more competition…? or u r not in the aff marketing?
January 24th, 2008 at 6:19 pm
Im trying to hide my refer recieved a little software that supposed to do it.
however i dont know if it does.its basacically making a javascript out of my cpa affialte link. I make a folder for the offer.place the index.html with theis jave code an when i click to my offer it stays on my site inthe directory i made but its the landing page of the offer…
I can see that the link click is tracked but what i need to know how can i see if im sending traffic to there and the lead is filled if the traffic refer wont be shown.
I need some help anyway to check this before hand?
January 24th, 2008 at 9:26 pm
so say cloak the affilaiate link with a java/m refresh redirection will leave no refer…
could that be looked as a email campaign? Saying traffic came from email newsletter link?
Or in a real email link would show referrer from the email software on desktop or isp location?
thanks in advance if any can understand my ramblings
January 24th, 2008 at 10:29 pm
Alright. Possibly, it could be. Also, a few odd browsers will slip through the cracks on this.
However the indicator of an e-mail campaigns is referrers from mail.google.com or f###.yahoo.com….or more likely, spam complaints.
January 25th, 2008 at 12:51 am
Im just trying to hide a “very cheap traffic source” referrer.
Im not to concerned with a few fish sneeking by, but ill give a test run with your experiments above .Thank you for insights.
another quick question anyone ever have to answer for ” blank referrers” ?
February 28th, 2008 at 1:50 pm
How do you track down to the keyword level using this method?
February 28th, 2008 at 1:59 pm
Hudoo, in what context? PPC or SEO?
Either way, I suppose on your end you could substitute in ID number that corresponds with an entry in your database. Use that as a SUBID.
April 7th, 2008 at 8:19 pm
[…] there’s been some talk recently in some blogs on the interwebs regarding cloaking/masking/hiding HTTP REFERERs to protect […]
April 23rd, 2008 at 10:46 am
Could someone post a working example?
April 24th, 2008 at 7:57 am
If I understand right, on the blackhatpage there is this iframe that has the whitehatpage as the source. On the whitehatpage there is the javascript code that makes the whitehatepage break out of the iframe. But what will redirect the user to affilatepage from the whitehatpage? If in your example I replace the window.location part to target the affiliatepage it goes there without problem but leaves no referrer because of the JS. And why do you need identify blackhat source users? Also the last step is confusing. At the end is the affiliate offer going to be iframed on the whitehat page?
Also could you explain how it can be applied to ppc traffic when you want to hide google as referrer, directlink to affiliate offer but user need to be able to navigate back to google’s serp page?
June 18th, 2008 at 10:06 am
Hi,
I have jst come across this post, I dont require redirecting my traffoc throgh a white hat site I just want a simple script or software that allows to block / blank out the referer.
I am sending pcc traffic to cpa networks and I would like to hide the traffic source from them, can anyone reccommend anything. Thanks
July 8th, 2008 at 1:41 pm
[…] Just like the baseball list above, some reasons for arriving without referers makes sense - but others you really have to think about.
July 11th, 2008 at 10:20 pm
Thanks for the information!
By the way, I’ve been using ReferHush.com for a while now and have found it very effective.
July 16th, 2008 at 3:03 pm
I’m trying to get this done by carefully following your steps but for some reason I don’t get the result I’ve expected. That myAffiliate.php should have myWhitehatPage.php as a referer (both in IE and FF).
This is where I get really confused …
“On the whitehat page, check to see if their IP is in the database of people …” if yes “execute a refresh to the affiliate”
“On the whitehat page, have php search for the blackhat referrer …” if yes “//Output the Javascript in Step 5″ and this means go to - http://www.mywhitehatdomain.com/myWhitehatPage.php - and this means that I should go back and do those “if’s”
July 16th, 2008 at 3:07 pm
Also Experiment 5: Frameset and Experiment 6: IFrame have the same source: “”
Why ?
July 16th, 2008 at 3:23 pm
AFAIK, in experiment #3 should be “” instead of “”
July 16th, 2008 at 6:29 pm
Please disregard my other comments. This is what I’ve understood and done so far.
myStartPage.php provides a link to myBlackhatPage.php
myBlackhatPage.php loads a frame of myWhitehatPage.php
myWhitehatPage.php checks if has already the blackhat visitor ip in database.
If it does will header(”Location: http://localhost/seo/myAffiliate.php“); and exit.
If not will check if the visitor comes from the Blackhat website and if it does will break out of the frame and go to myWhitehatPage.php unisng the js provided AND add the blackhat visitor to database.
We have now the myWhitehatPage.php reloaded and out of the frame so we start over.
This time we will have the IP in our database so we go through header(”Location: … to our aff link. In FF will have the right referer (myWhitehatPage.php) bun not in IE. Is there something I’m missing here ?
July 20th, 2008 at 1:41 pm
Problem solved. You can’t. This is a honeypot for ppl like me. I’ve visited this website for at least 50 times this week trying to figure out how is this working and yes … I’ve clicked some other posts because they “sound” interesting. Nice btw. I wounder if this will get posted
August 13th, 2008 at 9:31 am
I cant seem to get this to work for IE6 - anyone else having this problem?
November 30th, 2008 at 1:06 am
hey there again - long time passed heh I still like this article.
December 10th, 2008 at 4:17 pm
I found a way to pass the referer from the white hat site coming from the blackhat site to the CPA Offer.
Host of this site,If your interested Send me a email with your 4 TEST pages, I cant see how you havnt worked it out yourself… Or any of the people who commented