It looks like they are just querying yahoo for results and ads as a proxy and it sounds like, through some contractual/technical limitations, they have to make that request from a yahoo controlled domain name (duckduckgo-owned-server.yahoo.net). However, they say they fully control the server and yahoo does not get to touch it, as yahoo delegated DNS to them.
The change does not seem malicious and if what they say is true, then the implementation might not be too bad. The volume of traffic they get probably means you cannot profile an individual from Yahoo's side. However, this potentially means that your search history is leaked to Yahoo, albeit in an aggregate manner with other DDG users, which may have attacks that I'm not aware of.
What I can see, however, is a lot of noscript users get startled by this change as they will see DDG use a script from yahoo.net (actually duckduckgo-owned-server.yahoo.net) as by default noscript does not show the full domain.
The cynic in me says that it all starts like this. With multiple, gradual ToS changes, eventually, it will become like any other company out there. And what if Yahoo acquires DDG?
* Offer something for free, with VC money.
* Gain market share since you don't need to worry about profit.
* Realise you need to make money, so start turning into the companies you stole market share from.
Just want to point out that although that may be a model for many startups, DuckDuckGo is already profitable through (non-tracking) advertising and affiliate revenue. More info: https://duck.co/help/company/advertising-and-affiliates
I think it's very, very reasonable to be skeptical of any claims about DDGs actual concern about privacy. Actions speak far louder than words.
Gabe made his first few million by making a classmates.com clone that was the opposite of respectful about privacy. Those actions made clear that he has no ethical or moral stance regarding privacy, and that he will happily violate privacy for profit.
Add to that the fact that he took venture funding. Venture funding aims for a maximized exit price, not a moral stance. The moral stance will disappear the moment your site gains meaningful traction. It's what Ayn Rand (one of your founder's favorte authors) would have wanted.
DDG users who care about privacy are, to be blunt, laughably gullible. Your current business is pro-privacy because:
1) the search context creates sufficient value even without the personal target; and
2) the tech is a dumb layer on top of other tech, and it can't provide personalizaton.
As such, yegg's pretending he cares about privacy... but he's just selling the limitations of his lousy tech as benefits. It's a PT Barnum move. As soon as the tech is better, the privacy will disappear.
I don't want to be rude, but I don't think DuckDuckGo has actually taken much market share to worry about yet, so maybe it's not following the usual startup route.
Maybe. It's not difficult to be profitable though. Servers are cheap. I don't think it's going to trouble Google though, which is a shame. We could do with some serious search competition.
Might have used the wrong term, but the above comment was dead earlier, and several of his other comments are dead despite not seeming to violate any rules.
That link is pretty good at explaining the DNS delegation, TLS and NoScript. However, as DDG runs on AWS, they don't have sole control of their servers anyway.
Edit: output of a DNS/Ping tool I wrote for a book:
Enter a hostname or IP address:
duckduckgo.com
Performing DNS lookup of duckduckgo.com
Complete, duckduckgo.com = 54.229.105.92
Performing reverse DNS lookup of 54.229.105.92
Complete, 54.229.105.92 = ec2-54-229-105-92.eu-west-1.compute.amazonaws.com
Complete, duckduckgo.com = 54.229.105.203
Performing reverse DNS lookup of 54.229.105.203
Complete, 54.229.105.203 = ec2-54-229-105-203.eu-west-1.compute.amazonaws.com
Complete, duckduckgo.com = 46.51.197.89
Performing reverse DNS lookup of 46.51.197.89
Complete, 46.51.197.89 = ec2-46-51-197-89.eu-west-1.compute.amazonaws.com
Complete, duckduckgo.com = 176.34.135.167
Performing reverse DNS lookup of 176.34.135.167
Complete, 176.34.135.167 = ec2-176-34-135-167.eu-west-1.compute.amazonaws.com
Complete, duckduckgo.com = 176.34.155.20
Performing reverse DNS lookup of 176.34.155.20
Complete, 176.34.155.20 = ec2-176-34-155-20.eu-west-1.compute.amazonaws.com
Complete, duckduckgo.com = 176.34.131.233
Performing reverse DNS lookup of 176.34.131.233
Complete, 176.34.131.233 = ec2-176-34-131-233.eu-west-1.compute.amazonaws.com
Pinging duckduckgo.com 4 times
Ping attempt #1 of 4
Success
30 ms
Ping attempt #2 of 4
Success
30 ms
Ping attempt #3 of 4
Success
30 ms
Ping attempt #4 of 4
Success
29 ms
Press any key to exit...
Same source as before. Yes, I assume that's their hosting company.
Enter a hostname or IP address:
startpage.com
Performing DNS lookup of startpage.com
Complete, startpage.com = 37.0.87.7
Performing reverse DNS lookup of 37.0.87.7
Complete, 37.0.87.7 = rt87bb0-37-7.routit.net
Complete, startpage.com = 89.146.4.146
Performing reverse DNS lookup of 89.146.4.146
Complete, 89.146.4.146 = rt4bb146-89-146.routit.net
Complete, startpage.com = 145.131.132.81
Performing reverse DNS lookup of 145.131.132.81
Complete, 145.131.132.81 = rt132bb131-145-81.routit.net
Pinging startpage.com 4 times
Ping attempt #1 of 4
Success
26 ms
Ping attempt #2 of 4
Success
27 ms
Ping attempt #3 of 4
Success
26 ms
Ping attempt #4 of 4
Success
28 ms
Press any key to exit...
Protip for the supertip: I just use !s for startpage. One keystroke lesser. :) It's the one I use most of the time.
Although my default search engine is DDG, for my use (technical and otherwise, with date based searches) DDG is still more like a toy that doesn't provide me the relevant results more than half the time. So I stick to startpage most of the time and sometimes use Google directly (with tracking related precautions/protections taken).
So they data mine this traffic at the nameserver level, just to reasonably say that DDG doesn't share info? Making it technically your computer sharing the info, by way of their forced implementation? Seems like the ad blockers are making the right move here.
This comment is puzzling, since authoritative nameservers only see actual end-user traffic on an occasional basis given the cornucopia of resolver caches in near-unanimous use (as well as the fat TTL on the yahoo.net side of the delegation). This denies Yahoo! the reliable data that you seem to think they've successfully pulled over on everyone, at least if I understand your comment correctly which I'll admit I'm having a hard time doing.
It's also an odd concern given that most everybody these days uses Google or their ISP's resolvers which would be far more interesting "nameserver level data mining." (I have a hard time believing Google successfully logs that traffic in detail, however.)
I can't grok what you're asking me but I'm almost certain the answer is no, as you've already inferred. Extrapolating DNS analytics to actionable intel is incredibly misleading in nearly every case on the service side, particularly since there are a number of caches that consider TTL a guideline and since large shared caches are commonplace.
Understanding what your company is doing if you run the resolver is where it makes more sense. But on the service side? You might as well hire a cat to report metrics by pawing a ouija board, and then you at least get cuddles with your random numbers.
Think of it this way: DNS is basically a different form of ARP, in a certain manner of speaking. I wouldn't try to quantify performance data from either except to troubleshoot operations.
So they data mine this traffic at the nameserver level, just to reasonably say that DDG doesn't share info?
What does this mean? If you mean nameservers might get to see your IP, that isn't true. At most they'll see your primary DNS resolver's IP, but if you use Google DNS or Open DNS they will only see that (I've implemented CDNs, and this is a very big problem for them if they operate at the DNS level - which most do).
It looks like they are just querying yahoo for results and ads as a proxy and it sounds like, through some contractual/technical limitations, they have to make that request from a yahoo controlled domain name (duckduckgo-owned-server.yahoo.net). However, they say they fully control the server and yahoo does not get to touch it, as yahoo delegated DNS to them.
The change does not seem malicious and if what they say is true, then the implementation might not be too bad. The volume of traffic they get probably means you cannot profile an individual from Yahoo's side. However, this potentially means that your search history is leaked to Yahoo, albeit in an aggregate manner with other DDG users, which may have attacks that I'm not aware of.
What I can see, however, is a lot of noscript users get startled by this change as they will see DDG use a script from yahoo.net (actually duckduckgo-owned-server.yahoo.net) as by default noscript does not show the full domain.