W3Techs surveys are now based on the top 10 million sites

Posted by Matthias Gelbmann on 28 May 2013 in News

We used to include the top 1 million sites according to the Alexa ranking to calculate our statistics. When we started W3Techs almost 4 years ago, there were not many sites ranked outside the top 1 million. Alexa assigns a ranking to many more sites now, so we are confident that including the top 10 million sites makes our surveys even more accurate.

We are often asked: "why don't you simply include all sites?" That sounds like a good idea at first, but if you think about it, you can see the problems. Creating a website only takes minutes. You register a domain and you park it somewhere. Many people have thousands of domains, either waiting to be sold ("iphone9.com"), or participating in some link farm, or whatever use the domain grabbers can think of. Most of these domains have no useful content at all, very often nothing but some ads. If we would count these sites, we would to a large extend measure the technologies used by parked domains, not the technologies used by "real" websites. We don't want to do that. Therefore we need some quality criteria to decide whether or not to count a site. The Alexa ranking many not be the perfect quality criteria, but we have not yet seen a better one.

Extending our samples has some impact on the statistics. Technologies that are primarily used for very high traffic sites will get a smaller share from now onwards. In most cases that effect is not very dramatic, because the top 1 million is already a quite large sample, so that the technologies preferred by low traffic sites were already well represented. Nevertheless, some changes will be seen in the next one or two months primarily caused by the sampling effect. That can be a bit disturbing, but we decided to proceed with that change in our surveys, as the positive effects of having greater accuracy in the long run by far outweigh the temporary adjustment effects.

Share this page