Monday, December 23, 2024

Google won’t comment on a potentially massive leak of its search algorithm documentation

Must read

Google’s search algorithm is perhaps the most consequential system on the internet, dictating what sites live and die and what content on the web looks like. But how exactly Google ranks websites has long been a mystery, pieced together by journalists, researchers, and people working in search engine optimization.

Now, an explosive leak that purports to show thousands of pages of internal documents appears to offer an unprecedented look under the hood of how Search works — and suggests that Google hasn’t been entirely truthful about it for years. So far, Google hasn’t responded to multiple requests for comment on the legitimacy of the documents.

Rand Fishkin, who worked in SEO for more than a decade, says a source shared 2,500 pages of documents with him with the hopes that reporting on the leak would counter the “lies” that Google employees had shared about how the search algorithm works. The documents outline Google’s search API and break down what information is available to employees, according to Fishkin.

The details shared by Fishkin are dense and technical, likely more legible to developers and SEO experts than the layperson. The contents of the leak are also not necessarily proof that Google uses the specific data and signals it mentions for search rankings. Rather, the leak outlines what data Google collects from webpages, sites, and searchers and offers indirect hints to SEO experts about what Google seems to care about, as SEO expert Mike King wrote in his overview of the documents.

The leaked documents touch on topics like what kind of data Google collects and uses, which sites Google elevates for sensitive topics like elections, how Google handles small websites, and more. Some information in the documents appears to be in conflict with public statements by Google representatives, according to Fishkin and King.

“‘Lied’ is harsh, but it’s the only accurate word to use here,” King writes. “While I don’t necessarily fault Google’s public representatives for protecting their proprietary information, I do take issue with their efforts to actively discredit people in the marketing, tech, and journalism worlds who have presented reproducible discoveries.”

Google has not responded to The Verge’s requests for comment regarding the documents, including a direct request to refute their legitimacy. Fishkin told The Verge in an email that the company has not disputed the veracity of the leak, but that an employee asked him to change some language in the post regarding how an event was characterized.

Google’s secretive search algorithm has birthed an entire industry of marketers who closely follow Google’s public guidance and execute it for millions of companies around the world. The pervasive, often annoying tactics have led to a general narrative that Google Search results are getting worse, crowded with junk that website operators feel required to produce to have their sites seen. In response to The Verge’s past reporting on the SEO-driven tactics, Google representatives often fall back to a familiar defense: that’s not what the Google guidelines say.

But some details in the leaked documents call into question the accuracy of Google’s public statements regarding how Search works.

One example cited by Fishkin and King is whether Google Chrome data is used in ranking at all. Google representatives have repeatedly indicated that it doesn’t use Chrome data to rank pages, but Chrome is specifically mentioned in sections about how websites appear in Search. In the screenshot below, which I captured as an example, the links appearing below the main vogue.com URL may be created in part using Chrome data, according to the documents.

Chrome is mentioned in a section about how additional links are created.
Image: Google

Another question raised is what role, if any, E-E-A-T plays in ranking. E-E-A-T stands for experience, expertise, authoritativeness, and trustworthiness, a Google metric used to evaluate the quality of results. Google representatives have previously said E-E-A-T isn’t a ranking factor. Fishkin notes that he hasn’t found much in the documents mentioning E-E-A-T by name.

King, however, detailed how Google appears to collect author data from a page and has a field for whether an entity on the page is the author. A portion of the documents shared by King reads that the field was “mainly developed and tuned for news articles… but is also populated for other content (e.g., scientific articles).” Though this doesn’t confirm that bylines are an explicit ranking metric, it does show that Google is at least keeping track of this attribute. Google representatives have previously insisted that author bylines are something website owners should do for readers, not Google, because it doesn’t impact rankings.

Though the documents aren’t exactly a smoking gun, they provide a deep, unfiltered look at a tightly guarded black box system. The US government’s antitrust case against Google — which revolves around Search — has also led to internal documentation becoming public, offering further insights into how the company’s main product works.

Google’s general caginess on how Search works has led to websites looking the same as SEO marketers try to outsmart Google based on hints the company offers. Fishkin also calls out the publications credulously propping up Google’s public claims as truth without much further analysis.

“Historically, some of the search industry’s loudest voices and most prolific publishers have been happy to uncritically repeat Google’s public statements. They write headlines like ‘Google says XYZ is true,’ rather than ‘Google Claims XYZ; Evidence Suggests Otherwise,’” Fishkin writes. “Please, do better. If this leak and the DOJ trial can create just one change, I hope this is it.”

Latest article