One of the most common complaints critics (and competitors) have against Google is that the company — despite its numerous open products and initiatives — is very secretive about the countless algorithms that power its crown jewels, Google Search. In other words, search is a black box. Google’s argument has generally been that it needs to be this way — if it told everyone exactly how the algorithm worked, then it would be much easier to game, and search results would suffer.
In any case there’s plenty of mystery around Search, which is why the video posted today to the Google blog is so interesting: Google has given a quick video walk-through detailing how its engineers gradually adjust the algorithm, which — according to the video — gets tweaked more than 500 times per year.
The video is only four minutes long and is worth watching in its entirely, but here’s a quick rundown on how a change to search goes live:
- First, Google engineers identify a set of “motivating searches” that aren’t working as well as Google would like them to.
- Next, engineers try to identify various signals (Google doesn’t really get into what these signals entail) that could be used to better answer that query.
- These new results are then rated by a trained worker that isn’t a Google employee.
- Next, the results are pushed live to a small number of users.
- An analyst is then assigned to objectively look at the change and how it’s performing in tests.
- And finally, the proposed change is presented to the search quality team, which either approves or denies it.
The video also goes into a quick anecdote about what Google calls ‘Full Page Replacement’. This refers to the times Google looks at your query, decides you’ve misspelled it, and then presents the results for the correct spelling, rather than just suggesting the proper spelling and displaying the results for the misspelled results. To test this, Google looked at how many times users explicitly clicked on the link for the results for their original query (which Google believed was misspelled) — Google decided that so long as this ‘escape hatch’ was only clicked 1 in 50 times, it was a good change. And it passed the test.