By If you’re a Patreon user, you might be noticing a reduction in traffic to your Patreon page or perhaps even a decrease in new supporters over the next few days or weeks. That’s because someone at Patreon deindexed the entire website from online search engines like Google and Bing due to a meta robots tag with a ‘noindex’ setting that was in the HEAD of all pages on the website.
The issue was spotted by Twitter user Byuu_San who noted that the whole website was suddenly deindexed on search engines and reported the issue to Patreon. Patreon fixed the issue in 4 hours meaning the whole site was probably tagged with a meta robots noindex tag for anywhere from 28 hours to 52 hours before the issue was discovered and fixed.
In the conversation another Twitter user, FloreianeElbers, chimed in with a graph from Sistrix showing an insane drop in traffic.
Oops… pic.twitter.com/YXJIQYq8hm
— Florian Elbers (@florianelbers) October 2, 2019
How could something like this happen to site the size of Patreon?
Most likely it was a break in their process to take the staging site to production, a complete failure of their SEO team, or even just simple miscommunication. Companies the size of Patreon should always have a detailed process for building a website on a staging server and taking it live. An SEO best practice for this is to make sure that the staging site is undiscoverable by users on search engines, often times this leads to the addition of a meta robots nodindex tag to be, at the very least, a fail safe in case other methods fail at keeping a staging site out of search results. Once a staging site is ready to be pushed on to a production server another SEO best practice is to have a qualified and skilled SEO review the staging site both prior to and after the launch to ensure everything goes smoothly, including removing the meta robots noindex tag (or a robots.txt disallow directive) on the live version of the website.
If, as an SEO, you’ve been left out of the push to live conversations and the Development or I.T. is fully in charge, then take it upon yourself to check things like robots.txt and meta robots tags frequently, just in case this same thing happens to you.
They fixed it. My best guess is a dev pushed changes from staging site to production and forgot to ask an SEO to review it post-live.
— Joe Youngblood (@YoungbloodJoe) October 2, 2019
There’s no real reason for big mistakes like this to happen and in cases like this one it’ll end up hurting the services users more than Patreon themselves. Their also not alone, in the past I’ve discovered big SEO errors on Pinterest, The Verge, and Slideshare. The real lesson in this fiasco is to always make sure you have an SEO check the website before it goes live, after it goes live, or both.