It’s been just over a year since Twitter killed Google’s realtime search by not renewing their agreement with the search giant. Previously, Google had access to every public tweet and it was those tweets that Google would use to populate its realtime search results. Since that agreement ended, internet powerhouses haven’t been on the best terms, but it now appears that Twitter is getting back into the SEO game, albeit indirectly.
It all started at the end of September when Twitter updated its robots.txt file to allow millions of pages to be crawled and indexed by search engines like Google and Bing, which were previously shunned from the majority of Twitter’s properties. According to Search Engine Land, Twitter confirmed the change to the publication by saying the following:
This change will help people find popular and helpful Twitter pages, such as the #olympics hashtag page (https://twitter.com/search/?q=%23olympics)
Hashtags have long been one of the most unique features of Twitter, which is why Google+ (among others) used the idea for their own networks. The change to Twitter’s robots.txt file is putting a much bigger focus onto hashtags, potentially making them more important than ever before. However, the update doesn’t open up Twitter as a whole. There are still restrictions for search engines. For instance, it still won’t allow Google, Bing, etc. to crawl the site for users, videos or images. The update was designed to make make discovery easier, better and faster. Is this plan going to work, though?
Search results within search results
One issue that remains to be seen is whether or not Google and Bing actually want these pages in their SERPs. According to Google’s Webmasters Guidelines, webmasters are to “Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.” This is the exact opposite of what Twitter is currently doing. If you search Google for Twitter search pages, you’ll definitely find those pages in your SERPs, but considering Google and Twitter haven’t been playing nice, only time will tell how long they’ll be there.
SEO-friendly user directory
The other thing Twitter has done to placate to the search engines was create a new ultra SEO-friendly user directory. This directory makes Twitter’s user profiles much easier for Google, Bing, et al to find. The directory itself follows Google’s recommended rules for SEO, including no more than 100 links per page. Because the directory hasn’t been online for very long, we don’t know exactly how it will show up in the different search engines. What we do know, though, is that in spite of its spat with Google, Twitter is making an active effort to make parts of its site much more accessible to search engines.