robots.txt might be ruining your Mobile-Friendly Ranking with Google

robots.txt might be ruining your Mobile-Friendly Ranking with Google

Sometimes our computers try a little too hard to help us and end up hurting us instead. In Norwegian we call it a “Bear’s Favor”. Such may be the case with your robots.txt file and Google’s upcoming Mobile-Friendly Ranking.

Test your site

An unstyled WordPress page
Screenshot of without styles applied. Decidedly mobile-unfriendly

If  you haven’t done so already, head over to Google’s Mobile-Friendly Test page and enter your URL. If all is well and you’re running a responsive theme you should get a nice green all-clear sign meaning you won’t be penalized for mobile unfriendliness. However, there is a good chance your site will fail this test even if it is responsive because your robots.txt file is blocking your themes folder.

If so you’ll see something like the screenshot to the right.

If this is what you see your site will fail the mobile friendly test because your content is too wide, all your links are too close together, and everything is totally out of whack. Which is true.

Check your robots

The great thing is once you’ve run the test and failed, Google will provide you with directions to check what’s going on. What you need to do is go to Google Webmaster Tools and check your robots.txt file. Once on the page, enter the URL to your theme stylesheet (http://[]/wp-content/themes/[themename]/style.css) and see if it’s being blocked. If it is the tool will highlight the line in robots.txt that is blocking the stylesheet.

Fix robots.txt

Fixing robots.txt is relatively easy. If you are not using an SEO plugins you may have to FTP or SSH into your site and change the file manually. If you are using Yoast’s WordPress SEO plugin you can edit robots.txt directly by going to Tools -> File Editor.

Remove the offending line of text (probably Disallow: /wp-content/themes/), save the file, and go back to Google’s Mobile-Friendly Test page. Run the test again, the new robots.txt file will kick in, and you’ll see your site displayed in the preview with your styles and JavaScripts applied.

Easy as ?

Plugins, Automatic Updates, and the Average WordPress User

+-*The fallout over the automatic update of Yoast’s WordPress SEO plugin shows the WordPress community is suffering from a severe case of Developer Goggles. Background Yesterday (March 11th, 2015) Joost de Valk released a security update to the popular WordPress SEO plugin, currently running on over 1 million WordPress sites around the world. The release came after a […]

The Spirit of Flickr and the Problem of Intent

+-*I’m trying something new here: Audio versions of my essays. So, if you want to listen to me read this essay rather than read it, hit the play button below and let me know what you think about this idea! Over the weekend a conversation has started over the move by photo sharing site Flickr to […]

Simone 2.0

Simone 2.0: Custom backgrounds, sidebar position, RTL support, and more

+-*After much ado Simone 2.0 is now live in the WordPress Theme Directory. The theme is cresting 50,000 downloads already and for the new version I decided to bake in some of the most requested new features. Custom Background One of the standard WordPress theme features that were not available in the original version of […]

Thinking Inside the Box – My WordCamp San Francisco 2014 Lightning Talk

+-*What started as a challenge ended up as a lightning talk at WordCamp San Francisco 2014: Explain the CSS Box Model in an understandable way to a crowd of mainly bloggers and designers. In 5 minutes. With no live code. The video slides are unfortunately largely cut out of this video so you don’t get […]

Challenges with the new method for inheriting parent styles in WordPress child themes

+-*The new recommended code for inheriting parent theme styles in WordPress child themes have uncovered several issues with the concept of child themes. What follows is a rundown of how things work, what problems are caused by the status quo, and finally some proposed solutions.

Why Android Fragmentation is Irrelevant to Web Designers

+-*We’ve all seen them: Pictures of an array of hundreds of Android devices accompanied by some snide remark about how “Android Fragmentation” is making it impossible to build and test anything for the mobile web. I’m here to tell you that’s not true. You content doesn’t care about the size of the visiting browser. Neither should […]