Domain Manage

Googlebot cannot read CSS

Discussion in 'Website Design' started by admin, Jul 28, 2015.

Thread Status:
Not open for further replies.
  1. admin Spain

    admin Administrator Staff Member

    Joined:
    Jun 2004
    Posts:
    10,083
    Likes Received:
    115
    I got a message from the Googlebot saying:

    Site has been working fine for months and is a simple WP theme.

    Any idea what could suddenly cause this?

    Many thanks

    Admin
     
  2. Domain Forum

    Acorn Domains Elite Member

    Joined:
    1999
    Messages:
    Many
    Likes Received:
    Lots
     
  3. baldidiot United Kingdom

    baldidiot Active Member

    Joined:
    Sep 2012
    Posts:
    197
    Likes Received:
    6
    Probably where it's blocked in robots.txt.

    For example if you have the following line in there:

    Disallow: /templates/

    And your css file is:

    /templates/style.css

    It can't read it. Google first mentioned this being an issue a few months ago but looks like they're only just started sending messages out about it.

    You just need to allow the files in robots.txt and you'll be fine. So for example, using the /templates/ folder example you'd put this before the disallow for that folder:

    Allow: /templates/*.js
    Allow: /templates/*.css
     
  4. admin Spain

    admin Administrator Staff Member

    Joined:
    Jun 2004
    Posts:
    10,083
    Likes Received:
    115
    Thanks, all I have is:

    User-agent: *
    Disallow: /wp-admin/
     
  5. scottmccloud

    scottmccloud Active Member

    Joined:
    Sep 2011
    Posts:
    906
    Likes Received:
    26
    I got the exact same message and have the exact same robots.txt as above.
     
  6. Adam H

    Adam H Well-Known Member Exclusive Member

    Joined:
    May 2014
    Posts:
    1,046
    Likes Received:
    84
    Goto Webmaster tools , do a "fetch as google bot" ( Fetch and render )

    When its finished it will say "partial" or "complete", if it says partial it means Google cant access something which could impact on reliability of results.

    If you click the Partial , at the bottom it will give you a list of all thats blocked. It will be down to what you are blocking in the robots.txt.............if its saying wp-admin then its likely your using a plugin like Wordfence which outputs its Log URL from the wp-admin folder
     
  7. nick-harper United Kingdom

    nick-harper Active Member

    Joined:
    Jun 2011
    Posts:
    391
    Likes Received:
    8
    According to Wordpress forums, the fix is to allow full access.....
     
  8. Adam H

    Adam H Well-Known Member Exclusive Member

    Joined:
    May 2014
    Posts:
    1,046
    Likes Received:
    84
    It is. There is very little reason to not allow access to them anyway because the bots you really dont want trying to access those folders will completely ignore the robots.txt file rules

    Have a read of this page : https://yoast.com/wordpress-robots-txt-example/

    More to the point this part :
     
  9. dazc United Kingdom

    dazc Active Member

    Joined:
    Dec 2008
    Posts:
    205
    Likes Received:
    5
    Last edited: Jul 28, 2015
  10. Adam H

    Adam H Well-Known Member Exclusive Member

    Joined:
    May 2014
    Posts:
    1,046
    Likes Received:
    84


    Its not a bug, its been like this for years its just now they are openly telling everyone about it which imo is a good thing. Basically saying there is a "chance" of lower rankings if they cant read your entire site.

    In my opinion its the next step towards to the page layout algo, with blocking script and CSS to a site it may mean they can not tell if your hiding certain page elements with CSS, or arranging elements in a certain way...........such as positioning ads above the fold.

    Obviously speculation on my behalf but it wouldnt surprise me.
     
  11. dazc United Kingdom

    dazc Active Member

    Joined:
    Dec 2008
    Posts:
    205
    Likes Received:
    5
    The reason I say it's a bug is because css, etc isn't in /wp-admin/.

    I don't think there is any point blocking /wp-admin/ for the reasons you stated above but it's strange that it would be causing this current batch of warnings?
     
  12. Adam H

    Adam H Well-Known Member Exclusive Member

    Joined:
    May 2014
    Posts:
    1,046
    Likes Received:
    84
    Some plugins such as Wordfence use the admin-ajax.php in the wp-admin folder to identify site hits, as a result you would get a URL like this for each visitor loaded into the DOM :

    http://www.yourwebsite.com/wp-admin...E21715641DDEB557B359D34&r=0.16598502290435135

    Same for crawlers they are identified as bots, human etc etc. Most brute force plugins act in the same way. Although Google doesnt need to see them as they do not influence any kind of style or script, Google doesnt know that and could think your hiding something.

    Another spin off to that ( and theory's on my behalf through analysis ) is something that was discussed years ago about letting google see the plugin/theme folders for good and bad purposes, things like allowing google to see you've got good security plugins with recent version history which shows a site is active and being maintained and has not been abandoned.

    Flip side to that is plugins such as a search term plugin that was widely used about 5 years ago got whacked and every site using it left a huge foot print so if there was any questionable plugins in use today such as that then it would... like above give Google clear path to follow the patten.
     
  13. dazc United Kingdom

    dazc Active Member

    Joined:
    Dec 2008
    Posts:
    205
    Likes Received:
    5
    Right, that would explain it then. Thanks
     
Thread Status:
Not open for further replies.

Share This Page