Googlebot cannot read CSS

[Admin]

I got a message from the Googlebot saying:

Googlebot cannot access CSS and JS files on %URL%

Site has been working fine for months and is a simple WP theme.

Any idea what could suddenly cause this?

Many thanks

Admin

 

[baldidiot]

Probably where it's blocked in robots.txt.

For example if you have the following line in there:

Disallow: /templates/

And your css file is:

/templates/style.css

It can't read it. Google first mentioned this being an issue a few months ago but looks like they're only just started sending messages out about it.

You just need to allow the files in robots.txt and you'll be fine. So for example, using the /templates/ folder example you'd put this before the disallow for that folder:

Allow: /templates/*.js
Allow: /templates/*.css

 

[Admin]

Thanks, all I have is:

User-agent: *
Disallow: /wp-admin/

 

[scottmccloud]

Thanks, all I have is:

User-agent: *
Disallow: /wp-admin/

I got the exact same message and have the exact same robots.txt as above.

 

[Adam H]

Goto Webmaster tools , do a "fetch as google bot" ( Fetch and render )

When its finished it will say "partial" or "complete", if it says partial it means Google cant access something which could impact on reliability of results.

If you click the Partial , at the bottom it will give you a list of all thats blocked. It will be down to what you are blocking in the robots.txt.............if its saying wp-admin then its likely your using a plugin like Wordfence which outputs its Log URL from the wp-admin folder

 

[nick-harper]

According to Wordpress forums, the fix is to allow full access.....

 

[Adam H]

According to Wordpress forums, the fix is to allow full access.....

It is. There is very little reason to not allow access to them anyway because the bots you really dont want trying to access those folders will completely ignore the robots.txt file rules

Have a read of this page : https://yoast.com/wordpress-robots-txt-example/

More to the point this part :

Blocking your /wp-admin/ folder

In the comments many people asked whether I think you should block your /wp-admin/ folder. I think you shouldn’t. The reason is simple: if you block it, but link to it a few times, people will still be able to do a simple “inurl:wp-admin” query and find your site. This type of query is the type of query malicious hackers love to do. If you don’t do anything, WordPress has (by my doing) a robots meta x-http header on the admin pages that prevents search engines from showing these pages in the search results, a much cleaner solution.

 

[dazc]

I think this is a bug currently affecting WMT's globally.
https://www.seroundtable.com/google-warning-googlebot-css-js-20665.html

The advice about not blocking anything is good practice though.

 

[Adam H]

I think this is a bug currently affecting WMT's globally.
https://www.seroundtable.com/google-warning-googlebot-css-js-20665.html

Its not a bug, its been like this for years its just now they are openly telling everyone about it which imo is a good thing. Basically saying there is a "chance" of lower rankings if they cant read your entire site.

In my opinion its the next step towards to the page layout algo, with blocking script and CSS to a site it may mean they can not tell if your hiding certain page elements with CSS, or arranging elements in a certain way...........such as positioning ads above the fold.

Obviously speculation on my behalf but it wouldnt surprise me.

 

[dazc]

Its not a bug, its been like this for years its just now they are openly telling everyone about it which imo is a good thing. Basically saying there is a "chance" of lower rankings if they cant read your entire site.

The reason I say it's a bug is because css, etc isn't in /wp-admin/.

I don't think there is any point blocking /wp-admin/ for the reasons you stated above but it's strange that it would be causing this current batch of warnings?

 

[Adam H]

The reason I say it's a bug is because css, etc isn't in /wp-admin/.

Some plugins such as Wordfence use the admin-ajax.php in the wp-admin folder to identify site hits, as a result you would get a URL like this for each visitor loaded into the DOM :

http://www.yourwebsite.com/wp-admin...E21715641DDEB557B359D34&r=0.16598502290435135

Same for crawlers they are identified as bots, human etc etc. Most brute force plugins act in the same way. Although Google doesnt need to see them as they do not influence any kind of style or script, Google doesnt know that and could think your hiding something.

Another spin off to that ( and theory's on my behalf through analysis ) is something that was discussed years ago about letting google see the plugin/theme folders for good and bad purposes, things like allowing google to see you've got good security plugins with recent version history which shows a site is active and being maintained and has not been abandoned.

Flip side to that is plugins such as a search term plugin that was widely used about 5 years ago got whacked and every site using it left a huge foot print so if there was any questionable plugins in use today such as that then it would... like above give Google clear path to follow the patten.

 

[dazc]

Some plugins such as Wordfence use the admin-ajax.php in the wp-admin folder to identify site hits, as a result you would get a URL like this for each visitor loaded into the DOM .

Right, that would explain it then. Thanks