Membership is FREE – with unlimited access to all features, tools, and discussions. Premium accounts get benefits like banner ads and newsletter exposure. ✅ Signature links are now free for all. 🚫 No AI-generated (LLM) posts allowed. Share your own thoughts and experience — accounts may be terminated for violations.
Sedo Sell Domains

Stop a SE from spidering my site

Status
Not open for further replies.
Paullas

PaullasPaullas is verified member.

Staff member
Joined
Jul 13, 2004
Posts
5,852
Reaction score
387
Does anyone know the full Apache syntax i add in my .htaccess to stop all the SE's from spidering my site. I want it to be full search engine blocked.
 
create a robots.txt file in your root directory with..

User-agent: *
Disallow: /

That would block all SE's that read and follow the robots.txt

Not all bots do but that would work for the main ones.
 
Status
Not open for further replies.

Similar threads

it.com
it.com How to Fix Page Indexing Issues on Your Website
Replies
0
Views
1K
it.comit.com is verified member.
it.com
it.com
it.com What Is a Sitemap and How to Create One for Your Website
Replies
0
Views
1K
it.comit.com is verified member.
it.com
DomainManage
Domain Name Selling Support
Replies
6
Views
265
SystregSystreg is verified member.
Systreg
it.com
it.com Why Is Your Website Traffic Down? 5 Ways To Analyze And Fix Traffic Drops
Replies
0
Views
2K
it.comit.com is verified member.
it.com
Siusaidh
LOKMEG (Tom) and an excellent transaction
Replies
1
Views
2K
NiceNICDomainServer
NiceNICDomainServer

Rule #1: Be Respectful

Do not insult any other member. Be polite and do business. Thank you!

Members online

Total: 1,081 (members: 5, guests: 1,076)
IT.com

Premium Members

Sedo Global Domain Report
Catch Club
dot Hiphop

Latest Comments

Upcoming events

New Threads

Domain Forum Friends

DNForum
ConsultDomain.de
Domaineforum.fr
Domainforum.ro
DN.ca
DomainFragen.de
Inforum.in

Lastest Listings

Our Mods' Businesses

DomainUI Mod
*the exceptional businesses of our esteemed moderators
General chit-chat
Help Users
  • No one is chatting at the moment.
  • Siusaidh AcornBot:
    Siusaidh has left the room.
      Siusaidh AcornBot: Siusaidh has left the room.
      Top Bottom