How to add robots.txt file when i do not have access to production CRDXLite?

PuzanovsP wrote...

Why can't you just place robots.txt in your dam?

Then you would not need additional work apart from rewrite the url RewriteRule ^/robots.txt$  /content/dam/test/robots.txt [PT]

Regards,

Peter

 

I agree with this guy.  This makes it easier to update and you can restrict permissions to the file.

Why can't you just place robots.txt in your dam?

Then you would not need additional work apart from rewrite the url RewriteRule ^/robots.txt$  /content/dam/test/robots.txt [PT]

Regards,

Peter

Hi 

I think you can use this way :

1. Create a field in page properties

ex : you can create a field Robots in page properties of HomePage template. ( the author can update & change it )

2. Under the homepage template.path :  \jcr_root\apps\<your-project>\components\pages\homepage, you can create a JSP file or HTML file ( robots.html or robots.jsp to get page properties ( example : ${pageProperties.robotsTxt}

3. You need to rewrite the url RewriteRule ^/robots.txt$  /content/test/home_pate.robots.html [PT]

Thanks,

Thuong.

Access to production publish crxde is disabled for security reasons but you might have access production author crxde where you can create and then replicate it.

Thanks

Hi 

As mentioned by Scott, this[1] is good article to start with.

Link:- http://www.wemblog.com/2013/06/how-to-implement-robotstxt-sitemapxml.html?spref=bl

// Go to CRXDE or CRXDE light, Or you can directly put them in your CVS under jcr_root folder. You can create different version of robots.txt based environment and domain name.

Then you can configure sling rewriter (org.apache.sling.jcr.resource.internal.JcrResourceResolverFactoryImpl.xml) under /apps/sling/config to redirect to correct robots.txt or any site specific configuration. More information about configuration can be obtained from here http://www.wemblog.com/2012/10/how-to-work-with-configurations-in-cq.html

For Prod something like resource.resolver.virtual="[/robots.txt:/robots-prod.txt] and for all other env
 resource.resolver.virtual="[/robots.txt:/robots-qa.txt]"

Extra reference link:- http://adobe-consulting-services.github.io/acs-aem-commons/features/simple-sitemap.html

I hope this would be helpful to you.

Thanks and Regards

Kautuk Sahni

You need access to CRXDE lite to do this. See: http://www.wemblog.com/2013/06/how-to-implement-robotstxt-sitemapxml.html