Block all PDF files and URLs with a parameter in robots.txt
Description: Write a robots.txt rule to block all PDF files, and all URLs which contain ?sessionid=.
Hint:
*matches any text, and$marks the end of the URL.
Solution
User-agent: *
Disallow: /*.pdf$
Disallow: /*?sessionid=
Test result of the rules above
Googlebot /docs/manual.pdf Blocked
Googlebot /docs/manual.pdf?download=1 Allowed
Googlebot /shop/?sessionid=abc123 Blocked
Googlebot /shop/ Allowed
How it works
/*.pdf$matches any path that ends with.pdf. The URL with?download=1after it does not end with .pdf, so it is not matched by this rule./*?sessionid=matches any URL which has that parameter.
Read Robots.txt.
Learn more about the similar topics:
| Tutorials |
|---|
| No Content Found. |
| Exercises & Assignments |
|---|
| No Content Found. |
| Interview Questions & Answers |
|---|
| No Content Found. |