Tips for debugging .htaccess rewrite rules

Question

Many posters have problems debugging their RewriteRule and RewriteCond statements within their .htaccess files. Most of these are using a shar

Answer 1

as pointed out by @JCastell, the online tester does a good job of testing individual redirects against an .htaccess file. However, more interesting is the api exposed which can be used to batch test a list of urls using a json object. However, to make it more useful, I have written a small bash script file which makes use of curl and jq to submit a list of urls and parse the json response into a CSV formated output with the line number and rule matched in the htaccess file along with the redirected url, making it quite handy to compare a list of urls in a spreadsheet and quickly determine which rules are not working.

Answer 2

I'll leave this here, maybe obvious detail, but got me banging my head for hours: be careful using %{REQUEST_URI} because what @Krist van Besien say in his answer is totally right, but not for the REQUEST_URI string, because the out put of this TestString starts with a /. So take care:

RewriteCond %{REQUEST_URI} ^/assets/$  
                            ^
                            | check this pesky fella right here if missing

Answer 3

Set environment variables and use headers to receive them:

You can create new environment variables with RewriteRule lines, as mentioned by OP:

RewriteRule ^(.*) - [E=TEST0:%{DOCUMENT_ROOT}/blog/html_cache/$1.html]

But if you can't get a server-side script to work, how can you then read this environment variable? One solution is to set a header:

Header set TEST_FOOBAR "%{REDIRECT_TEST0}e"

The value accepts format specifiers, including the %{NAME}e specifier for environment variables (don't forget the lowercase e). Sometimes, you'll need to add the REDIRECT_ prefix, but I haven't worked out when the prefix gets added and when it doesn't.

Answer 4

If you're planning on writing more than just one line of rules in .htacesss,
don't even think about trying one of those hot-fix methods to debug it.

I have wasted days setting multiple rules, without feedback from LOGs, only to finally give up.
I got Apache on my PC, copied the whole site to its HDD, and got the whole rule-set sorted out, using the logs, real fast.
Then I reviewed my old rules, which been working. I saw they were not really doing what was desired. A time bomb, given a slightly different address.

There are so many pit falls in rewrite rules, it's not a straight logic thing at all.
You can get Apache up and running in ten minutes, it's 10MB, good license, *NIX/WIN/MAC ready, even without install.
Also, check the header lines of your server and get the same version of Apache from their archive if it's old. My OP is still on 2.0; many things are not supported.

Answer 5

Here are a few additional tips on testing rules that may ease the debugging for users on shared hosting

1. Use a Fake-user agent

When testing a new rule, add a condition to only execute it with a fake user-agent that you will use for your requests. This way it will not affect anyone else on your site.

e.g

#protect with a fake user agent
RewriteCond %{HTTP_USER_AGENT}  ^my-fake-user-agent$
#Here is the actual rule I am testing
RewriteCond %{HTTP_HOST} !^www\.domain\.com$ [NC] 
RewriteRule ^ http://www.domain.com%{REQUEST_URI} [L,R=302] 

If you are using Firefox, you can use the User Agent Switcher to create the fake user agent string and test.

2. Do not use 301 until you are done testing

I have seen so many posts where people are still testing their rules and they are using 301's. DON'T.

If you are not using suggestion 1 on your site, not only you, but anyone visiting your site at the time will be affected by the 301.

Remember that they are permanent, and aggressively cached by your browser. Use a 302 instead till you are sure, then change it to a 301.

3. Remember that 301's are aggressively cached in your browser

If your rule does not work and it looks right to you, and you were not using suggestions 1 and 2, then re-test after clearing your browser cache or while in private browsing.

4. Use a HTTP Capture tool

Use a HTTP capture tool like Fiddler to see the actual HTTP traffic between your browser and the server.

While others might say that your site does not look right, you could instead see and report that all of the images, css and js are returning 404 errors, quickly narrowing down the problem.

While others will report that you started at URL A and ended at URL C, you will be able to see that they started at URL A, were 302 redirected to URL B and 301 redirected to URL C. Even if URL C was the ultimate goal, you will know that this is bad for SEO and needs to be fixed.

You will be able to see cache headers that were set on the server side, replay requests, modify request headers to test ....

---

Answer 6

Regarding 4., you still need to ensure that your "dummy script stub" is actually the target URL after all the rewriting is done, or you won't see anything!

A similar/related trick (see this question) is to insert a temporary rule such as:

RewriteRule (.*) /show.php?url=$1 [END]

Where show.php is some very simple script that just displays its $_GET parameters (you can display environment variables too, if you want).

This will stop the rewriting at the point you insert it into the ruleset, rather like a breakpoint in a debugger.

If you're using Apache <2.3.9, you'll need to use [L] rather than [END], and you may then need to add:

RewriteRule ^show.php$ - [L]

At the very top of your ruleset, if the URL /show.php is itself being rewritten.