php inserts HEX number of characters before the content

Question

I\'m moving a website to a new server. (Old server had php 5.3.2, the new one has php 5.5.9) Centos, httpd Apache/2.2.26.

I\'ve copied files and it works fine except

Answer 1

Had this problem for serving HTML as well as plain text. EXPLICITLY setting header Transfer-Encoding to "chunked"

header("Transfer-Encoding: chunked");

indeed solved the problem (not sure how reliably though). That Transfer-Encoding header was being added automatically even if i didn't set it - but the problem was still there. Setting the header explicitly in PHP code solved it, but not exactly - see below. Tested using websniffer.cc, PHP 7.4.9, CentOS 7.8

UPDATE: adding the header might confuse some clients. https://securityheaders.com/ failed to connect to the site with this header explicitly set on (regardless of content length)

Answer 2

This is not PHP nor a BOM. You have a problem with Content-Transfer-Encoding.

The server is sending along a Chunked encoding (this is usually done when the Content-Length is unavailable) which the client apparently doesn't know how to handle, or the header combination makes the client believe that it can bypass dechunking.

Therefore, what is actually the "length of next chunk" gets interpreted by the client, and you see it as a hex bit before the content:

05
These
05
 Are 
03
the
1F
 first characters of the senten
03
ce.

instead of

Content-Length: 48
These are the first characters of the sentence.

(I calculated the lengths by eye, they're probably wrong)

The likely cause is that you have some kind of buffering which interferes with Content Encoding. If everything stays in the buffer, all well and good, a Content-Length is available, gets sent and Bob's your uncle. But if you send more than the 8000 bytes of the buffer, the buffer is flushed and something untoward happens. Try looking in the documentation for zlib and output buffering, you might have some conflict in php.ini between what Apache does and what PHP does.

Interesting links that show how exactly 8000 is (was?) a buffer size used in some portions of apache:

https://bugs.php.net/bug.php?id=45945

https://serverfault.com/questions/366996/how-can-the-apache-2-2-deflate-module-length-limit-be-increased

This last link suggests me to suggest you to try checking whether zlib, mod_gzip, ob_gzhandler and/or mod_deflate are enabled, and possibly conflicting, or to try exchanging the one for the other.

Update (as per comment)

Adding the appropriate Transfer-Encoding header fixes the problem.

So what really happened? We know that the output was correctly chunked-encoded (i.e. the chunking itself was correct), and the client was able to interpret it after being told to. So what was missing was simply the knowledge of the content being chunked, which looks absurd and a good bit buggy: for whatever chunked the content had to know it would get chunked (d'oh!), so it had responsibility of adding the appropriate header, and yet it did not (or it did, but something else stripped it), until the OP rectified the situation adding a header himself by hand.

The problem is now solved, but I think there must be still a bug lingering somewhere in the workflow, either in the modules, the application, or in how the headers are processed.

Answer 3

As discussed in the PHP chat room yesterday 9F 1A is the BOM for UTF-16BE Encoding

In my case only two headers are sent from php script:

header("HTTP/1.1 200 OK"); header("Status: 200");

What's going here? Why are you setting both? One is a proper http response, the other is a CGI response. You shouldn't need to set either, as PHP will default to a 200 OK if the script completes. And setting a CGI style header is probably wrong, unless you're definitely running PHP in CGI mode through Apache.