linux curl save as utf-8

Question

Trying to use linux curl to download an xml file from an url.

Pretty sure that the xml is encoded in UTF-8,

suspecting curl -o doesnt save as UTF-8.

Answer 1

Have you tried adding the Accept-Charset header? I had a similar issue downloading a file which was downloading with the wrong encoding. When I set the Accept-Charset header it works:

curl -H "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7" URL | iconv -f iso8859-1 -t utf-8 > output.xml

Answer 2

curl does not do any conversion of the file it downloads. If the HTTP server serves you the XML in another encoding (e.g., ISO8859-1) that his how curl will save it to disk too.

To workaround your problem, you can use "iconv" as follows:

curl URL | iconv -f iso8859-1 -t utf-8 > output.xml

Hope this help.