curl: read headers from file

Question

After the --dump-header writes a file, how to do read those headers back into the next request? I would like to read them from a file because there are a number of them. <

Answer 1

since curl 7.55.0

Easy:

$ curl -H @header_file $URL

... where the header file is a plain text file with a HTTP header on each line. Like this:

Color: red
Shoesize: 11
Secret: yes
User-Agent: foobar/3000
Name: "Joe Smith"

before curl 7.55.0

curl had no way to "bulk change" headers like that, not even from a file.

Your best approach with an old curl version is probably to instead write a shell script that gathers all the headers from the file and use them, like:

#!/bin/sh
while read line; do
  args="$args -H '$line'";
done
curl $args $URL

Invoke the script like this:

$ sh script.sh < header_file

Answer 2

As answered by @dmitry-sutyagin, if your curl is at least version 7.55.0 you can use the @ notation to read headers from a file:

curl -H @headerfile.txt https://www.google.com/  # requires curl 7.55.0

If your curl is NOT 7.55.0 or newer, there's a useful hack:

• Use the option -K/--config <config file>, and put several -H/--header <header> lines in the text file.

For instance:

1. curl --dump-header foo.txt https://www.google.com/
2. If necessary, dos2unix foo.txt

3. Convert the file to -H 'header' lines, manually or with a script:

   cat foo.txt |
     awk '$1 == "Set-Cookie:"' |
     perl -ne "chomp; next if /^\\s*\$/; if (/'/) { warn; next } print \"-H '\$_'\\n\";" |
     tee headerfile.txt
   

   This might output something like:

   -H 'Set-Cookie: 1P_JAR=2018-02-13-08; [...]'
   -H 'Set-Cookie: NID=123=n7vY1W8IDElvf [...]'
   

4. curl --config headerfile.txt https://www.google.com/

Answer 3

Starting with curl 7.55.0 it can now read headers from a file:

curl -H @filename

It's that easy now.

Answer 4

how about this:

curl -v -H "$(cat headers.txt)" yourhost.com

where headers.txt looks like

Header1: bla
Header2: blupp

works in BASH.

Answer 5

curl $(xargs -a headers.txt printf "-H '%s'") example.org