发表新帖
发表新帖

Get final URL after curl is redirected

前端 未结
关注
11 1806
清酒与你
清酒与你 2020-12-04 07:38

I need to get the final URL after a page redirect preferably with curl or wget.

For example http://google.com may redirect to http://www.google.com

相关标签:
11条回答
  • [愿得一人]
    2020-12-04 07:41

    Thanks, that helped me. I made some improvements and wrapped that in a helper script "finalurl":

    #!/bin/bash
curl $1 -s -L -I -o /dev/null -w '%{url_effective}'
    • -o output to /dev/null
    • -I don't actually download, just discover the final URL
    • -s silent mode, no progressbars

    This made it possible to call the command from other scripts like this:

    echo `finalurl http://someurl/`
    0 讨论(0)
  • 一个人的身影
    2020-12-04 07:47

    as another option:

    $ curl -i http://google.com
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Sat, 19 Jun 2010 04:15:10 GMT
Expires: Mon, 19 Jul 2010 04:15:10 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 1; mode=block

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

    But it doesn't go past the first one.

    0 讨论(0)
  • 情书的邮戳
    2020-12-04 07:51

    You could use grep. doesn't wget tell you where it's redirecting too? Just grep that out.

    0 讨论(0)
  • 一整个雨季
    2020-12-04 07:53

    I'm not sure how to do it with curl, but libwww-perl installs the GET alias.

    $ GET -S -d -e http://google.com
GET http://google.com --> 301 Moved Permanently
GET http://www.google.com/ --> 302 Found
GET http://www.google.ca/ --> 200 OK
Cache-Control: private, max-age=0
Connection: close
Date: Sat, 19 Jun 2010 04:11:01 GMT
Server: gws
Content-Type: text/html; charset=ISO-8859-1
Expires: -1
Client-Date: Sat, 19 Jun 2010 04:11:01 GMT
Client-Peer: 74.125.155.105:80
Client-Response-Num: 1
Set-Cookie: PREF=ID=a1925ca9f8af11b9:TM=1276920661:LM=1276920661:S=ULFrHqOiFDDzDVFB; expires=Mon, 18-Jun-2012 04:11:01 GMT; path=/; domain=.google.ca
Title: Google
X-XSS-Protection: 1; mode=block
    0 讨论(0)
  • 没有蜡笔的小新
    2020-12-04 07:55

    The parameters -L (--location) and -I (--head) still doing unnecessary HEAD-request to the location-url.

    If you are sure that you will have no more than one redirect, it is better to disable follow location and use a curl-variable %{redirect_url}.

    This code do only one HEAD-request to the specified URL and takes redirect_url from location-header:

    curl --head --silent --write-out "%{redirect_url}\n" --output /dev/null "https://""goo.gl/QeJeQ4"

    Speed test

    all_videos_link.txt - 50 links of goo.gl+bit.ly which redirect to youtube

    1. With follow location

    time while read -r line; do
    curl -kIsL -w "%{url_effective}\n" -o /dev/null  $line
done < all_videos_link.txt

    Results:

    real    1m40.832s
user    0m9.266s
sys     0m15.375s

    2. Without follow location

    time while read -r line; do
    curl -kIs -w "%{redirect_url}\n" -o /dev/null  $line
done < all_videos_link.txt

    Results:

    real    0m51.037s
user    0m5.297s
sys     0m8.094s
    0 讨论(0)
  • 遇见更好的自我
    2020-12-04 07:55

    Can you try with it?

    #!/bin/bash 
LOCATION=`curl -I 'http://your-domain.com/url/redirect?r=something&a=values-VALUES_FILES&e=zip' | perl -n -e '/^Location: (.*)$/ && print "$1\n"'` 
echo "$LOCATION"

    Note: when you execute the command curl -I http://your-domain.com have to use single quotes in the command like curl -I 'http://your-domain.com'

    0 讨论(0)
 看不清?
提交回复
热议问题