How to stop NodeJS “Request” module changes request when using proxy

后端 未结 4 1963
小鲜肉
小鲜肉 2021-02-19 22:49

Sorry if this comes off as confusing.

I have written a script using the NodeJS request module that runs and performs a function on a website then returns with the data.

4条回答
  •  陌清茗
    陌清茗 (楼主)
    2021-02-19 23:18

    There are some scenarios that I can think of

    • Proxy is actually adding some headers to the final request (in order to identify you to the server)
    • The website you're trying to reach has your proxy IPs blacklisted (public/paid ones?)

    It really depends on why you need to use that proxy

    • Is it because of network restrictions?
    • Is it because you want to hide the original request address?

    Also, if you have control over the proxy server, can you log the requests being made to the final server?

    My suggestion

    Try writing your own proxy (a reverse one) and host it somewhere. Instead of requesting to https://target.com, to a request to your http[s]://proxy.com/ and let the reverse proxy do the work. Also, remember to disable X headers on the implementation as it will change the request headers

    Reference for node.js implementation:

    https://github.com/nodejitsu/node-http-proxy

    Note: let me know about the questions I made in the comments

提交回复
热议问题