400 Http Errors Using Jsoup in Multithreaded Program

匆匆过客 提交于 2020-01-16 01:03:46

问题


I've created a program that parses html pages. I use jsoup connect function within a callable class inside ThreadPool. The problem is that I'm connecting to the same website and with a thread pool size of 5+, I get IO Exceptions - 400 errors.

How do I not make that happen?


回答1:


If you're getting a 400 HTTP response, check the content of the response for an error message. A 400 means a bad request of some kind: you didn't include all the required information or included malformed information. Some people also use it as kind of a catch-all for when the client did something they don't like. If you're making lots of different requests, examine the ones that caused 400's to see if there's something wrong with them. If they all look right or if you're sending the same requests repeatedly, then maybe the site you're hitting has some kind of rate limiting that disallows you from making too many concurrent requests or too many requests within a particular timeframe. If it's something like that, I'd expect there to be a message in the response telling you what's going on.



来源:https://stackoverflow.com/questions/6885076/400-http-errors-using-jsoup-in-multithreaded-program

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!