Retrieving Data in Java

问题

I'm a java novice. Is it possible to get data from a website and then store it in some sort of data structure? For example, the program gets the value of a stock from yahoo finance at a given time and stores it. Like I said, I'm not that proficient with Java and I'd like to know if this could be done. If it can be, is it very hard to do it?

回答1:

    public class GetYahooData
    {
        public ArrayList<JSONObject> getOutputFromUrl(String url) 
        {
            ArrayList<JSONObject> output = new ArrayList<JSONObject>();
            HttpClient httpClient = new DefaultHttpClient();
            HttpPost httpPost = new HttpPost(url);
            HttpResponse response;
            StringBuilder builder= new StringBuilder();
            JSONObject myjson ;
            JSONArray the_json_array;
            try 
            {
                response = httpClient.execute(httpPost);
                BufferedReader in = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
                char[] buf = new char[8000];
                int l = 0;
                    while (l >= 0) 
                    {
                        builder.append(buf, 0, l);
                        l = in.read(buf);
                    }
                myjson = new JSONObject("{child:"+builder.toString()+"}");
                JSONObject mmm = new JSONObject(builder.toString());
                JSONArray mmmArr = mmm.getJSONArray("status");
                the_json_array = myjson.getJSONArray("child");
                for (int i = 0; i < the_json_array.length(); i++) 
                {
                    JSONObject another_json_object =  the_json_array.getJSONObject(i);//the_json_array.getJSONObject(i);
                    output.add(another_json_object);
                }
            } catch (ClientProtocolException e) {
                System.out.println("ClientProtocolException :"+e);
                e.printStackTrace();
            } catch (IOException e) {
                System.out.println("IOException :"+e);
                e.printStackTrace();
            } catch (JSONException e) {
                System.out.println("JSONException hussain :"+e);
                e.printStackTrace();
            }
            return output;
        }
    }

public class useYahoo
{
    public static void main(String args[])
    {
        String url = "the url you want the response from";
        getYahooData object = new GetYahooData();
        ArrayList<JSONObject> output = object.getOutputFromUrl(url);
    }
}

回答2:

I've used JSoup extensively. If you only need to customize a program to extract data from a website whose layout or structure does not change often, JSoup would be enough and handy.

Assuming you know the basics about how to program(not necessarily familiar with Java) and understand what constitutes the Web(e.g., what is html,dom,etc), I'd expect you to pick up how to do Web scraping with JSoup pretty quick.

回答3:

Yes you can download arbitrary web page into a Java string and parse the contents, however such solution wouldn't be reliable. If the author changes the structure of the website your code would immediately break.

Popular way of doing such integration is by RESTful web service. The data provider would have a set of URL & parameters which you can call, and returns the stock price data (in xml or JSON format)

回答4:

Yes it is possible with the help of webservice.

1. Yahoo or other will expose webservices.
2. Your program will consume that webservice and will fetch the data and do manipulation at your end

来源：https://stackoverflow.com/questions/15103944/retrieving-data-in-java