如何解析android中的Yahoo网站数据

时间:2018-03-31 19:14:45

标签: android html parsing jsoup

如何在android中为失败者,Gainer&解析雅虎网站最活跃的 给我分享任何想法或示例我的不工作代码如下:

try {
    Document mBlogDocument =  Jsoup.connect("https://sg.finance.yahoo.com/most-active").userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6").get();

    // done for etfs   Jsoup.connect("https://in.investing.com/etfs/india-etfs").timeout(10000).userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6").get().getElementById("etfs").select("tr").iterator();
    Elements links = mBlogDocument.getElementsByTag("a");
    // Locate the content attribute

} catch (IOException e) {
    e.printStackTrace();
}

1 个答案:

答案 0 :(得分:1)

我能够从雅虎网站提取数据,所以确保 -

  1. <uses-permission android:name="android.permission.INTERNET" />
  2. 中添加AndroidManifest.xml
  3. Jsoup操作在后台线程

    @Override
    protected void onCreate(Bundle savedInstanceState) {
    // ....
        new MyTask().execute();
    }
    
    
    private class MyTask extends AsyncTask<Void, Void, String> {
    
        @Override
        protected String doInBackground(Void... params) {
            String title = "";
    
            try {
                Document mBlogDocument = Jsoup.connect("https://sg.finance.yahoo.com/most-active").userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6").get();
                Elements links = mBlogDocument.getElementsByTag("a");
                title = mBlogDocument.title();
                for (Element l: links) {
                    Log.d("log","Element <a>: " + l);
                }
                return title;
            } catch (IOException e) {
                e.printStackTrace();
            }
    
            return title;
        }
    
    
        @Override
        protected void onPostExecute (String result){
            // perform UI stuff
            // ((TextView)findViewById (R.id.textview)).setText (result);
        }
    }
    
  4. 以下是提取匹配股票数据单元格的代码

    private class MyTask extends AsyncTask<Void, Void, String> {
    
        @Override
        protected String doInBackground(Void... params) {
            String title = "";
            String Symbol = "";
            String Name = "";
            String Price = "";
            String Change = "";
            String Change_percentage = "";
            String Volume = "";
            String Avg_Volume = "";
            String Market_cap = "";
            String PE_ratio = "";
    
            try {
    
    
                Document mBlogDocument = Jsoup.connect("https://sg.finance.yahoo.com/most-active").userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6").get();
                Elements tbody_table = mBlogDocument.select("tbody[data-reactid$=73]");
                Log.d("log", "tbody_table size: " + tbody_table.size());
    
                title = mBlogDocument.title();
                System.out.print(title);
    
                Elements Stocks = tbody_table.select("tr");
                for (Element Stock : Stocks) {
                    Elements Stock_item = Stock.select("td");
                    Symbol = Stock_item.get(1).text();
                    Name = Stock_item.get(2).text();
                    Price = Stock_item.get(3).text();
                    Change = Stock_item.get(4).text();
                    Change_percentage = Stock_item.get(5).text();
                    Volume = Stock_item.get(6).text();
                    Avg_Volume = Stock_item.get(7).text();
                    Market_cap = Stock_item.get(8).text();
                    PE_ratio = Stock_item.get(9).text();
    
                    Log.d("log","Symbol: " + Symbol);
                    Log.d("log","Name: " + Name);
                    Log.d("log","Price: " + Price);
                    Log.d("log","Change: " + Change);
                    Log.d("log","Change_percentage: " + Change_percentage);
                    Log.d("log","Volume: " + Volume);
                    Log.d("log","Avg_Volume: " + Avg_Volume);
                    Log.d("log","Market_cap: " + Market_cap);
                    Log.d("log","PE_ratio: " + PE_ratio);
                }
    
    
                return title;
            } catch (IOException e) {
                e.printStackTrace();
            }
    
            return title;
        }
    
    
        @Override
        protected void onPostExecute(String result) {
            //if you had a ui element, you could display the title
            //   ((TextView)findViewById (R.id.myTextView)).setText (result);
        }
    }
    

    请记住,有1114个匹配股票结果,由于某些保护(https://sg.finance.yahoo.com/most-active?offset=0&count=250),我所能得到的是每个URL 250条记录。因此,如果要检索所有匹配股票,您可以对以下网址进行一些循环:

    https://sg.finance.yahoo.com/most-active?offset=0&count=250 https://sg.finance.yahoo.com/most-active?offset=250&count=250 https://sg.finance.yahoo.com/most-active?offset=500&count=250 https://sg.finance.yahoo.com/most-active?offset=750&count=250 https://sg.finance.yahoo.com/most-active?offset=1000&count=114