How to extract a value from the Yahoo Finance Cash Flow statement with Java (Android)?

问题

This is a follow up to the solution to this question: How to extract data from HTML page source of (a tab within) a webpage?

I am trying to do the same for Cash Flow Stmt at finance.yahoo.com/quote/AAPL/cash-flow?p=AAPL - using

.getJSONObject("dispatcher") 
.getJSONObject("stores") 
.getJSONObject("QuoteSummaryStore") 
.getJSONObject("cashflowStatementHistory") 
.getJSONArray("cashflowStatements");

trying to extract the value of the key trailingFreeCashFlow - BUT, it failes with the error "No value for trailingFreeCashFlow".

 public static Map<String, Map<String, String>> getCashFlowTableNames() {
        final Map<String, String> cashFlow = new LinkedHashMap<String, String>() {
            {
                put("trailingFreeCashFlow", "trailingFreeCashFlow");
            }
        };

        Map<String, Map<String, String>> allTableNames = new LinkedHashMap<String, Map<String, String>>() {
            {
                put("cashFlow", cashFlow);
            }
        };
        return allTableNames;
    }

and

public static String getCashFlowYear(String requestURL) throws IOException {
    String userAgent1 = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 OPR/56.0.3051.43";
    try {
        Document doc1 = Jsoup.connect(requestURL).userAgent(userAgent1).get();
        Elements scriptTags = doc1.getElementsByTag("script");
        String re = "root\\.App\\.main\\s*\\=\\s*(.*?);\\s*\\}\\(this\\)\\)\\s*;";

        for (Element script : scriptTags) {
            Pattern pattern = Pattern.compile(re, Pattern.DOTALL);
            Matcher matcher = pattern.matcher(script.html());

            if (matcher.find()) {
                String data = matcher.group(1);
                //Log.e("CashFlowData", data);

                JSONObject jo = new JSONObject(data);
                JSONArray table = getCashFlowTable(jo);
                JSONArray tableQ = getCashFlowTableQ(jo);
                Map<String, Map<String, String>> tableNames = getCashFlowTableNames();

                String[] dates = getDates(table);
                String[] datesQ = getDates(tableQ); //works
                List<String> tableData = new ArrayList<>();

                for (Map.Entry<String, Map<String, String>> tableEntry : tableNames.entrySet()) {
                    tableData.add(tableEntry.getKey());
                    tableData.addAll(Arrays.asList(dates));

                    for (Map.Entry<String, String> row1 : tableEntry.getValue().entrySet()) {
                        String[] tableRow1 = getRow(table, row1.getValue());
                        tableData.add(row1.getKey());
                        for (String column : tableRow1) {
                            tableData.add(column);
                        }
                    }
                }
                cashFlowData = TextUtils.join(" ", tableData);
                cashFlowData = cashFlowData.replaceAll("[^a-zA-Z0-9 /-]", "");
                cashFlowData = cashFlowData.trim().replaceAll("(?<=[A-Za-z])\\s+(?=[A-Za-z])", "");
                Log.e("cashFlowData", cashFlowData);

                List<String> tableDataQ = new ArrayList<>();

                for (Map.Entry<String, Map<String, String>> tableEntry : tableNames.entrySet()) {
                    tableDataQ.add(tableEntry.getKey());
                    tableDataQ.addAll(Arrays.asList(datesQ));

                    for (Map.Entry<String, String> row1 : tableEntry.getValue().entrySet()) {
                        String[] tableRow1 = getRow(tableQ, row1.getValue());
                        tableDataQ.add(row1.getKey());
                        for (String column : tableRow1) {
                            tableDataQ.add(column);
                        }
                    }
                }
                cashFlowDataQ = TextUtils.join(" ", tableDataQ);
                cashFlowDataQ = cashFlowDataQ.replaceAll("[^a-zA-Z0-9 /-]", "");
                cashFlowDataQ = cashFlowDataQ.trim().replaceAll("(?<=[A-Za-z])\\s+(?=[A-Za-z])", "");
                //Log.e("balanceDATAQ", balanceDataQ);
            }
        }
    } catch (Exception e) {
        Log.e("err", "err", e);
    }
    return cashFlowData;
}

Any suggestions?

Thanks!

来源：https://stackoverflow.com/questions/66095236/how-to-extract-a-value-from-the-yahoo-finance-cash-flow-statement-with-java-and

标签

java

android

web-scraping

yahoo-finance