How to solve the XML parsing performance issue on Android

我只是一个虾纸丫 提交于 2019-11-28 22:11:14

问题


I have to read a XML file with about ~4000 lines on Android. First I tried the SimpleXML library because it's the easiest and it took about 2 minutes on my HTC Desire. So I thought maybe SimpleXML is so slow because of reflection and all the other magic that this library uses. I rewrote my parser and used the built-in DOM parsing method with some special attention for performance. That helped a bit but it still took about 60 seconds which is still totally unacceptable. After a bit of research I found this article on developer.com. There are some graphs that show that the other two available methods - the SAX parser and Android's XML Pull-Parser - are equally slow. And at the end of the article you'll find the following statement:

The first surprise I had was at how slow all three methods were. Users don't want to wait long for results on mobile phones, so parsing anything more than a few dozen records may mandate a different method.

What might be a "different method"? What to do if you have more than "a few dozen records"?


回答1:


Original answer, in 2012

(note: make sure you read the 2016 update below!)

I just did some perf testing comparing parsers on Android (and other platforms). The XML file being parsed is only 500 lines or so (its a Twitter search Atom feed), but Pull and DOM parsing can churn through about 5 such documents a second on a Samsung Galaxy S2 or Motorola Xoom2. SimpleXML (pink in the chart) as used by the OP ties for slowest with DOM parsing.

SAX Parsing is an order of magnitude faster on both of my Android devices, managing 40 docs/sec single-threaded, and 65+/sec multi-threaded.

Android 2.3.4:

The code is available in github, and a discussion here.

Update 18th March 2016

OK, so its been almost 4 years and the world has moved on. I finally got around to re-running the tests on:

  1. A Samsung Galaxy S3 running Android 4.1.2
  2. A Nexus7 (2012) running Android 4.4.4
  3. A Nexus5 running Android 6.0.1

Somewhere between Android 4.4.4 and Android 6.0.1 the situation changed drastically and we have a new winner: Pull Parsing FTW at more than twice the throughput of SAX. Unfortunately I don't know exactly when this change arrived as I don't have any devices running Android > 4.4.4 and < 6.0.1.

Android 4.1.2:

Android 4.4.4:

Android 6.0.1:




回答2:


I think the best way to work with XML on Android is use VDT-XML library

My XML file contains more then 60 000 lines and VDT-XML handles it as following:

Nexus 5 : 2055 millisec

Galaxy Note 4 : 2498 milisec

You can find more benchmark reports by link : VTD-XML Benchmark

Short example of XML file

 <database name="products">         <table name="category">             <column name="catId">20</column>             <column name="catName">Fruit</column>         </table>         <table name="category">             <column name="catId">31</column>             <column name="catName">Vegetables</column>         </table>         <table name="category">             <column name="catId">45</column>             <column name="catName">Rice</column>         </table>         <table name="category">             <column name="catId">50</column>             <column name="catName">Potatoes</column>         </table> </database> 

Configuration of "build.gradle" file

dependencies {     compile files('libs/vtd-xml.jar') } 

Source code example:

import com.ximpleware.AutoPilot; import com.ximpleware.VTDGen; import com.ximpleware.VTDNav;   String fileName = "products.xml";  VTDGen vg = new VTDGen();  if (vg.parseFile(fileName, true)) {       VTDNav vn = vg.getNav();      AutoPilot table = new AutoPilot(vn);      table.selectXPath("database/table");       while (table.iterate()) {         String tableName = vn.toString(vn.getAttrVal("name"));          if (tableName.equals("category")) {             AutoPilot column = new AutoPilot(vn);             column.selectElement("column");              while (column.iterate()) {                  String text = vn.toNormalizedString(vn.getText());                  String name = vn.toString(vn.getAttrVal("name"));                   if (name.equals("catId")) {                     Log.d("Category ID = " + text);                  } else if (name.equals("catName")) {                     Log.d("Category Name = " + text);                  }               }         }      } } 

Result

Category ID = 20 Category Name = Fruit  Category ID = 31 Category Name = Vegetables  Category ID = 45 Category Name = Rice  Category ID = 50 Category Name = Potatoes 

it works for me and hope it helps you.




回答3:


Using the SAX parser, I can parse a 15,000-line XML file in around 10 seconds on my HTC Desire. I suspect there is some other issue involved.

Are you populating a database from the XML? If so, are you remembering to wrap your entire parse operation in a DB transaction? That alone can speed things up by an order of magnitude.




回答4:


If you are parsing Dates within your XML that can significantly slow down your parsing. With the more recent versions of Android this becomes less of a problem (as they optimised the loading of timezone info)

If you have Dates that are being parsed and you don't need them, then you could use a SAX parser and ignore any of the Date elements.

Or if you can change your XML schema, consider storing the Dates as integers rather than formatted strings.

You mentioned you are making string comparisons, this can be pretty expensive as well. Perhaps consider using a HashMap for the strings you are comparing, this can give noticeable performance benifits.




回答5:


It's very hard to tell you why your code is slow without seeing your code, and it's very hard to believe your assertion that the slowness is due to the XML parser when you haven't provided details of any measurements to prove this.




回答6:


we're using the pull-parser very effectively for 1MB XML Files - and they are read in about 10-20 Seconds on my Desire. So if your code is okay, the speed will be as well. It's obvious that DOM is very slow on a limited memory environment, but pull or SAX really aren't




回答7:


If your parsing from a Socket its the I/O thats taking the time, not the parsing. Try consume the data first, then parse once loaded and measure the performance. If the file is too big then consider a BufferedInputStream with a very large buffer, this should improve performance for you.

I very seriously doubt Simple XML is going to take 2 minutes to load 4000 lines, I realise a handset is going to be a lot slower than a workstation, however I can load 200,000 lines of XML in 600ms on my workstation.




回答8:


Rather than making it a synchronous process, make it asynchronous. You can have a button that starts an IntentService which will process the data for you and will update the results and show a notification when it is done. That way you don't stop the UI thread.



来源:https://stackoverflow.com/questions/7224318/how-to-solve-the-xml-parsing-performance-issue-on-android

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!