How to fetch Android Market data when there is no API?

Question

As far as I can tell, there is no API (official or unofficial) to access information about Apps on the Android Market (info such as Title, Icon, Description, Downloads, Comm

Answer 1

Another option to capture data is using the tcpdump tool that comes with some custom images (like Cyanogenmod and others). You can capture all the packets and then analyze them using Wireshark. This route is easier than hijacking wifi.

Checkout this splendid posts to clear out some things:

• Gathering packets with tcpdump from Android
• Downloading Market applications without the vending app

Update I have been working on this a bit more and @gsmd is right and they use Protocol Buffers to transfer the information and with the technique described above you can capture the packets. Then you'll need to dig into protobuffers to understand the messages.

Answer 2

There is a really good way of getting data from the Developer console. Take a look at: https://github.com/tmurakam/googleplay_dev_scraper

I have it running on my raspberry pi and it works really good. It's pretty simple to install and further automate it with a bash script. I'd really recommend this scraper to anyone interested in getting data from the developer console!

Answer 3

Google have recently added official support for this. Reports are exported to a Google cloud storage bucket

See https://support.google.com/googleplay/android-developer/answer/6135870?hl=en&ref_topic=3450942 for more information.

Answer 4

Unfortunately there is still no official API for Google Play Store (previously known as Android Market) to access Android apps' meta-data (or any other stats).

As mentioned in other answers, you could develop your own web crawler, parse the HTML page and extract the app meta-data you need (e.g. title, descriptions, price, etc). This topic has been covered in this other question.

Another option is to use an open-source library based on ProtoBuf to fetch meta-data about an app, here the link to the project: https://code.google.com/archive/p/android-market-api. This library fetches app meta-data from Google Play on behalf of a valid Google account, but you still need a crawler to "find" which apps are available and schedule their meta-data retrieval. This other open-source project can help you with that: https://code.google.com/archive/p/android-marketplace-crawler.

If you don't want to implement all that by yourself, you could use a third-party service to access Android apps meta-data through a JSON-based API. For instance, 42matters.com (the company I work for) offers an API for both Android and iOS to retrieve apps' meta-data, here more details:

https://42matters.com/app-market-data

In order to get the Title, Icon, Description, Downloads for an app you can use the "lookup" endpoint as documented here:

https://42matters.com/docs/app-market-data/android/apps/lookup

This is an example of the JSON response for the "Angry Birds Space Premium" app:

{
    "package_name": "com.rovio.angrybirdsspace.premium",
    "title": "Angry Birds Space Premium",
    "description": "Play over 300 interstellar levels across 10 planets...",
    "short_desc": "The #1 mobile game of all time blasts off into space!",
    "rating": 4.3046236038208,
    "category": "Arcade",
    "cat_key": "GAME_ARCADE",
    "cat_keys": [
        "GAME_ARCADE",
        "GAME",
        "FAMILY_EDUCATION",
        "FAMILY"
    ],
    "price": "$1.15",
    "downloads": "1,000,000 - 5,000,000",
    "version": "2.2.1",
    "content_rating": "Everyone",
    "promo_video": "https://www.youtube.com/embed/g6AL9YqRHaI?ps=play&vq=large&rel=0&autohide=1&showinfo=0&autoplay=1",
    "market_update": "2015-07-03T00:00:00+00:00",
    "screenshots": [
        "https://lh3.googleusercontent.com/ZmuBQzIy1G74coPrQ1R7fCeKdJmjTdpJhNrIHBOaFyM0N2EYdUPwZaQjnQUtiUDGmac=h310",
        "https://lh3.googleusercontent.com/Xg2Aq70ZH0SnNhtSKH7xg9jCfisWgmmq3C7xQbx6YMhTVAIRqlRJeH8GYtjxapb_qR4=h310",
        "https://lh3.googleusercontent.com/T4o5-2_UP82sj4fSSegbjrGmslNHlfvtEYuZacXMSOC55-7eyiKySw05lNF1QQGO2FeU=h310",
        "https://lh3.googleusercontent.com/f2ennaLdivFu5cQQaVPKsRcWxB8FS5T4Bkoy3l0iPW9-GDDnTVRhvR5kz6l4m8FL1c8=h310",
        "https://lh3.googleusercontent.com/H-9M03_-O9Df1nHr2-rUdjtk2aeBY3bAxnqSX3m2zh_aV8-K1t0qU1DxLXnK0GrDAw=h310"
    ],
    "created": "2012-03-22T08:24:00+00:00",
    "developer": "Rovio Entertainment Ltd.",
    "number_ratings": 20812,
    "price_currency": "$",
    "icon": "https://lh3.ggpht.com/aQaIEGrmba1ENSEgUtArdm3yhJUug7BRWlu_WaspoJusZyHv1rjlWtYqe_qRjE_Kmh1E=w300",
    "icon_72": "https://lh3.ggpht.com/aQaIEGrmba1ENSEgUtArdm3yhJUug7BRWlu_WaspoJusZyHv1rjlWtYqe_qRjE_Kmh1E=w72",
    "market_url": "https://play.google.com/store/apps/details?id=com.rovio.angrybirdsspace.premium&referrer=utm_source%3D42matters.com%26utm_medium%3Dapi"
}

Alternatively, we also have a DATA FEED DUMP to access all mobile app data at once: https://42matters.com/docs/app-market-data/file-dump/android-app-details (on this page you can also find a free sample of data for your tests).

I hope this helps, otherwise feel free to get in touch with me. I know this topic quite well and can point you in the right direction.

Regards,

Andrea

Answer 5

As far as I know, they are using Protocol Buffers. You'll have to sniff phone traffic while it's accessing the market connected over wifi.

Answer 6

There is a Java project on Google code which attempts to build an open source api for the market: http://code.google.com/p/android-market-api/

There're also some JRuby bindings for it: http://github.com/jberkel/supermarket.

However they both require a Google/Market account, and the API is only partially implemented (searching, comments and images at the moment). Also, the reconstructed protocol buffer descriptor (market.proto) could be used to generate bindings for languages other than Java.