Querying the MediaStore: Joining thumbnails and images (on ID)

Question

I\'m working on a \"photo gallery\"-type app for Android. It started as a Final Project for the Developing Android Apps at Udacity, so it\'s overall structure (activities, conte

Answer 1

OK, so it seems I finally figured all this out. Thought I'd share this here, for anyone else who might be interested.

What am I trying to achieve?

• Query thumbnails and images on device (via MediaStore)
• Join these into one cursor, ordered descending (newest images on top)
• Handle the case of missing thumbnails

After lots of trial and error, and playing around with the MediaStore, I've learned that the thumbnails table (MediaStore.Images.Thumbnails) can not be expected to be up-to-date, at any given time. There will be images missing thumbnails, and vice versa (orphaned thumbnails). Especially when the camera app takes a new photo, apparently it doesn't immediately create a thumbnail. Not until the Gallery app (or equivalent) is opened, is the thumbnail table updated.

I got various helpful suggestions on how to work myself around this problem, mainly centered on just querying the images table (MediaStore.Images.Media) and then, somehow, extend the cursor with thumbnails one row at a time. While that did work, it caused the app to be extremely slow and consumed a lot of memory for ~2000 images on my device.

It really should be possible to simply JOIN (left outer join) the thumbnails table with the images table, such that we get all images and the thumbnails when these exist. Otherwise, we leave the thumbnail DATA column to null, and just generate those particular missing thumbnails ourselves. What would be really cool is to actually insert those thumbnails into the MediaStore, but that I have not looked into yet.

The main problem with all this was using the CursorJoiner. For some reason, it requires both cursors to be ordered in ascending order, let's say on ID. However, that means oldest images first, which really makes for a crappy gallery app. I found that the CursorJoiner can be "fooled", however, into permitting descending order by simply ordering by ID*(-1):

Cursor c_thumbs = getContext().getContentResolver().query(
                    MediaStore.Images.Thumnails.EXTERNAL_CONTENT_URI,
                    null, null, null, 
                    "(" + MediaStore.Images.Thumnails.IMAGE_ID + "*(-1))");

Cursor c_images= getContext().getContentResolver().query(
                    MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
                    null, null, null, 
                    "(" + MediaStore.Images.Media._ID + "*(-1))");

As long as the rows match up, though, this works fine (the BOTH case). But when you run into rows where either cursor is unique (the LEFT or RIGHT cases) the reversed ordering messes up the inner workings of the CursorJoiner class. However, a simple compensation on the left and right cursors is sufficient to "re-align" the join, getting it back on track. Note the moveToNext() and moveToPrevious() calls.

// join these and return
// the join is on images._ID = thumbnails.IMAGE_ID
CursorJoiner joiner = new CursorJoiner(
        c_thumbs, new String[] { MediaStore.Images.Thumnails.IMAGE_ID },  // left = thumbnails
        c_images, new String[] { MediaStore.Images.Media._ID }   // right = images
);

String[] projection = new String{"thumb_path", "ID", "title", "desc", "datetaken", "filename", "image_path"};

MatrixCursor retCursor = new MatrixCursor(projection);

try {
    for (CursorJoiner.Result joinerResult : joiner) {

        switch (joinerResult) {
            case LEFT:
                // handle case where a row in cursorA is unique
                // images is unique (missing thumbnail)

                // we want to show ALL images, even (new) ones without thumbnail!
                // data = null will cause a temporary thumbnail to be generated in PhotoAdapter.bindView()

                retCursor.addRow(new Object[]{
                        null, // data
                        c_images.getLong(1), // image id
                        c_images.getString(2), // title
                        c_images.getString(3),  // desc
                        c_images.getLong(4),  // date
                        c_images.getString(5),  // filename
                        c_images.getString(6)
                });

                // compensate for CursorJoiner expecting cursors ordered ascending...
                c_images.moveToNext();
                c_thumbs.moveToPrevious();
                break;

            case RIGHT:
                // handle case where a row in cursorB is unique
                // thumbs is unique (missing image)

                // compensate for CursorJoiner expecting cursors ordered ascending...
                c_thumbs.moveToNext();
                c_images.moveToPrevious();
                break;

            case BOTH:

                // handle case where a row with the same key is in both cursors
                retCursor.addRow(new Object[]{
                        c_thumbs.getString(1), // data
                        c_images.getLong(1), // image id
                        c_images.getString(2), // title
                        c_images.getString(3),  // desc
                        c_images.getLong(4),  // date
                        c_images.getString(5),  // filename
                        c_images.getString(6)
                });

                break;
        }
    }
} catch (Exception e) {
    Log.e("myapp", "JOIN FAILED: " + e);
}

c_thumbs.close();
c_images.close();

return retCursor;

Then, in the "PhotoAdapter" class, which creates elements for my GridView and binds data into these from the cursor returned from the ContentProvider (retCursor above), I create a thumbnail in the following manner (when the thumb_path field is null):

String thumbData = cursor.getString(0);  // thumb_path
if (thumbData != null) {
    Bitmap thumbBitmap;
    try {
        thumbBitmap = BitmapFactory.decodeFile(thumbData);
        viewHolder.iconView.setImageBitmap(thumbBitmap);
    } catch (Exception e) {
        Log.e("myapp", "PhotoAdapter.bindView() can't find thumbnail (file) on disk (thumbdata = " + thumbData + ")");
        return;
    }

} else {

    String imgPath = cursor.getString(6);   // image_path
    String imgId = cursor.getString(1);  // ID 
    Log.v("myapp", "PhotoAdapter.bindView() thumb path for image ID " + imgId + " is null. Trying to generate, with path = " + imgPath);

    try {
        Bitmap thumbBitmap = ThumbnailUtils.extractThumbnail(BitmapFactory.decodeFile(imgPath), 512, 384);
        viewHolder.iconView.setImageBitmap(thumbBitmap);
    }  catch (Exception e) {
        Log.e("myapp", "PhotoAdapter.bindView() can't generate thumbnail for image path: " + imgPath);
        return;
    }
}

Answer 2

Here is my test case, which demonstrates the lack of support in CursorJoiner for descending ordered cursors. This, however, is documented specifically in the CursorJoiner source code, so I'm not trying to critize but merely show how this can be circumvented (or hacked).

The test case shows how the assumption of ascending ordering makes the need for "flipping", or reversing, all of the choices made by the CursorJoiner (comparator result, incrementation of cursors, etc). What I'd really like to try next is to modify the CursorJoiner class directly, to try to add support for DESC ordering.

Please note that it seems the part about ordering by ID*(-1) maybe is not strictly necessary for this to work. In the following example, I did not negate the ID columns (plain DESC ordering, not "pseudo-ASC" with negative sequences), and it still works.

Test case

String[] colA = new String[] { "_id", "data", "B_id" };
String[] colB = new String[] { "_id", "data" };

MatrixCursor cursorA = new MatrixCursor(colA);
MatrixCursor cursorB = new MatrixCursor(colB);

// add 4 items to cursor A, linked to cursor B
// the data is ordered DESCENDING
// all cases, LEFT/RIGHT/BOTH, are included
cursorA.addRow(new Object[] { 5, "Item A", 1004 });  // BOTH
cursorA.addRow(new Object[] { 4, "Item B", 1003 });  // LEFT
cursorA.addRow(new Object[] { 3, "Item C", 1002 });  // BOTH
cursorA.addRow(new Object[] { 2, "Item D", 1001 });  // LEFT
cursorA.addRow(new Object[] { 1, "Item E", 1000 });  // BOTH
cursorA.addRow(new Object[] { 0, "Item F", 500 });  // LEFT

// similarily for cursorB (DESC)
cursorB.addRow(new Object[] { 1004, "X" });   // BOTH
cursorB.addRow(new Object[] { 1002, "Y" });   // BOTH
cursorB.addRow(new Object[] { 999,  "Z" });    // RIGHT
cursorB.addRow(new Object[] { 998,  "S" });    // RIGHT
cursorB.addRow(new Object[] { 900,  "A" });    // RIGHT
cursorB.addRow(new Object[] { 1000, "G" });   // BOTH

// join these on ID
CursorJoiner cjoiner = new CursorJoiner(
        cursorA, new String[] { "B_id" },   // left = A
        cursorB, new String[] { "_id" }     // right = B
);

// enable workaround
boolean desc = true;

int count = 0;
for (CursorJoiner.Result joinerResult : cjoiner) {
    Log.v("TEST", "Processing (left)=" + (cursorA.isAfterLast() ? "<empty>" : cursorA.getLong(2))
                + " / (right)=" + (cursorB.isAfterLast() ? "<empty>" : cursorB.getLong(0)));

     // flip the CursorJoiner.Result (unless Result.BOTH, or either cursor is exhausted)
    if (desc && joinerResult != CursorJoiner.Result.BOTH
             && !cursorB.isAfterLast() && !cursorA.isAfterLast())
        joinerResult = (joinerResult == CursorJoiner.Result.LEFT ? CursorJoiner.Result.RIGHT : CursorJoiner.Result.LEFT);

    switch (joinerResult) {
        case LEFT:
            // handle case where a row in cursorA is unique
            Log.v("TEST", count + ") join LEFT. cursorA is unique");

            if (desc) {
                // compensate cursor increments
                if (!cursorB.isAfterLast()) cursorB.moveToPrevious();
                if (!cursorA.isLast()) cursorA.moveToNext();
            }
            break;

        case RIGHT:
            Log.v("TEST", count + ") join RIGHT. cursorB is unique");
            // handle case where a row in cursorB is unique

            if (desc) {
                if (!cursorB.isLast()) cursorB.moveToNext();
                if (!cursorA.isAfterLast()) cursorA.moveToPrevious();
            }
            break;

        case BOTH:
            Log.v("TEST", count + ") join BOTH: " + cursorA.getInt(0) + "," + cursorA.getString(1) + "," + cursorA.getInt(2) + "/" + cursorB.getInt(0) + "," + cursorB.getString(1));
            // handle case where a row with the same key is in both cursors
            break;

    }

    count++;
}
Log.v("TEST", "Join done!");

and the output:

V/TEST: Processing (left)=5 / (right)=1004
V/TEST: 0) join BOTH: 4,Item A,1004/1004,X
V/TEST: Processing (left)=4 / (right)=1002
V/TEST: 1) join LEFT. cursorA is unique
V/TEST: Processing (left)=3 / (right)=1002
V/TEST: 2) join BOTH: 2,Item C,1002/1002,Y
V/TEST: Processing (left)=2 / (right)=999
V/TEST: 3) join RIGHT. cursorB is unique
V/TEST: Processing (left)=2 / (right)=998
V/TEST: 4) join RIGHT. cursorB is unique
V/TEST: Processing (left)=2 / (right)=900
V/TEST: 5) join RIGHT. cursorB is unique
V/TEST: Processing (left)=2 / (right)=1000
V/TEST: 6) join LEFT. cursorA is unique
V/TEST: Processing (left)=1 / (right)=1000
V/TEST: 7) join BOTH: 0,Item D,1000/1000,F
V/TEST: Processing (left)=0 / (right)=---
V/TEST: 8) join LEFT. cursorA is unique
V/TEST: Join done!

Answer 3

The accepted answer got me started on this question, but it contain a couple of small errors.

case LEFT:
            // handle case where a row in cursorA is unique
            // images is unique (missing thumbnail)
case RIGHT:
            // handle case where a row in cursorB is unique
            // thumbs is unique (missing image)

These are backwards. The documentation contradicts itself, and is likely where the mistake got made. From the source code of CursorJoiner:

case LEFT:
        // handle case where a row in cursorA is unique

Then in the enum for Result from the source code:

public enum Result {
    /** The row currently pointed to by the left cursor is unique */
    RIGHT,
    /** The row currently pointed to by the right cursor is unique */
    LEFT,
    /** The rows pointed to by both cursors are the same */
    BOTH
}

So I am guessing this is why you were force incrementing the cursors.

 //compensate for CursorJoiner expecting cursors ordered ascending...
                c_images.moveToNext();
                c_thumbs.moveToPrevious();

The iterator in CursorJoiner automatically increments the cursors for you.

This should be the working code (This code will also merge internal storage and external storage into a single cursor):

        Cursor[] thumbs = new Cursor[2];
        thumbs[0] = mActivity.getContentResolver().query(
                MediaStore.Images.Thumbnails.EXTERNAL_CONTENT_URI,
                new String[]{
                        MediaStore.Images.Thumbnails._ID ,
                        MediaStore.Images.Thumbnails.IMAGE_ID,
                        MediaStore.Images.Thumbnails.DATA
                },
                null,
                null,
                MediaStore.Images.Thumbnails.IMAGE_ID + "*(-1)"
        );
        thumbs[1] = mActivity.getContentResolver().query(
                MediaStore.Images.Thumbnails.INTERNAL_CONTENT_URI,
                new String[]{
                        MediaStore.Images.Thumbnails._ID ,
                        MediaStore.Images.Thumbnails.IMAGE_ID,
                        MediaStore.Images.Thumbnails.DATA
                },
                null,
                null,
                MediaStore.Images.Thumbnails.IMAGE_ID + "*(-1)"
        );
        Cursor thumbCursor = new MergeCursor(thumbs);
        Cursor[] cursors = new Cursor[2];
        cursors[0] = mActivity.getContentResolver().query(
                MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
                new String[]{
                        MediaStore.Images.Media._ID,
                        MediaStore.Images.Media.DATA,
                        MediaStore.Images.Media.ORIENTATION,
                        MediaStore.Images.Media.BUCKET_DISPLAY_NAME,
                        MediaStore.Images.Media.BUCKET_ID,
                        MediaStore.Images.Media.MIME_TYPE
                },
                null,
                null,
                MediaStore.Images.Media._ID + "*(-1)"
        );
        cursors[1] = mActivity.getContentResolver().query(
                MediaStore.Images.Media.INTERNAL_CONTENT_URI,
                new String[]{
                        MediaStore.Images.Media._ID,
                        MediaStore.Images.Media.DATA,
                        MediaStore.Images.Media.ORIENTATION,
                        MediaStore.Images.Media.BUCKET_DISPLAY_NAME,
                        MediaStore.Images.Media.BUCKET_ID,
                        MediaStore.Images.Media.MIME_TYPE
                },
                null,
                null,
                MediaStore.Images.Media._ID + "*(-1)"
        );
        Cursor photoCursor = new MergeCursor(cursors);
        CursorJoiner cursorJoiner = new CursorJoiner(
                thumbCursor,
                new String[]{
                        MediaStore.Images.Thumbnails.IMAGE_ID
                },
                photoCursor,
                new String[]{
                        MediaStore.Images.Media._ID,
                }
        );
        Cursor finalCursor= new MatrixCursor(new String[]{
                MediaStore.Images.Media._ID,
                MediaStore.Images.Media.DATA,
                MediaStore.Images.Media.ORIENTATION,
                MediaStore.Images.Media.BUCKET_DISPLAY_NAME,
                MediaStore.Images.Media.BUCKET_ID,
                MediaStore.Images.Media.MIME_TYPE,
                "thumb_data"
        });
        for (CursorJoiner.Result joinerResult : cursorJoiner) {
            switch (joinerResult) {
                case RIGHT:
                    finalCursor.addRow(new Object[]{
                            photoCursor.getLong(photoCursor.getColumnIndex(MediaStore.Images.Media._ID)),
                            photoCursor.getString(photoCursor.getColumnIndex(MediaStore.Images.Media.DATA)),
                            photoCursor.getLong(photoCursor.getColumnIndex(MediaStore.Images.Media.ORIENTATION)),
                            photoCursor.getString(photoCursor.getColumnIndex(MediaStore.Images.Media.BUCKET_DISPLAY_NAME)),
                            photoCursor.getLong(photoCursor.getColumnIndex(MediaStore.Images.Media.BUCKET_ID)),
                            photoCursor.getString(photoCursor.getColumnIndex(MediaStore.Images.Media.MIME_TYPE)),
                            null
                    });
                    break;
                case BOTH:
                    finalCursor.addRow(new Object[]{
                            photoCursor.getLong(photoCursor.getColumnIndex(MediaStore.Images.Media._ID)),
                            photoCursor.getString(photoCursor.getColumnIndex(MediaStore.Images.Media.DATA)),
                            photoCursor.getLong(photoCursor.getColumnIndex(MediaStore.Images.Media.ORIENTATION)),
                            photoCursor.getString(photoCursor.getColumnIndex(MediaStore.Images.Media.BUCKET_DISPLAY_NAME)),
                            photoCursor.getLong(photoCursor.getColumnIndex(MediaStore.Images.Media.BUCKET_ID)),
                            photoCursor.getString(photoCursor.getColumnIndex(MediaStore.Images.Media.MIME_TYPE)),
                            thumbCursor.getString(thumbCursor.getColumnIndex(MediaStore.Images.Thumbnails.DATA)),
                    });
                    break;
            }

        }
        photoCursor.close();
        thumbCursor.close();