Retrieve large blob from Android sqlite database

Question

I stored chunks of binary data (protobufs) in the sqlite database of an Android app without realizing that Android\'s Cursor can only hold a maximum of 1MB of d

Answer 1

CL. answer will only with blobs<5MB. If you tried to use it with blobs bigger than 5 megabytes you will still get the exception. To fetch large blobs you need to use a library called sqlite4java that uses native calls to the database without using cursors. Here is an example of using this library to fetch a large blob:

SQLiteConnection sqLiteConnection=null;
SQLiteStatement sqLiteStatement=null;
try
{
    File databaseFile = context.getDatabasePath("database.db");
    sqLiteConnection=new SQLiteConnection(databaseFile);
    sqLiteConnection.open();
    sqLiteStatement=sqLiteConnection.prepare("SELECT blob FROM table WHERE id=?");
    sqLiteStatement.bind(1, id);
    sqLiteStatement.step();
    byte[] blob=sqLiteStatement.columnBlob(0);
}
finally
{
    if(sqLiteStatement!=null)
        sqLiteStatement.dispose();
    if(sqLiteConnection!=null)
        sqLiteConnection.dispose();
}

Answer 2

You can read large blobs in pieces. First find out which ones need this treatment:

SELECT id, length(blobcolumn) FROM mytable WHERE length(blobcolumn) > 1000000

and then read chunks with substr:

SELECT substr(blobcolumn,       1, 1000000) FROM mytable WHERE id = 123
SELECT substr(blobcolumn, 1000001, 1000000) FROM mytable WHERE id = 123
...

---

You could also compile your own copy of SQLite and access either the BLOB stream I/O functions or the normal query functions of the C API with the NDK, but that would be too complex in this case.