Android throw DeadObjectException with LOG: Transaction failed on small parcel; remote process probably died

Question

07-22 04:38:07.933  1579  3338 E JavaBinder: !!! FAILED BINDER TRANSACTION !!!  (parcel size = 352)
07-22 04:38:07.933  1579  3338 W BroadcastQueue: Can\'t deliver b         


        

Answer 1

I finally found the root cause, it happened in the binder kernel.

For now, I discovered two reasons for what can cause a DeadObjectException to be thrown in BroadcastQueue and therafter a RemoteServiceException in ActivityThread in the app:

1. There are no more asynchronous space to execute the binder transaction when AMS sends a one-way binder call to ActivityThread in order to trigger BroadcastReceiver.onReceive.

Related code shown below:

kernel/msm-4.4/drivers/android/binder_alloc.c
290     if (is_async &&
291            alloc->free_async_space < size + sizeof(struct binder_buffer)) {
292           binder_alloc_debug(BINDER_DEBUG_BUFFER_ALLOC,
293                         "%d: binder_alloc_buf size %zd failed, no async space left\n",
294                           alloc->pid, size);
295            eret = ERR_PTR(-ENOSPC);
296              goto error_unlock;
297    }

Therefore, this will not "end up destabilizing the system". It will only influences the application itself.

2. The user application had been force closed because BroadcastQueue send scheduleCrash binder call to ActivityThread. The root cause of this problem is that there are no binder buffer in the application side because some binder threads occupy most of it.

The bug can be triggered with the following steps:

1. Process1 sends large data (e.g. 980kB) to Process2, the Process2 need sleep for 30 seconds, and the large binder buffer will not be released.
2. Process1 sends a broadcast to Process2, consisting of e.g. 50kB data. That would go beyond the make the buffer capacity of 1016kB, since 980kB + 50kB is larger than the buffer capacity.
3. BroadcastQueue will throw a DeadObjectException and then pass scheduleCrash to ActivityThread in the application side.

Here is the code:

kernel/msm-4.4/drivers/android/binder_alloc.c
 315     if (best_fit == NULL) {
...
341         pr_err("%d: binder_alloc_buf size %zd failed, no address space\n",
342                   alloc->pid, size);
343         pr_err("allocated: %zd (num: %zd largest: %zd), free: %zd (num: %zd largest: %zd)\n",
344                       total_alloc_size, allocated_buffers, largest_alloc_size,
345                  total_free_size, free_buffers, largest_free_size);
346            eret = ERR_PTR(-ENOSPC);
347              goto error_unlock;
348    }

In conclusion, DeadObjectException can be thrown even if the application process haven't died.

The root cause is most likely because of full binder buffer for the application and does not influence the system.

So I think it is not necessary to make the application crash after catching a DeadObjectException in BroadcastQueue.

Answer 2

Basically everything Rick Ai's answer to their own question is correct but here is a a real world example:

If your app creates and registers a whole bunch of BroadcastReceiver instances all listening to the same action--perhaps due to a leak or bug in your app--then the ActivityManagerService in the system process will invoke android.app.IApplicationThread method scheduleRegisteredReceiver for each registered instance. Notice that the binder transaction for this particular method is oneway. Since it is oneway each invocation will return immediately and the calls to the binder driver will occur very rapidly before each transaction is complete thus effectively running them all in parallel.

Lets say you have 100 receivers in your app and the broadcast being received contains 20 KiB of data. Now you've got 2 MiB trying to pass through the binder driver and it will fail due to the limit of 1 MiB.

In kernel logs you will see:

binder: 1282:1298 transaction failed 29201/-28, size 28052-8 line 3072

So beware leaking BroadcastReceiver and beware oneway binder transactions. Note that apparently the AIDL file may not declare a method oneway but it may end up that way if the AIDL compiler decides it is possible.