I am struggling to better grasp the rationale of exception and error handling in TPL (and with some more luck in .NET 4.5 async/await tasks)
The slightly modified
From what I can deduce the reason this occurs is that the await signals that task to wait for the task to finish. When an exception is thrown, the task is finished (since an exception crashes it) and the exception propagates outwards to your async function where it will be caught. This means that you will always catch one exception with this setup.
To always catch both, remove await and instead use Task.Factory.StartNew(..).Wait(); The Wait function will keep a count of all child processes and will not return until all of them have finished. Since multiple exceptions are thrown (one from each child) they are bundled in a new AggregateException, which later is caught and its children are flattened and the inner exceptions are printed. This should give you the output:
From 1st child
From 2nd child
** AggregateException **
ArgumentException
NullReferenceException