Take this code:
using System;
namespace OddThrow
{
class Program
{
static void Main(string[] args)
{
try
{
A try/finally without a catch will use the default handler which does exactly what you see. I use it all the time, e.g., in cases where handling the exception would be covering an error but there's still some cleanup you want to do.
Also remember that output to standard error and standard out are buffered.