Rather than having my unhandled exceptions go into EasyNetQ_Default_Error_Queue I wondered if there is a way that I can explicitly state the name of an Erro
Yes, you can customize the naming conventions by overriding the delegates on the IConventions object (or just create your own implementation of it and register that as a dependency):
https://github.com/EasyNetQ/EasyNetQ/blob/master/Source/EasyNetQ.Tests/ConventionsTests.cs
This should probably work:
var bus = RabbitHutch.CreateBus("host=localhost");
bus.Advanced.Container.Resolve().ErrorExchangeNamingConvention = info => "MyExchangeNaming";
bus.Advanced.Container.Resolve().ErrorQueueNamingConvention = () => "MyErrorQueueNaming";