I am looking at possible technology choices for queues (or perhaps streams are a better description) in a JVM-based system.
Some requirements:
HSQLDB provides an in-process database engine where you can use RAM, the local disk or a network server to store the database. That might float your boat, especially if you want to seamlessly move to a network solution rather than the local disk later on. Transitioning from small to large queues would then involve moving data from one database to another. There are standard ways to do this, but they might be pretty slow.
There more I think about it, the more I think this is not a good match. For what it's worth, the in-memory DB is very fast in my experience.