Are they any decent on-disk implementations of Java's Map?

Question

I\'m looking for an on-disk implementation of java.util.Map. Nothing too fancy, just something that I can point at a directory or file and have it store its co

Answer 1

If you are looking for key-object based structures to persist data then NoSQL databases are a very good choice. You'll find that some of them such MongoDB or Redis scale and perform for big datasets and apart from hash based look ups they provide interesting query and transactional features.

In essence these types of systems are a Map implementation. And it shouldn't be too complicated to implement your own adapter that implements java.util.Map to bridge them.

Answer 2

You could have a look at the Disk-Backed-map project.

A library that implements a disk backed map in Java

A small library that provide a disk backed map implementation for storing large number of key value pairs. The map implementations (HashMap, HashTable) max out around 3-4Million keys/GB of memory for very simple key/value pairs and in most cases the limit is much lower. DiskBacked map on the other hand can store betweeen 16Million (64bit JVM) to 20Million(32bit JVM) keys/GB, regardless the size of the key/value pairs.

Answer 3

Chronicle Map is a modern and the fastest solution to this problem. It implements ConcurrentMap interface and persists the data to disk (under the hood, it is done by mapping Chronicle Map's memory to a file).

Answer 4

MapDB (mapdb.org) does exactly what you are looking for. Besides disk backed TreeMap and HashMap it gives you other collection types.

It's maps are also thread-safe and have really good performance.

See Features

Answer 5

This seems like a relatively new open source solution to the problem, I've used it, and like it so far

https://github.com/jankotek/JDBM4

Answer 6

You could use a simple EHCache implementation? The nice thing about EHCache being that it can be very simple to implement :-)

I take it you've ruled out serialising / deserialising an actual Map instance?