I need a way to do key-value lookups across (potentially) hundreds of GB of data. Ideally something based on a distributed hashtable, that works nicely with Java. It should be
DNS has the capability to do this, I don't know how large each one of your records is (8GB of tons of small data?), but it may work.