How Should I Implement a Huge but Simple Indexed StringList in Delphi?

Question

I am using Delphi 2009. I have a very simple data structure, with 2 fields:

1. A string that is the key field I need to retrieve by and is usually 4 to 15 charact

Answer 1

You should analyse your data. If

1. a sizeable part of the data values is larger than the default file system block size,
2. you don't want to search in the data values using SQL (so it doesn't matter what format they are stored in), and
3. you really need random access over the whole database,

then you should test whether compressing your data values increases performance. The decompression of data values (especially on a modern machine with multiple cores, performed in background threads) should incur only a small performance hit, but the gains from having to read fewer blocks from the hard disc (especially if they are not in the cache) could be much larger.

But you need to measure, maybe the database engine stores compressed data anyway.