How to store dynamic data (unknown number of fields) to a file?

Question

I need to store some data in a file. FOR THE MOMENT each record (data set) consists in:

• a string (variable length),
• an array of integers (variable len

Answer 1

In order of your level of effort to implement, I suggest, in this order:

1. CSV or INI files (TMemIniFile, or TJvCsvDataSet). This is the least work for you. You can support millions of lines in a single file, but the memory consumed will be enormous. I have envisioned a "data writer" component to replace my TJvCsvDataSet with something that only appends records, and does not load them into memory. This would allow you to write out to CSV files, and even read them back, row by row, but not load them all at once. This approach might be idea for you. A simple CSV reader/writer class that is NOT a dataset object.

2. one-XML-tag-per-line files. This is more flexible than INI files and can be hierarchical. INI files are non-hierarchical. Neither SAX or DOM are required if you simply open a file stream, and append a line of text, that is in this form, ending in cr+lf:

   < logitem attrib1="value1" attrib2="value2" />

3. List item

4. Some kind of binary nosql db like bsddb, couchdb, etc.