Best way to store data locally in .NET (C#)

Question

I\'m writing an application that takes user data and stores it locally for use later. The application will be started and stopped fairly often, and I\'d like to make it save

Answer 1

Depending on the compelexity of your Account object, I would recomend either XML or Flat file.

If there are just a couple of values to store for each account, you could store them on a properties file, like this:

account.1.somekey=Some value
account.1.someotherkey=Some other value
account.1.somedate=2009-12-21
account.2.somekey=Some value 2
account.2.someotherkey=Some other value 2

... and so forth. Reading from a properties file should be easy, as it maps directly to a string dictionary.

As to where to store this file, the best choise would be to store into AppData folder, inside a subfolder for your program. This is a location where current users will always have access to write, and it's kept safe from other users by the OS itself.

Answer 2

It really depends on what you're storing. If you're talking about structured data, then either XML or a very lightweight SQL RDBMS like SQLite or SQL Server Compact Edition will work well for you. The SQL solution becomes especially compelling if the data moves beyond a trivial size.

If you're storing large pieces of relatively unstructured data (binary objects like images, for example) then obviously neither a database nor XML solution are appropriate, but given your question I'm guessing it's more of the former than the latter.

Answer 3

All of the above are good answers, and generally solve the problem.

If you need an easy, free way to scale to millions of pieces of data, try out the ESENT Managed Interface project on GitHub or from NuGet.

ESENT is an embeddable database storage engine (ISAM) which is part of Windows. It provides reliable, transacted, concurrent, high-performance data storage with row-level locking, write-ahead logging and snapshot isolation. This is a managed wrapper for the ESENT Win32 API.

It has a PersistentDictionary object that is quite easy to use. Think of it as a Dictionary() object, but it is automatically loaded from and saved to disk without extra code.

For example:

/// <summary>
/// Ask the user for their first name and see if we remember 
/// their last name.
/// </summary>
public static void Main()
{
    PersistentDictionary<string, string> dictionary = new PersistentDictionary<string, string>("Names");
    Console.WriteLine("What is your first name?");
    string firstName = Console.ReadLine();
    if (dictionary.ContainsKey(firstName))
    {
        Console.WriteLine("Welcome back {0} {1}", firstName, dictionary[firstName]);
    }
    else
    {
        Console.WriteLine("I don't know you, {0}. What is your last name?", firstName);
        dictionary[firstName] = Console.ReadLine();
    }

To answer George's question:

Supported Key Types

Only these types are supported as dictionary keys:

Boolean Byte Int16 UInt16 Int32 UInt32 Int64 UInt64 Float Double Guid DateTime TimeSpan String

Supported Value Types

Dictionary values can be any of the key types, Nullable versions of the key types, Uri, IPAddress or a serializable structure. A structure is only considered serializable if it meets all these criteria:

• The structure is marked as serializable • Every member of the struct is either: 1. A primitive data type (e.g. Int32) 2. A String, Uri or IPAddress 3. A serializable structure.

Or, to put it another way, a serializable structure cannot contain any references to a class object. This is done to preserve API consistency. Adding an object to a PersistentDictionary creates a copy of the object though serialization. Modifying the original object will not modify the copy, which would lead to confusing behavior. To avoid those problems the PersistentDictionary will only accept value types as values.

Can Be Serialized [Serializable] struct Good { public DateTime? Received; public string Name; public Decimal Price; public Uri Url; }

Can’t Be Serialized [Serializable] struct Bad { public byte[] Data; // arrays aren’t supported public Exception Error; // reference object }

Answer 4

If your collection gets too big, I have found that Xml serialization gets quite slow. Another option to serialize your dictionary would be "roll your own" using a BinaryReader and BinaryWriter.

Here's some sample code just to get you started. You can make these generic extension methods to handle any type of Dictionary, and it works quite well, but is too verbose to post here.

class Account
{
    public string AccountName { get; set; }
    public int AccountNumber { get; set; }

    internal void Serialize(BinaryWriter bw)
    {
        // Add logic to serialize everything you need here
        // Keep in synch with Deserialize
        bw.Write(AccountName);
        bw.Write(AccountNumber);
    }

    internal void Deserialize(BinaryReader br)
    {
        // Add logic to deserialize everythin you need here, 
        // Keep in synch with Serialize
        AccountName = br.ReadString();
        AccountNumber = br.ReadInt32();
    }
}


class Program
{
    static void Serialize(string OutputFile)
    {
        // Write to disk 
        using (Stream stream = File.Open(OutputFile, FileMode.Create))
        {
            BinaryWriter bw = new BinaryWriter(stream);
            // Save number of entries
            bw.Write(accounts.Count);

            foreach (KeyValuePair<string, List<Account>> accountKvp in accounts)
            {
                // Save each key/value pair
                bw.Write(accountKvp.Key);
                bw.Write(accountKvp.Value.Count);
                foreach (Account account in accountKvp.Value)
                {
                    account.Serialize(bw);
                }
            }
        }
    }

    static void Deserialize(string InputFile)
    {
        accounts.Clear();

        // Read from disk
        using (Stream stream = File.Open(InputFile, FileMode.Open))
        {
            BinaryReader br = new BinaryReader(stream);
            int entryCount = br.ReadInt32();
            for (int entries = 0; entries < entryCount; entries++)
            {
                // Read in the key-value pairs
                string key = br.ReadString();
                int accountCount = br.ReadInt32();
                List<Account> accountList = new List<Account>();
                for (int i = 0; i < accountCount; i++)
                {
                    Account account = new Account();
                    account.Deserialize(br);
                    accountList.Add(account);
                }
                accounts.Add(key, accountList);
            }
        }
    }

    static Dictionary<string, List<Account>> accounts = new Dictionary<string, List<Account>>();

    static void Main(string[] args)
    {
        string accountName = "Bob";
        List<Account> newAccounts = new List<Account>();
        newAccounts.Add(AddAccount("A", 1));
        newAccounts.Add(AddAccount("B", 2));
        newAccounts.Add(AddAccount("C", 3));
        accounts.Add(accountName, newAccounts);

        accountName = "Tom";
        newAccounts = new List<Account>();
        newAccounts.Add(AddAccount("A1", 11));
        newAccounts.Add(AddAccount("B1", 22));
        newAccounts.Add(AddAccount("C1", 33));
        accounts.Add(accountName, newAccounts);

        string saveFile = @"C:\accounts.bin";

        Serialize(saveFile);

        // clear it out to prove it works
        accounts.Clear();

        Deserialize(saveFile);
    }

    static Account AddAccount(string AccountName, int AccountNumber)
    {
        Account account = new Account();
        account.AccountName = AccountName;
        account.AccountNumber = AccountNumber;
        return account;
    }
}

Answer 5

A fourth option to those you mention are binary files. Although that sounds arcane and difficult, it's really easy with the serialization API in .NET.

Whether you choose binary or XML files, you can use the same serialization API, although you would use different serializers.

To binary serialize a class, it must be marked with the [Serializable] attribute or implement ISerializable.

You can do something similar with XML, although there the interface is called IXmlSerializable, and the attributes are [XmlRoot] and other attributes in the System.Xml.Serialization namespace.

If you want to use a relational database, SQL Server Compact Edition is free and very lightweight and based on a single file.

Answer 6

In my experience in most cases JSON in a file is enough (mostly you need to store an array or an object or just a single number or string). I rarely need SQLite (which needs more time for setting it up and using it, most of the times it's overkill).