Bulk C# datatable to postgresql table

Question

I have got a datatable with thousands of records. I have got a postgres table with the same fields of the datatable. I want everyday to truncate this table and fill again wi

Answer 1

Above mentioned solutions require you to specify number of columns and their types, thus making your code table specific. If your tables are relatively small and have the same amount of columns and same/compatible column types, you can do it in a generic way. Assume you want to migrate a Sqlite table to PosgreSql:

// Get data from SqlLite database table
SQLiteConnection sqliteConnection = new SQLiteConnection(new SQLiteConnectionStringBuilder() { DataSource = @"C:\dataBase.sqlite" }.ConnectionString);
sqliteConnection.Open();
var reader = new SQLiteCommand($"SELECT * from table_which_we_want_to_migrate").ExecuteReader();
var dataFromSqliteTable = new DataTable() { CaseSensitive = true };
dataFromSqliteTable.Load(reader);

// Connect to PostgreSql database
var connection = new NpgsqlConnection(new NpgsqlConnectionStringBuilder()
{
    Host = "localhost",
    Port = 5432,
    Database = "DatabaseName",
    Username = "UserName",
    Password = "Password"
}.ToString());
connection.Open();

// Insert every row from the Sqlite table into PostgreSql table
foreach (DataRow row in dataFromSqliteTable.Rows)
{
    // Create an NpgsqlParameter for every field in the column
    var parameters = new List();
    for (var i = 0; i < dataFromSqliteTable.Columns.Count; i++)
    {
        parameters.Add(new NpgsqlParameter($"@p{i}", row[i]));
    }
    var parameterNames = string.Join(", ", parameters.Select(p => p.ParameterName));
    
    // Create an INSERT SQL query which inserts the data from the current row into PostgreSql table
    var command = new NpgsqlCommand(
        $"INSERT INTO table_which_we_want_to_migrate VALUES ({parameterNames})",
        connection);
    command.Parameters.AddRange(parameters.ToArray());
    command.ExecuteNonQuery();
}

Alternative way is by using command line utilities and import/export via CSV file. This way is much faster and works even with big tables:

sqlite3 dataBase.sqlite ".output 'temporaryFile.csv.tmp'" ".headers off" ".mode csv" "SELECT * FROM table_which_we_want_to_migrate;" ".quit"
psql --command="\copy table_which_we_want_to_migrate FROM 'temporaryFile.csv.tmp' DELIMITER ',' CSV"