Importing a CSV file using BULK INSERT command into SQL Server table

Question

I have CSV file which have a couple of data columns.

The CSV file looks like

field1: Test1
field2: Test2
field3: Test3, Test4, Test5

Answer 1

I don't think you're going to be able to import that format without some type of pre-processing. As Aaron implied, that's not a standard CSV.

If you can't get the file re-formatted, there are several ways to get the data into an importable format:

http://www.timmitchell.net/post/2013/01/14/ragged-flat-file-processing-in-ssis/

http://www.rad.pasfu.com/index.php?/archives/38-Script-Component-as-Source-SSIS.html

http://msdn.microsoft.com/en-us/library/ms136060.aspx (scroll down to the "Flat File Source Example")

These are all SSIS solutions which leverage .NET for the bulk of the work. I prefer SSIS simply because of the built in productivity tools. It could be done with any text processor like a console app or powershell script (if you really have some time on your hands).

I prefer the script component source with a stream reader, but Tim Mitchell has an interesting alternative.

Answer 2

using System;
using System.Data;
using System.Data.SqlClient;

namespace SqlBulkInsertExample
{
    class Program
    {
      static void Main(string[] args)
        {
            DataTable prodSalesData = new DataTable("ProductSalesData");

            // Create Column 1: SaleDate
            DataColumn dateColumn = new DataColumn();
            dateColumn.DataType = Type.GetType("System.DateTime");
            dateColumn.ColumnName = "SaleDate";

            // Create Column 2: ProductName
            DataColumn productNameColumn = new DataColumn();
            productNameColumn.ColumnName = "ProductName";

            // Create Column 3: TotalSales
            DataColumn totalSalesColumn = new DataColumn();
            totalSalesColumn.DataType = Type.GetType("System.Int32");
            totalSalesColumn.ColumnName = "TotalSales";

            // Add the columns to the ProductSalesData DataTable
            prodSalesData.Columns.Add(dateColumn);
            prodSalesData.Columns.Add(productNameColumn);
            prodSalesData.Columns.Add(totalSalesColumn);

            // Let's populate the datatable with our stats.
            // You can add as many rows as you want here!

            // Create a new row
            DataRow dailyProductSalesRow = prodSalesData.NewRow();
            dailyProductSalesRow["SaleDate"] = DateTime.Now.Date;
            dailyProductSalesRow["ProductName"] = "Nike";
            dailyProductSalesRow["TotalSales"] = 10;

            // Add the row to the ProductSalesData DataTable
            prodSalesData.Rows.Add(dailyProductSalesRow);

            // Copy the DataTable to SQL Server using SqlBulkCopy
            using (SqlConnection dbConnection = new SqlConnection("Data Source=ProductHost;Initial Catalog=dbProduct;Integrated Security=SSPI;Connection Timeout=60;Min Pool Size=2;Max Pool Size=20;"))
            {
                dbConnection.Open();
                using (SqlBulkCopy s = new SqlBulkCopy(dbConnection))
                {
                    s.DestinationTableName = prodSalesData.TableName;

                    foreach (var column in prodSalesData.Columns)
                        s.ColumnMappings.Add(column.ToString(), column.ToString());

                    s.WriteToServer(prodSalesData);
                }
            }
        }
    }
}

Answer 3

Try using:
ROWTERMINATOR = '0x0a'

Answer 4

This is the code for importing text or csv file into the database:

String SQL = "BULK INSERT [staging_db].[dbo].[TEST] FROM 'D:\\test.txt' WITH (FIELDTERMINATOR = ',',ROWTERMINATOR = '\n')";