simplest way to do a recurring text file upload to sql server

Question

i have a TAB DELIMITTED log file where new data in this log file needs to be dumped into a sql server 2008 database. i am wondering what is the best way to automate this? is the

Answer 1

If you have SQL Server Integration Services (SSIS) installed, you could use that to load the Tab delimited file into SQL Server table. In this link (SQL Integration Services to load tab delimited file), I have explained it step-by-step on how to load a tab delimited file into SQL Server table using SSIS.

If you would also like to loop through multiple files of same format in a given folder and load the data into a table, here is another Stack Overflow link (Loading multiple files) that might help you do that.

Hope that helps.

Answer 2

I would use SSIS (if available) and automate through that: http://msdn.microsoft.com/en-us/library/dd440761%28v=sql.100%29.aspx

Failing that, you could do a joined process of [scripting language of choice] + SSIS to have something watching the correct folder for the specific file and then have that kick off the SSIS package.

If SSIS is not available at all, then you're stuck with parsing out the file in code, and turning that into insert/update statements or calls to a stored proc that will do the same.

Answer 3

The best way and the simplest way might not be the same answer. Probably the best way, since this is recurring, would be to build a SSIS job that converts the file and puts it into the database. This will allow you to put any business logic into the import (cleaning up data, insert/update if needed, kicking out errors, automation, etc.)

Here is a simple tutorial that walks you through how to utilize SSIS to do this task:

http://decipherinfosys.wordpress.com/2008/07/28/ssis-importing-data-from-a-text-file-using-a-package/

If you want simple, create a little script to import the data in and then run it every time you have a file. Here is a blog entry that talks about doing an import using just SQL commands. It is for comma delimited, but you can modify it for tab delimited:

http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/