How can I import a month of csv files (date named) into Excel via VBA?

Question

I need to load a month of CSV files into Excel for analysis via VBA. Each day of the month is a separate file with the date name (YYYYMMDD).

Currently, I can load two files created by two different circumstances, A and B using

With ActiveSheet.QueryTables.Add(Connection:=Full_F_Name_A, _
                                 Destination:=Range("$H$4"))

I use a loop to change A and B (and the destination). I have not figured out how to increment the date. I use an input box to get the date of the first file in the month.

F_Name = InputBox("Enter name of first data file eg YYYYMMDD, target=H4, EG4")

Any help would be great as I am stuck...and a beginner.

OK OK, see VBA code below. Received Run-time error '3001' Arguments are of the wrong type, are out of acceptable range or are in conflict with one another. Debugger points to the ".cursorlocation = aduseclient" line. Perhaps there is some software missing on my PC. The intro video on the ADO website no longer exists so I did not see the intro. I will try the other way I know of just opening the files and dumping them into excel while I await further advice.

Sub Month_wdata_import()

Set cN = CreateObject("ADODB.Connection")
Set rS = CreateObject("ADODB.Recordset")

Dim sDate As String
Dim sDataPath As String
Dim i As Integer
Dim  mMax As Integer

sDataPath = Worksheets("D&L").Cells(1, "G").Value ' values located in 2nd sheet of workbook
mMax = Worksheets("D&L").Cells(1, "D").Value  'values located in 2nd sheet of workbook

For i = 1 To mMax
    sDate = "A_" + CStr(Worksheets("D&L").Cells(1 + i, "A").Value) ' looping through list of dates in sheet

    With cN
     .cursorlocation = aduseclient
     .CursorType = adopenstatic
     .LockType = adLockreadonly

       .Open ("Provider=Microsoft.Jet.OLEDB.4.0;" & _
       "Data Source=" & sDataPath & ";" & _
       "Extended Properties=""text; HDR=Yes; FMT=Delimited; IMEX=1;""")
    End With

    With rS
      .ActiveConnection = cN
      .Source = "select * from data_" & sDate & "_.csv"
      .Open
    End With

Next

Range("A1").CopyFromRecordset rS

End Sub

Answer 1

have you condidered using ADODB and the ODBC Text File Driver / Jet 4.0 to retrieve your data into recordsets, then dump them into worksheets:

dim cN as new adodb.connection
dim rS as new adodb.recordset
dim sDate as string
dim sDataPath as string

sDataPath="C:\Data"
sdate=date ' maybe loop through date arrary, or list of dates in sheet?

with cN
    .CursorLocation = 3 ' adUseClient 
    .CursorType = 3 ' adopenstatic
    .LockType = 1 ' adLockReadOnly         
    .Open ("Provider=Microsoft.Jet.OLEDB.4.0;" & _
           "Data Source=" & sDataPath & ";" & _
           "Extended Properties=""text; HDR=Yes; FMT=Delimited; IMEX=1;""")
end with 

with RS
    .ActiveConnection = cN
    .Source = "select * from data_" & sdate & "_.csv"
    .open
end with

range("A1").copyfromrecordset rs

so put your csv files in the path defined the variable sDataPath, set the date variable sDate (maybe within a loop) and start testing!

for more info on this type pf technique, here is the original MSDN article by Scripting Clinic (in the good old days):

MSDN: Much ADO about Text Files

Plus you'll fine a veritable plethora of information on the net using Google:)

Answer 2

OK yes I am providing an answer to my own question....will work on getting the program for the suggested method.

Here is the code that is working (important bits only):

Dim all kinds of stuff

' Get the year and month of interest [User inputs the year and month they want the data analyzed YYYYMM

YY_MM = InputBox("Enter year and month of the daily cycles you want to analyze eg YYYYMM, no day, A_, B_ or _TC needed", target = D4, AI4)

'separate the YYYYMM date to YYYY and MM
F_year = Left(YY_MM, Len(YY_MM) - 2)
F_month = Right(YY_MM, Len(YY_MM) - 4)

'Determine # of days in the month 'The code below is from http://msdn.microsoft.com/en-us/library/aa227538(v=vs.60).aspx and it saved me from an If than nest from hell

mMax = DateSerial(CInt(F_year), (CInt(F_month) + 1), 1) - DateSerial(CInt(F_year), CInt(F_month), 1)

'The user must say where their data is by listing the path on the worksheet.  in this case it is a set template

sDataPath = Worksheets("Data").Cells(1, 4).Value ' value located in 1st sheet of workbook


    For i = 1 To mMax 'OK this is the file insertion For the A
set of data files

        If i < 10 Then   '  Need to add a zero to get the correct file name
            Z_singdig = "0" + CStr(i)
        Else
            Z_singdig = CStr(i)
        End If

        sDate = "A_" + YY_MM + Z_singdig + ".csv"   ' looping through list of dates ..

         'Label the data column with several file names so one can read it
        Label_F_Name = sDate + "........................."
        F_name = sDataPath + sDate

        With ActiveSheet.QueryTables.Add(Connection:="TEXT;" + F_name, Destination:=Range("D1048576").End(xlUp).Offset(1, 0)) 'insert the next file at the bottom of the first
            .Name = Label_F_Name
            .FieldNames = True
            .RowNumbers = False
            .FillAdjacentFormulas = False
            .PreserveFormatting = True
            .RefreshOnFileOpen = False
            .RefreshStyle = xlInsertEntireRows
            .SavePassword = False
            .SaveData = True
            .AdjustColumnWidth = False
            .RefreshPeriod = 0
            .TextFilePromptOnRefresh = False
            .TextFilePlatform = 437
            .TextFileStartRow = 1
            .TextFileParseType = xlDelimited
            .TextFileCommaDelimiter = True
            .TextFileColumnDataTypes = Array(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
            .TextFileTrailingMinusNumbers = True
            .Refresh BackgroundQuery:=False
        End With

'report out the number of rows in each data file so I can use those as cell ref's later
        numofrows = Application.CountA(Range("D:D"))
        rownumout = numofrows - Corrtrow ' num of rows just added by the last file
        rowprtinc = 30 + i
        Numrowprt = "'data'!" + "DE" + CStr(rowprtinc)
        Range(Numrowprt) = rownumout ' entring the number of rows into a table on sheet 2
        Corrtrow = numofrows ' must track the current row num so the next row num can be corrected ...yeah that worked

    Next i

' do the same for the B set of data files..not shown here

End Sub