OutOfMemoryException while trying to read big Excel file into DataTable

Question

I\'m using SSIS package to clean and load data from .Xlsx file to SQL Server table. I have also to highlight cells containing wrong data in .Xlsx file, for this I have to g

Answer 1

When trying to read data from an Excel with huge number of Rows, it is better to read data by chunk (in OleDbDataAdapter you can use paging option to achieve that).

int result = 1;
int intPagingIndex = 0;
int intPagingInterval = 1000;

while (result > 0){

    result = daGetDataFromSheet.Fill(dsErrorColumns,intPagingIndex, intPagingInterval , Error_Sheet);
    System.Data.DataTable dtErrorColumns = dsErrorColumns.Tables[Error_Sheet];

    //Implement your logic here

    intPagingIndex += intPagingInterval ;

}

This will prevent an OutOfMemory Exception. And no more need to specify a range like AC1:AC10000

References

• Paging Through a Query Result
• Fill(DataSet, Int32, Int32, String)