reading Excel Open XML is ignoring blank cells

Question

I am using the accepted solution here to convert an excel sheet into a datatable. This works fine if I have \"perfect\" data but if I have a blank cell in the middle of my

Answer 1

I can't resist optimizing the subroutines from Amurra's answer to remove need for Regex's.

The first function isn't actually needed since the second one can will accept a cell reference (C3) or a column name (C) (but still a nice helper function). The indices are also one-based (only because our implementation used one-based for the rows to match visually with Excel).

    /// <summary>
    /// Given a cell name, return the cell column name.
    /// </summary>
    /// <param name="cellReference">Address of the cell (ie. B2)</param>
    /// <returns>Column Name (ie. B)</returns>
    /// <exception cref="ArgumentOutOfRangeException">cellReference</exception>
    public static string GetColumnName(string cellReference)
    {
        // Advance from L to R until a number, then return 0 through previous position
        //
        for (int lastCharPos = 0; lastCharPos <= 3; lastCharPos++)
            if (Char.IsNumber(cellReference[lastCharPos]))
                return cellReference.Substring(0, lastCharPos);

        throw new ArgumentOutOfRangeException("cellReference");
    }

    /// <summary>
    /// Return one-based column index given a cell name or column name
    /// </summary>
    /// <param name="columnNameOrCellReference">Column Name (ie. A, AB3, or AB44)</param>
    /// <returns>One based index if the conversion was successful; otherwise null</returns>
    public static int GetColumnIndexFromName(string columnNameOrCellReference)
    {
        int columnIndex = 0;            
        int factor = 1;
        for (int pos = columnNameOrCellReference.Length - 1; pos >= 0; pos--)   // R to L
        {
            if (Char.IsLetter(columnNameOrCellReference[pos]))  // for letters (columnName)
            {
                columnIndex += factor * ((columnNameOrCellReference[pos] - 'A') + 1);
                factor *= 26;
            }
        }
        return columnIndex;
    }

Answer 2

All good examples. Here is the one I am using since I need to keep track of all rows, cells, values, and titles for correlation and analysis.

The method ReadSpreadsheet opens an xlxs file and goes through each worksheet, row, and column. Since the values are stored in a referenced string table, I also explicitly use that per worksheet. There are other classes used: DSFunction and StaticVariables. The latter holds oft used parameter values, such as the referenced 'quotdouble' ( quotdouble = "\u0022"; ) and 'crlf' (crlf = "\u000D" + "\u000A"; ).

The relevant DSFunction method GetIntColIndexForLetter is included below. It returns an integer value for the column index corresponding to letter names such as (A,B, AA, ADE, etc.). This is used along with the parameter 'ncellcolref' to determine if any columns have been skipped and to enter empty string values for each one that is missing.

I also do some cleaning of the values before storing temporarily in a List object (using Replace method).

Subsequently, I use the hash table (Dictionary) of column names to extract values across different worksheets, correlate them, create normalized values, and then create an object used in our product which is then stored as an XML file. None of this is shown but is why this approach is used.

    public static class DSFunction {

    /// <summary>
    /// Creates an integer value for a column letter name starting at 1 for 'a'
    /// </summary>
    /// <param name="lettstr">Column name as letters</param>
    /// <returns>int value</returns>
    public static int GetIntColIndexForLetter(string lettstr) {
        string txt = "", txt1="";
        int n1, result = 0, nbeg=-1, nitem=0;
        try {
            nbeg = (int)("a".ToCharArray()[0]) - 1; //1 based
            txt = lettstr;
            if (txt != "") txt = txt.ToLower().Trim();
            while (txt != "") {
                if (txt.Length > 1) {
                    txt1 = txt.Substring(0, 1);
                    txt = txt.Substring(1);
                }
                else {
                    txt1 = txt;
                    txt = "";
                }
                if (!DSFunction.IsNumberString(txt1, "real")) {
                    nitem++;
                    n1 = (int)(txt1.ToCharArray()[0]) - nbeg;
                    result += n1 + (nitem - 1) * 26;
                }
                else {
                    break;
                }
            }
        }
        catch (Exception ex) {
            txt = ex.Message;
        }
        return result;
    }


}


    public static class Extractor {

    public static string ReadSpreadsheet(string fileUri) {
        string msg = "", txt = "", txt1 = "";
        int i, n1, n2, nrow = -1, ncell = -1, ncellcolref = -1;
        Boolean haveheader = true;
        Dictionary<string, int> hashcolnames = new Dictionary<string, int>();
        List<string> colvalues = new List<string>();
        try {
            if (!File.Exists(fileUri)) { throw new Exception("file does not exist"); }
            using (SpreadsheetDocument ssdoc = SpreadsheetDocument.Open(fileUri, true)) {
                var stringTable = ssdoc.WorkbookPart.GetPartsOfType<SharedStringTablePart>().FirstOrDefault();
                foreach (Sheet sht in ssdoc.WorkbookPart.Workbook.Descendants<Sheet>()) {
                    nrow = 0;
                    foreach (Row ssrow in ((WorksheetPart)(ssdoc.WorkbookPart.GetPartById(sht.Id))).Worksheet.Descendants<Row>()) {
                        ncell = 0;
                        ncellcolref = 0;
                        nrow++;
                        colvalues.Clear();
                        foreach (Cell sscell in ssrow.Elements<Cell>()) {
                            ncell++;
                            n1 = DSFunction.GetIntColIndexForLetter(sscell.CellReference);
                            for (i = 0; i < (n1 - ncellcolref - 1); i++) {
                                if (nrow == 1 && haveheader) {
                                    txt1 = "-missing" + (ncellcolref + 1 + i).ToString() + "-";
                                    if (!hashcolnames.TryGetValue(txt1, out n2)) {
                                        hashcolnames.Add(txt1, ncell - 1);
                                    }
                                }
                                else {
                                    colvalues.Add("");
                                }
                            }
                            ncellcolref = n1;
                            if (sscell.DataType != null) {
                                if (sscell.DataType.Value == CellValues.SharedString && stringTable != null) {
                                    txt = stringTable.SharedStringTable.ElementAt(int.Parse(sscell.InnerText)).InnerText;
                                }
                                else if (sscell.DataType.Value == CellValues.String) {
                                    txt = sscell.InnerText;
                                }
                                else txt = sscell.InnerText.ToString();
                            }
                            else txt = sscell.InnerText;
                            if (txt != "") txt1 = txt.ToLower().Trim(); else txt1 = "";
                            if (nrow == 1 && haveheader) {
                                txt1 = txt1.Replace(" ", "");
                                if (txt1 == "table/viewname") txt1 = "tablename";
                                else if (txt1 == "schemaownername") txt1 = "schemaowner";
                                else if (txt1 == "subjectareaname") txt1 = "subjectarea";
                                else if (txt1.StartsWith("column")) {
                                    txt1 = txt1.Substring("column".Length);
                                }
                                if (!hashcolnames.TryGetValue(txt1, out n1)) {
                                    hashcolnames.Add(txt1, ncell - 1);
                                }
                            }
                            else {
                                txt = txt.Replace(((char)8220).ToString(), "'");  //special "
                                txt = txt.Replace(((char)8221).ToString(), "'"); //special "
                                txt = txt.Replace(StaticVariables.quotdouble, "'");
                                txt = txt.Replace(StaticVariables.crlf, " ");
                                txt = txt.Replace("  ", " ");
                                txt = txt.Replace("<", "");
                                txt = txt.Replace(">", "");
                                colvalues.Add(txt);
                            }
                        }
                    }
                }
            }
        }
        catch (Exception ex) {
            msg = "notok:" + ex.Message;
        }
        return msg;
    }





}

Answer 3

Added yet another implementation, this time where the number of columns is known in advance:

        /// <summary>
        /// Gets a list cells that are padded with empty cells where necessary.
        /// </summary>
        /// <param name="numberOfColumns">The number of columns expected.</param>
        /// <param name="cells">The cells.</param>
        /// <returns>List of padded cells</returns>
        private static IList<Cell> GetPaddedCells(int numberOfColumns, IList<Cell> cells)
        {
            // Only perform the padding operation if existing column count is less than required
            if (cells.Count < numberOfColumns - 1)
            {
                IList<Cell> padded = new List<Cell>();
                int cellIndex = 0;

                for (int paddedIndex = 0; paddedIndex < numberOfColumns; paddedIndex++)
                {
                    if (cellIndex < cells.Count)
                    {
                        // Grab column reference (ignore row) <seealso cref="https://stackoverflow.com/a/7316298/674776"/>
                        string columnReference = new string(cells[cellIndex].CellReference.ToString().Where(char.IsLetter).ToArray());

                        // Convert reference to index <seealso cref="https://stackoverflow.com/a/848552/674776"/>
                        int indexOfReference = columnReference.ToUpper().Aggregate(0, (column, letter) => (26 * column) + letter - 'A' + 1) - 1;

                        // Add padding cells where current cell index is less than required
                        while (indexOfReference > paddedIndex)
                        {
                            padded.Add(new Cell());
                            paddedIndex++;
                        }

                        padded.Add(cells[cellIndex++]);
                    }
                    else
                    {
                        // Add padding cells when passed existing cells
                        padded.Add(new Cell());
                    }
                }

                return padded;
            }
            else
            {
                return cells;
            }
        }

Call using:

IList<Cell> cells = GetPaddedCells(38, row.Descendants<Cell>().ToList());

Where 38 is the required number of columns.

Answer 4

Here is my solution. I found the above didn't seem to work well when the missing fields where at the end of a row.

Assuming the first row in the Excel sheet has ALL the columns (via headers), then grab the number of columns expected per row (row == 1). Then loop through the data rows (row > 1). The key to processing the missing cells is in method getRowCells, where the known number of column cells is passed in as well as the current row to process.

int columnCount = worksheetPart.Worksheet.Descendants<Row>().Where(r => r.RowIndex == 1).FirstOrDefault().Descendants<Cell>().Count();

IEnumerable<Row> rows = worksheetPart.Worksheet.Descendants<Row>().Where(r => r.RowIndex > 1);

List<List<string>> docData = new List<List<string>>();

foreach (Row row in rows)
{
    List<Cell> cells = getRowCells(columnCount, row);

    List<string> rowData = new List<string>();

    foreach (Cell cell in cells)
    {
        rowData.Add(getCellValue(workbookPart, cell));
    }

    docData.Add(rowData);
}

Method getRowCells has a current limitation of only being able to support a sheet (row) that has less an 26 columns. A loop based on the known column count is used to find missing columns (cells). If found, a new Cell value is inserted into the cells collection, with the new Cell having a default value of "" instead of 'null'. The modified Cell collection is then returned.

private static List<Cell> getRowCells(int columnCount, Row row)
{
    const string COLUMN_LETTERS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";

    if (columnCount > COLUMN_LETTERS.Length)
    {
       throw new ArgumentException(string.Format("Invalid columnCount ({0}).  Cannot be greater than {1}",
                columnCount, COLUMN_LETTERS.Length));
    }

    List<Cell> cells = row.Descendants<Cell>().ToList();

    for (int i = 0; i < columnCount; i++)
    {
       if (i < cells.Count)
       {
           string cellColumnReference = cells.ElementAt(i).CellReference.ToString();
            if (cellColumnReference[0] != COLUMN_LETTERS[i])
            {
                cells.Insert(i, new Cell() { CellValue = new CellValue("") });             }
        }
        else
        {
            cells.Insert(i, new Cell() { CellValue = new CellValue("") });
        }
    }

    return cells;
}

private static string getCellValue(WorkbookPart workbookPart, Cell cell)
{
    SharedStringTablePart stringTablePart = workbookPart.SharedStringTablePart;
    string value = (cell.CellValue != null) ? cell.CellValue.InnerXml : string.Empty;

    if ((cell.DataType != null) && (cell.DataType.Value == CellValues.SharedString))
    {
        return stringTablePart.SharedStringTable.ChildElements[Int32.Parse(value)].InnerText;
    }
    else
    {
        return value;
    }
}      

Answer 5

This makes sense since Excel will not store a value for a cell that is null. If you open your file using the Open XML SDK 2.0 Productivity Tool and traverse the XML down to the cell level you will see that only the cells that have data are going to be in that file.

Your options are to insert blank data in the range of cells you are going to traverse or programmatically figure out a cell was skipped and adjust your index appropriately.

I made an example excel document with a string in cell reference A1 and C1. I then opened up the excel document in the Open XML Productivity Tool and here is the XML that was stored:

<x:row r="1" spans="1:3" 
   xmlns:x="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
  <x:c r="A1" t="s">
    <x:v>0</x:v>
  </x:c>
  <x:c r="C1" t="s">
    <x:v>1</x:v>
  </x:c>
</x:row>

Here you will see that the data corresponds to the first row and that only two cells worth of data are saved for that row. The data saved corresponds to A1 and C1 and that no cells with null values are saved.

To get the functionality that you need, you can traverse over the Cells as you are doing above, but you will need to check what the value the Cell is referencing and determine if any Cells have been skipped. to do that you will need two utility functions to get the Column Name from the cell reference and to then translate that column name into a zero based index:

    private static List<char> Letters = new List<char>() { 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', ' ' };

    /// <summary>
    /// Given a cell name, parses the specified cell to get the column name.
    /// </summary>
    /// <param name="cellReference">Address of the cell (ie. B2)</param>
    /// <returns>Column Name (ie. B)</returns>
    public static string GetColumnName(string cellReference)
    {
        // Create a regular expression to match the column name portion of the cell name.
        Regex regex = new Regex("[A-Za-z]+");
        Match match = regex.Match(cellReference);

        return match.Value;
    }

    /// <summary>
    /// Given just the column name (no row index), it will return the zero based column index.
    /// Note: This method will only handle columns with a length of up to two (ie. A to Z and AA to ZZ). 
    /// A length of three can be implemented when needed.
    /// </summary>
    /// <param name="columnName">Column Name (ie. A or AB)</param>
    /// <returns>Zero based index if the conversion was successful; otherwise null</returns>
    public static int? GetColumnIndexFromName(string columnName)
    {
        int? columnIndex = null;

        string[] colLetters = Regex.Split(columnName, "([A-Z]+)");
        colLetters = colLetters.Where(s => !string.IsNullOrEmpty(s)).ToArray();

        if (colLetters.Count() <= 2)
        {
            int index = 0;
            foreach (string col in colLetters)
            {
                List<char> col1 = colLetters.ElementAt(index).ToCharArray().ToList();
                int? indexValue = Letters.IndexOf(col1.ElementAt(index));

                if (indexValue != -1)
                {
                    // The first letter of a two digit column needs some extra calculations
                    if (index == 0 && colLetters.Count() == 2)
                    {
                        columnIndex = columnIndex == null ? (indexValue + 1) * 26 : columnIndex + ((indexValue + 1) * 26);
                    }
                    else
                    {
                        columnIndex = columnIndex == null ? indexValue : columnIndex + indexValue;
                    }
                }

                index++;
            }
        }

        return columnIndex;
    }

Then you can iterate over the Cells and check to see what the cell reference is compared to the columnIndex. If it is less than then you add blank data to your tempRow, otherwise just read in the value contained in the cell. (Note: I did not test the code below, but the general idea should help):

DataRow tempRow = dt.NewRow();

int columnIndex = 0;
foreach (Cell cell in row.Descendants<Cell>())
{
   // Gets the column index of the cell with data
   int cellColumnIndex = (int)GetColumnIndexFromName(GetColumnName(cell.CellReference));

   if (columnIndex < cellColumnIndex)
   {
      do
      {
         tempRow[columnIndex] = //Insert blank data here;
         columnIndex++;
      }
      while(columnIndex < cellColumnIndex);
    }
    tempRow[columnIndex] = GetCellValue(spreadSheetDocument, cell);

    if (tempRow[i].ToString().IndexOf("Latency issues in") > -1)
    {
       Console.Write(tempRow[i].ToString());
    }
    columnIndex++;
}

Answer 6

Here's an implementation of IEnumerable that should do what you want, compiled and unit tested.

    ///<summary>returns an empty cell when a blank cell is encountered
    ///</summary>
    public IEnumerator<Cell> GetEnumerator()
    {
        int currentCount = 0;

        // row is a class level variable representing the current
        // DocumentFormat.OpenXml.Spreadsheet.Row
        foreach (DocumentFormat.OpenXml.Spreadsheet.Cell cell in
            row.Descendants<DocumentFormat.OpenXml.Spreadsheet.Cell>())
        {
            string columnName = GetColumnName(cell.CellReference);

            int currentColumnIndex = ConvertColumnNameToNumber(columnName);

            for ( ; currentCount < currentColumnIndex; currentCount++)
            {
                yield return new DocumentFormat.OpenXml.Spreadsheet.Cell();
            }

            yield return cell;
            currentCount++;
        }
    }

Here are the functions it relies on:

    /// <summary>
    /// Given a cell name, parses the specified cell to get the column name.
    /// </summary>
    /// <param name="cellReference">Address of the cell (ie. B2)</param>
    /// <returns>Column Name (ie. B)</returns>
    public static string GetColumnName(string cellReference)
    {
        // Match the column name portion of the cell name.
        Regex regex = new Regex("[A-Za-z]+");
        Match match = regex.Match(cellReference);

        return match.Value;
    }

    /// <summary>
    /// Given just the column name (no row index),
    /// it will return the zero based column index.
    /// </summary>
    /// <param name="columnName">Column Name (ie. A or AB)</param>
    /// <returns>Zero based index if the conversion was successful</returns>
    /// <exception cref="ArgumentException">thrown if the given string
    /// contains characters other than uppercase letters</exception>
    public static int ConvertColumnNameToNumber(string columnName)
    {
        Regex alpha = new Regex("^[A-Z]+$");
        if (!alpha.IsMatch(columnName)) throw new ArgumentException();

        char[] colLetters = columnName.ToCharArray();
        Array.Reverse(colLetters);

        int convertedValue = 0;
        for (int i = 0; i < colLetters.Length; i++)
        {
            char letter = colLetters[i];
            int current = i == 0 ? letter - 65 : letter - 64; // ASCII 'A' = 65
            convertedValue += current * (int)Math.Pow(26, i);
        }

        return convertedValue;
    }

Throw it in a class and give it a try.