IndexError: too many indices for array

Question

I know there is a ton of these threads but all of them are for very simple cases like 3x3 matrices and things of that sort and the solutions do not even begin to apply to my

Answer 1

I think the problem is given in the error message, although it is not very easy to spot:

IndexError: too many indices for array
xs  = data[:, col["l1"     ]]

'Too many indices' means you've given too many index values. You've given 2 values as you're expecting data to be a 2D array. Numpy is complaining because data is not 2D (it's either 1D or None).

This is a bit of a guess - I wonder if one of the filenames you pass to loadfile() points to an empty file, or a badly formatted one? If so, you might get an array returned that is either 1D, or even empty (np.array(None) does not throw an Error, so you would never know...). If you want to guard against this failure, you can insert some error checking into your loadfile function.

I highly recommend in your for loop inserting:

print(data)

This will work in Python 2.x or 3.x and might reveal the source of the issue. You might well find it is only one value of your outputs_l1 list (i.e. one file) that is giving the issue.

Answer 2

The message that you are getting is not for the default Exception of Python:

For a fresh python list, IndexError is thrown only on index not being in range (even docs say so).

>>> l = []
>>> l[1]
IndexError: list index out of range

If we try passing multiple items to list, or some other value, we get the TypeError:

>>> l[1, 2]
TypeError: list indices must be integers, not tuple

>>> l[float('NaN')]
TypeError: list indices must be integers, not float

However, here, you seem to be using matplotlib that internally uses numpy for handling arrays. On digging deeper through the codebase for numpy, we see:

static NPY_INLINE npy_intp
unpack_tuple(PyTupleObject *index, PyObject **result, npy_intp result_n)
{
    npy_intp n, i;
    n = PyTuple_GET_SIZE(index);
    if (n > result_n) {
        PyErr_SetString(PyExc_IndexError,
                        "too many indices for array");
        return -1;
    }
    for (i = 0; i < n; i++) {
        result[i] = PyTuple_GET_ITEM(index, i);
        Py_INCREF(result[i]);
    }
    return n;
}

where, the unpack method will throw an error if it the size of the index is greater than that of the results.

So, Unlike Python which raises a TypeError on incorrect Indexes, Numpy raises the IndexError because it supports multidimensional arrays.