How to transform a 3d arrays into a dataframe in python

Question

I have a 3d arrays as follows:

ThreeD_Arrays = np.random.randint(0, 1000, (5, 4, 3))

array([[[715, 226, 632],
        [305,  97, 534],
        [ 88, 592, 90         


        

Answer 1

Based on the answer to this question, we can use a MultiIndex. First, create the MultiIndex and a flattened DataFrame.

A = np.random.randint(0, 1000, (5, 4, 3))

names = ['x', 'y', 'z']
index = pd.MultiIndex.from_product([range(s)for s in A.shape], names=names)
df = pd.DataFrame({'A': A.flatten()}, index=index)['A']

Now we can reshape it however we like:

df = df.unstack(level='x').swaplevel().sort_index()
df.columns = ['A', 'B', 'C']
df.index.names = ['DATE', 'i']

This is the result:

          A    B    C
DATE i           
0    0  715  226  632
     1  895  837  431
     2  520  692  230
     3  286  358  462
     4   44  119  757
1    0  305   97  534
     1  649  717   39
     2  452  816  887
     3  831   26  332
     4  908  937  728
2    0   88  592  902
     1  363  121  274
     2  688  509  770
     3  424  178  642
     4  809   28  442
3    0  172  932  263
     1  334  359  816
     2  290  856  584
     3  955   42  938
     4  832  220  348

Answer 2

ThreeD_Arrays = np.random.randint(0, 1000, (5, 4, 3))
df = pd.DataFrame([list(l) for l in ThreeD_Arrays]).stack().apply(pd.Series).reset_index(1, drop=True)
df.index.name = 'Date'
df.columns = list('ABC')

Answer 3

You could convert your 3D array to a Pandas Panel, then flatten it to a 2D DataFrame (using .to_frame()):

import numpy as np
import pandas as pd
np.random.seed(2016)

arr = np.random.randint(0, 1000, (5, 4, 3))
pan = pd.Panel(arr)
df = pan.swapaxes(0, 2).to_frame()
df.index = df.index.droplevel('minor')
df.index.name = 'Date'
df.index = df.index+1
df.columns = list('ABC')

yields

        A    B    C
Date               
1     875  702  266
1     940  180  971
1     254  649  353
1     824  677  745
...
4     675  488  939
4     382  238  225
4     923  926  633
4     664  639  616
4     770  274  378

---

Alternatively, you could reshape the array to shape (20, 3), form the DataFrame as usual, and then fix the index:

import numpy as np
import pandas as pd
np.random.seed(2016)

arr = np.random.randint(0, 1000, (5, 4, 3))
df = pd.DataFrame(arr.reshape(-1, 3), columns=list('ABC'))
df.index = np.repeat(np.arange(arr.shape[0]), arr.shape[1]) + 1
df.index.name = 'Date'
print(df)

yields the same result.