Turning my dictionary into a pandas dataframe

Question

I have a function which create several dicts of dicts, based on some conditions.

However, I would really like to turn the dict into a dataframe after collecting it.

Answer 1

x = {'TSLA': {2011: {'negative': {'lowPrice': 185.16,
    'lowDate': '05/27/19',
    'highPrice': 365.71,
    'highDate': '12/10/18',
    'change': -0.49}},
  2012: {'negative': {'lowPrice': 185.16,
    'lowDate': '05/27/19',
    'highPrice': 365.71,
    'highDate': '12/10/18',
    'change': -0.49}},
  2013: {'negative': {'lowPrice': 32.91,
    'lowDate': '01/07/13',
    'highPrice': 37.24,
    'highDate': '03/26/12',
    'change': -0.12},
   'positive': {'lowPrice': 32.91,
    'lowDate': '01/07/13',
    'highPrice': 190.9,
    'highDate': '09/23/13',
    'change': 4.8}}}}

y = []
z = []
for k0 in x:
    for k1 in x[k0]:
        for k2 in x[k0][k1]:
            y .append((k0, k1, k2))     
            col = x[k0][k1][k2].keys()
            for c in col:
                z.append(x[k0][k1][k2][c])


index = pd.MultiIndex.from_tuples(y)
df = pd.DataFrame(columns=col, index=index)
z  = np.array(z).reshape(df.shape)
df = pd.DataFrame(columns=col, index=index, data=z)

print(df)

                   lowPrice   lowDate highPrice  highDate change
TSLA 2011 negative   185.16  05/27/19    365.71  12/10/18  -0.49
     2012 negative   185.16  05/27/19    365.71  12/10/18  -0.49
     2013 negative    32.91  01/07/13     37.24  03/26/12  -0.12
          positive    32.91  01/07/13     190.9  09/23/13    4.8

Answer 2

refer: Construct pandas DataFrame from items in nested dictionary

df = pd.DataFrame.from_dict({(i,j): dict_[i][j][z] 
                               for i in dict_.keys() 
                               for j in dict_[i].keys()
                               for z in dict_[i][j].keys()},
                           orient='index')
df


           lowPrice   lowDate  highPrice  highDate  change
TSLA 2011    185.16  05/27/19     365.71  12/10/18   -0.49
     2012    185.16  05/27/19     365.71  12/10/18   -0.49
     2013     32.91  01/07/13     190.90  09/23/13    4.80

Answer 3

You can flatten nested dictionaries 2 times for tuples for keys and pass to DataFrame.from_dict:

d1 = {(k1, k2, k3): v3 
      for k1, v1 in d.items() 
      for k2, v2 in v1.items()
      for k3, v3 in v2.items()}

df = pd.DataFrame.from_dict(d1, orient='index')
#alternative
#df = pd.DataFrame(d1).T

---

print (df)
                   lowPrice   lowDate highPrice  highDate change
TSLA 2011 negative   185.16  05/27/19    365.71  12/10/18  -0.49
     2012 negative   185.16  05/27/19    365.71  12/10/18  -0.49
     2013 negative    32.91  01/07/13     37.24  03/26/12  -0.12
          positive    32.91  01/07/13     190.9  09/23/13    4.8

Answer 4

Similar but you can also use from_dict:

df=pd.DataFrame.from_dict({(i, j, x) : y
                           for i in d.keys()
                           for j in d[i].keys()
                           for x, y in d[i][j].items()},
                           orient='index')

print (df)

                    lowPrice   lowDate  highPrice  highDate  change
TSLA 2011 negative    185.16  05/27/19     365.71  12/10/18   -0.49
     2012 negative    185.16  05/27/19     365.71  12/10/18   -0.49
     2013 negative     32.91  01/07/13      37.24  03/26/12   -0.12
          positive     32.91  01/07/13     190.90  09/23/13    4.80