I have a function which create several dicts of dicts, based on some conditions.
However, I would really like to turn the dict into a dataframe after collecting it.
x = {'TSLA': {2011: {'negative': {'lowPrice': 185.16,
'lowDate': '05/27/19',
'highPrice': 365.71,
'highDate': '12/10/18',
'change': -0.49}},
2012: {'negative': {'lowPrice': 185.16,
'lowDate': '05/27/19',
'highPrice': 365.71,
'highDate': '12/10/18',
'change': -0.49}},
2013: {'negative': {'lowPrice': 32.91,
'lowDate': '01/07/13',
'highPrice': 37.24,
'highDate': '03/26/12',
'change': -0.12},
'positive': {'lowPrice': 32.91,
'lowDate': '01/07/13',
'highPrice': 190.9,
'highDate': '09/23/13',
'change': 4.8}}}}
y = []
z = []
for k0 in x:
for k1 in x[k0]:
for k2 in x[k0][k1]:
y .append((k0, k1, k2))
col = x[k0][k1][k2].keys()
for c in col:
z.append(x[k0][k1][k2][c])
index = pd.MultiIndex.from_tuples(y)
df = pd.DataFrame(columns=col, index=index)
z = np.array(z).reshape(df.shape)
df = pd.DataFrame(columns=col, index=index, data=z)
print(df)
lowPrice lowDate highPrice highDate change
TSLA 2011 negative 185.16 05/27/19 365.71 12/10/18 -0.49
2012 negative 185.16 05/27/19 365.71 12/10/18 -0.49
2013 negative 32.91 01/07/13 37.24 03/26/12 -0.12
positive 32.91 01/07/13 190.9 09/23/13 4.8
refer: Construct pandas DataFrame from items in nested dictionary
df = pd.DataFrame.from_dict({(i,j): dict_[i][j][z]
for i in dict_.keys()
for j in dict_[i].keys()
for z in dict_[i][j].keys()},
orient='index')
df
lowPrice lowDate highPrice highDate change
TSLA 2011 185.16 05/27/19 365.71 12/10/18 -0.49
2012 185.16 05/27/19 365.71 12/10/18 -0.49
2013 32.91 01/07/13 190.90 09/23/13 4.80
You can flatten nested dictionaries 2 times for tuples for keys and pass to DataFrame.from_dict:
d1 = {(k1, k2, k3): v3
for k1, v1 in d.items()
for k2, v2 in v1.items()
for k3, v3 in v2.items()}
df = pd.DataFrame.from_dict(d1, orient='index')
#alternative
#df = pd.DataFrame(d1).T
print (df)
lowPrice lowDate highPrice highDate change
TSLA 2011 negative 185.16 05/27/19 365.71 12/10/18 -0.49
2012 negative 185.16 05/27/19 365.71 12/10/18 -0.49
2013 negative 32.91 01/07/13 37.24 03/26/12 -0.12
positive 32.91 01/07/13 190.9 09/23/13 4.8
Similar but you can also use from_dict
:
df=pd.DataFrame.from_dict({(i, j, x) : y
for i in d.keys()
for j in d[i].keys()
for x, y in d[i][j].items()},
orient='index')
print (df)
lowPrice lowDate highPrice highDate change
TSLA 2011 negative 185.16 05/27/19 365.71 12/10/18 -0.49
2012 negative 185.16 05/27/19 365.71 12/10/18 -0.49
2013 negative 32.91 01/07/13 37.24 03/26/12 -0.12
positive 32.91 01/07/13 190.90 09/23/13 4.80