Elegantly adding data to a pandas.Panel within a running simulation

问题

Disclaimer

This is a follow-up question from here, where I had a pandas.Panel with several items consisting of pandas.DataFrames. I wanted to plot a certain column in my DataFrame (minor_axis in the Panel) from each item in only one command, avoiding a code cluster like plt.plot(x, DataFrame1[y1]) plt.plot(x, DataFrame2[y1]) ... It was brought as an answer that I could switch my axes in the Panel so that instead of one item containing all the information of one dataset (of a simulation with a certain starting parameter), but rather just one information (e.g. yvalue y1) for all the different simulations an storing other parameters in other items (DataFrames).

---

My basic simulation code

Even though my code is to simulate the behaviour of a pendulum I'll break it down to a general simulation code with returned values y1-y3 instead of the real physical parameters. This simulation will be done for 2 different starting parameters k.

import pandas as pd

data = pd.Panel(major_axis=[], minor_axis=['x', 'sim1', 'sim2'])

# some kind of simulation resulting in 3 simulated values and with a
# starting parameter for different simulation "strengths"
# not sure whether to use a list or dict here
ks = {'sim1' = 0.5, 'sim2' = 1.0}
for k in ks:
    x, y1, y2, y3 = 0, 0, 0, 0
    while x<100:
        x += 1
        y1 += 1*ks[k]*x
        y2 += 2*ks[k]*x
        y3 += 3*ks[k]*x
        ...

# for example the y2 value for the different k values should be plottable like this 
data['y2'].plot()

---

Question

My question now is how to elegantly (as few lines of code as possible) add/append each value for each simulation to data, considering there could be 5 or more simulations with 10 or more values for each simulation step?

E.g. in my problem mentioned before I'd create a new DataFrame and append it to my existing dataset for the given simulation - something like data.append(pd.DataFrame([[x, y1, y2, y3]], columns=['x', 'y1', 'y2', 'y3'])). But from there I couldn't plot properly with a single command but rather had to add a new graph for each simulation manually.

I'd be very happy if someone could help me understand how to build a Panel like this "on the run" - from my previous question I already know how to plot one :)

---

UPDATE I was asked for some example data, but since I want to consecutively add my simulated values into an Panel/item instead of generating a list first, I can only show how the data should look like in the end. In the beginning the Panel should look like this:

In [1]: print(data)
Out[1]:
<class 'pandas.core.panel.Panel'>
Dimensions: 2 (items) x 0 (major_axis) x 3 (minor_axis)
Items axis: y1 to y2
Major_axis axis: None
Minor_axis axis: x to sim2

In the following is shown how the simulations works and how for example the y1-item should look like in the end

In [2]: ks = {'sim1' : 0.5, 'sim2' : 1.0}
Out[2]: {'sim1': 0.5, 'sim2': 1.0}

In [3]:
for k in ks:
    x, y1, y2 = 0, 0, 0

    while x<3:
        x += 1
        y1 += 1*ks[k]*x
        y2 += 2*ks[k]*x
        # HERE is missing what I'm looking for
        # it should append e.g. the y1 value to data['y1'] for both k
Out[3]: ...

In [4]: print(data['y1'])
Out[4]:         
     x    sim1    sim2
0    1    0.5     1.0
1    2    1.5     3.0
2    3    3.0     6.0

I hope through this it's clearer now what I'm looking for - if not let me know

回答1:

I think the easies way to build a Pandas.Panel would be to build a dictionary of the following form:

d = {
    'items_axis_element0': DataFrame0,
    'items_axis_element1': DataFrame1,
    'items_axis_element2': DataFrame2,
    ...
}

now you can easily build up a Panel:

p = pd.Panel(d)

You may find some usefull examples in Pandas Cookbook

---

UPDATE: here is slightly modified example from Pandas Cookbook:

rng = pd.date_range('1/1/2013',periods=100,freq='D')
data = np.random.randn(100, 4)
cols = ['A','B','C','D']
df1, df2, df3 = pd.DataFrame(data, rng, cols), pd.DataFrame(data, rng, cols), pd.DataFrame(data, rng, cols)

pf = pd.Panel({'df1':df1,'df2':df2})

In [21]: pf
Out[21]:
<class 'pandas.core.panel.Panel'>
Dimensions: 2 (items) x 100 (major_axis) x 4 (minor_axis)
Items axis: df1 to df2
Major_axis axis: 2013-01-01 00:00:00 to 2013-04-10 00:00:00
Minor_axis axis: A to D

now we can add df3 as follows:

In [22]: pf.join(pd.Panel({'df3':df3}))
Out[22]:
<class 'pandas.core.panel.Panel'>
Dimensions: 3 (items) x 100 (major_axis) x 4 (minor_axis)
Items axis: df1 to df3
Major_axis axis: 2013-01-01 00:00:00 to 2013-04-10 00:00:00
Minor_axis axis: A to D

来源：https://stackoverflow.com/questions/43652896/elegantly-adding-data-to-a-pandas-panel-within-a-running-simulation