Error message when appending data to pandas dataframe

问题

Can someone give me a hand with this:

I created a loop to append successive intervals of historical price data from Coinbase.

My loop iterates successfully a few times then crashes.

Error message (under data_temp code line):

"ValueError: If using all scalar values, you must pass an index"

days = 10
end = datetime.now().replace(microsecond=0)
start = end - timedelta(days=days)
data_price = pd.DataFrame()

for i in range(1,50):
    print(start)
    print(end)
    data_temp = pd.DataFrame(public_client.get_product_historic_rates(product_id='BTC-USD', granularity=3600, start=start, end=end))
    data_price = data_price.append(data_temp)
    end = start
    start = end - timedelta(days=days)

Would love to understand how to fix this and why this is happening in the first place.

Thank you!

Here's the full trace:

Traceback (most recent call last): File "\coinbase_bot.py", line 46, in data_temp = pd.DataFrame(public_client.get_product_historic_rates(product_id='BTC-USD', granularity=3600, start=start, end=end)) File "D:\Program Files\Python37\lib\site-packages\pandas\core\frame.py", line 411, in init mgr = init_dict(data, index, columns, dtype=dtype) File "D:\Program Files\Python37\lib\site-packages\pandas\core\internals\construction.py", line 257, in init_dict return arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype) File "D:\Program Files\Python37\lib\site-packages\pandas\core\internals\construction.py", line 77, in arrays_to_mgr index = extract_index(arrays) File "D:\Program Files\Python37\lib\site-packages\pandas\core\internals\construction.py", line 358, in extract_index raise ValueError("If using all scalar values, you must pass an index") ValueError: If using all scalar values, you must pass an index

Here's json returned via simple url call:

[[1454716800,370.05,384.54,384.44,375.44,6276.66473729],[1454630400,382.99,389.36,387.99,384.5,7443.92933224],[1454544000,368.74,390.63,368.87,387.99,8887.7572324],[1454457600,365.63,373.01,372.93,368.87,7147.95657328],[1454371200,371.17,374.41,371.33,372.93,6856.21815799],[1454284800,366.26,379,367.89,371.33,7931.22922922],[1454198400,365,382.5,378.46,367.95,5506.77681302]]

Very similar to this user's issue but cannot put my finger on it: When attempting to merge multiple dataframes, how to resolve "ValueError: If using all scalar values, you must pass an index"

回答1:

-- Hi DashOfProgramming,

Your problem is that the data_temp is initialised with only a single row and pandas requires you to provide it with an index for that.

The following snippet should resolve this. I replaced your API call with a simple dictionary that resembles what I would expect the API to return and used i as index for the dataframe (this has the advantage that you can keep track as well):

import pandas as pd
from datetime import datetime, timedelta

days = 10
end = datetime.now().replace(microsecond=0)
start = end - timedelta(days=days)
data_price = pd.DataFrame()

temp_dict = {'start': '2019-09-30', 'end': '2019-10-01', 'price': '-111.0928', 
'currency': 'USD'}

for i in range(1,50):
  print(start)
  print(end)
  data_temp = pd.DataFrame(temp_dict, index=[i])
  data_price = data_price.append(data_temp)
  end = start
  start = end - timedelta(days=days)

print(data_price)

EDIT

Just saw that your API output is a nested list. pd.DataFrame() thinks the list is only one row, because it's nested. I suggest you store your columns in a separate variable and then do this:

cols = ['ts', 'low', 'high', 'open', 'close', 'sth_else']

v = [[...], [...], [...]] # your list of lists

data_temp = pd.DataFrame.from_records(v, columns=cols)

来源：https://stackoverflow.com/questions/58206816/error-message-when-appending-data-to-pandas-dataframe

标签

python-3.x

pandas

dataframe

append