Question

我想将所有库存从sp500转到csv格式的文件夹中。现在，在扫描sp500时，一切工作都很好，但似乎在某些情况下，缺少Indax的日期是因为库存不存在或在特定时间内没有日期，无论我尝试更改开始日期和结束日期但没有效果-在在较早的帖子中，有人说我过滤了这些日期，但是由于python对我来说是新大陆，所以我就像一个外星人...有人可以帮助我吗？

如果给出此错误：

Traceback (most recent call last):
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Date'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mu351i/PycharmProjects/untitled/get_sp500_beautifulsoup_intro.py", line 44, in get_data_from_yahoo
    df = web.DataReader (ticker, 'yahoo', start, end)
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas/util/_decorators.py", line 208, in wrapper
    return func(*args, **kwargs)
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas_datareader/data.py", line 387, in DataReader
    session=session,
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas_datareader/base.py", line 251, in read
    df = self._read_one_data(self.url, params=self._get_params(self.symbols))
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas_datareader/yahoo/daily.py", line 165, in _read_one_data
    prices["Date"] = to_datetime(to_datetime(prices["Date"], unit="s").dt.date)
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas/core/frame.py", line 2995, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/mu351i/PycharmProjects/untitled/venv/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2899, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Date'

During handling of the above exception, another exception occurred:


Traceback (most recent call last):
  File "/home/mu351i/PycharmProjects/untitled/get_sp500_beautifulsoup_intro.py", line 57, in <module>
    get_data_from_yahoo()
  File "/home/mu351i/PycharmProjects/untitled/get_sp500_beautifulsoup_intro.py", line 48, in get_data_from_yahoo
    except RemoteDataError:
NameError: name 'RemoteDataError' is not defined

Process finished with exit code 1

您如何通过更改此代码来避免这种情况？

import datetime as dt
import os
import pickle

import bs4 as bs
import pandas_datareader.data as web
import requests


def safe_sp500_tickers():
    resp = requests.get('https://en.wikipedia.org/wiki/List_of_S%26P_500_companies')
    soup = bs.BeautifulSoup(resp.text,'lxml')
    table = soup.find('table',{'class':'wikitable sortable'})

    tickers = []
    for row in table.findAll('tr')[1:]:
        ticker=row.findAll('td')[0].text.strip()
        tickers.append(ticker)

    with open('sp500tickers.pickle','wb') as f:
        pickle.dump(tickers,f)

    return tickers

safe_sp500_tickers()

def get_data_from_yahoo(reload_sp500=False):
    if reload_sp500:
        tickers=safe_sp500_tickers()
    else:
        with open('sp500tickers.pickle', 'rb') as f:
            tickers = pickle.load(f)

    if not os.path.exists('stock_dfs'):
        os.makedirs('stock_dfs')

    start = dt.datetime(1999,1,1)
    end = dt.datetime(2019,12,19)

    for ticker in tickers:

        try:
            if not os.path.exists ('stock_dfs/{}.csv'.format (ticker)):
                df = web.DataReader (ticker, 'yahoo', start, end)
                df.to_csv ('stock_dfs/{}.csv'.format (ticker))
            else:
                print ("Ticker from {} already availablle".format (ticker))
        except RemoteDataError:
            print ("No information for ticker '%s'" % i)
            continue
        except KeyError:
            print("no Date for Ticker: " +ticker )
            continue



get_data_from_yahoo()

评论员要求提供一些数据样本，这是TSLA.csv形式的数据

Date,High,Low,Open,Close,Volume,Adj Close
2010-06-29,25.0,17.540000915527344,19.0,23.889999389648438,18766300,23.889999389648438
2010-06-30,30.420000076293945,23.299999237060547,25.790000915527344,23.829999923706055,17187100,23.829999923706055
2010-07-01,25.920000076293945,20.270000457763672,25.0,21.959999084472656,8218800,21.959999084472656
2010-07-02,23.100000381469727,18.709999084472656,23.0,19.200000762939453,5139800,19.200000762939453
2010-07-06,20.0,15.829999923706055,20.0,16.110000610351562,6866900,16.110000610351562
2010-07-07,16.6299991607666,14.979999542236328,16.399999618530273,15.800000190734863,6921700,15.800000190734863
2010-07-08,17.520000457763672,15.569999694824219,16.139999389648438,17.459999084472656,7711400,17.459999084472656
2010-07-09,17.899999618530273,16.549999237060547,17.579999923706055,17.399999618530273,4050600,17.399999618530273
2010-07-12,18.06999969482422,17.0,17.950000762939453,17.049999237060547,2202500,17.049999237060547
2010-07-13,18.639999389648438,16.899999618530273,17.389999389648438,18.139999389648438,2680100,18.139999389648438
2010-07-14,20.149999618530273,17.760000228881836,17.940000534057617,19.84000015258789,4195200,19.84000015258789
2010-07-15,21.5,19.0,19.940000534057617,19.889999389648438,3739800,19.889999389648438
2010-07-16,21.299999237060547,20.049999237060547,20.700000762939453,20.639999389648438,2621300,20.639999389648438

请给我建设性的反馈，因为我是新来的。

感谢：）

Answer 1

您缺少import

在脚本顶部添加以下import

from pandas_datareader._utils import RemoteDataError

Answer 2

import pandas as pd

df = pd.read_html(
    "https://en.wikipedia.org/wiki/List_of_S%26P_500_companies")[0]

sort = pd.DataFrame(df).sort_values(by=['Date first added'])
sort['Date first added'] = pd.to_datetime(sort['Date first added'])

start_date = '1-1-1999'
end_date = '11-12-2019'

mask = (sort['Date first added'] > start_date) & (
    sort['Date first added'] <= end_date)

sort = sort.loc[mask]

pd.DataFrame(sort).to_csv('result.csv', index=False)

输出：View Online

ScreenShot：

熊猫数据读取器出现故障

2 个答案: