从CoinMarketCap导入数据

时间:2019-01-25 19:33:19

标签: python beautifulsoup python-requests pickle pandas-datareader

我正在关注一个YouTube教程,该教程正在从Yahoo为S&P500导入数据,而我正尝试从coinmarketcap导入数据以跟踪加密货币数据。

我正在尝试使用Pandas DataReader从coinmarketcap获取数据,但出现错误(NotImplementedError:data_source ='coinmarketcap'未实现)。

我不确定我的语法是否错误,或者我是否缺少模块,或者我使用的是不正确的data_source实现。

代码如下:

import coinmarketcap

import bs4 as bs
import datetime as dt
import os
import pandas_datareader as web
import pickle
import requests

def save_cmc_tickers():
    resp = requests.get('https://coinmarketcap.com')
    soup = bs.BeautifulSoup(resp.text, "lxml")
    table = soup.find('table', {'class': 'table floating-header'})
    #table1 = soup.find('table', {'class': 'no-wrap text-right'})
    tickers = []
    for row in table.findAll('tr')[1:]:
        ticker = row.findAll('a')[0].text
        price = row.findAll('a', {'class': 'price'})[0].text
        tickers.append(ticker)
        tickers.append(price)

    with open("cmctickers.pickle", "wb") as f:
        pickle.dump(tickers, f)

    print(tickers)

    return tickers

    #save_cmc_tickers()

    def get_data_from_cmc(reload_cmc100=False):

    if reload_cmc100:
        tickers = save_cmc_tickers()
    else:
        with open("cmctickers.pickle", "rb") as f:
            tickers = pickle.load(f)

    if not os.path.exists('stock_dfs'):
        os.makedirs('stock_dfs')

    start = dt.datetime(2010, 1, 1)
    end = dt.datetime(2017, 12, 31)

    for ticker in tickers:
        print(ticker)
        if not os.path.exists('stock_dfs/{}.csv'.format(ticker)):
            df = web.DataReader(ticker, 'coinmarketcap', start, end)
            df.to_csv('stock_dfs/{}.csv'.format(ticker))
        else:
            print("Already have {}".format(ticker))

get_data_from_cmc()

我之前也遇到了“ EOFError:输入不足”错误,所以我不知道这是否与它有关,但是不再出现。

任何信息都会有很大帮助。谢谢。

1 个答案:

答案 0 :(得分:0)

pandas_datareader不提供对CoinMarketCap的支持。这就是提出NotImplementedError的原因。

单独的注释:由于CoinMarketCap具有开发人员API(link),可以按以下方式(在obtaining an API key之后进行访问),因此不必为要查找的数据刮取CoinMarketCap,尽管免费计划仅涵盖某些端点。

# python 3
import requests

url = "https://pro-api.coinmarketcap.com/v1/cryptocurrency/ohlcv/historical"

params = {
    "time_start": "2017-01-01",
    "time_end": "2018-01-01",
    "interval": "1d"
}

headers = {
    "X-CMC_PRO_API_KEY": "<YOUR_API_KEY>"   
}

r = requests.get(url, params=params, headers=headers)
r.json()