Recieve JSON data from link in whattomine without scraping HTML

时间:2017-11-13 06:40:25

标签: json python-2.7 pandas python-requests urllib2

Explanation

This link is where you are sent to after entering in your hardware stats (hashrate, power, power cost, etc.). On the top bar (below the blue Twitter follow button) is a link to a JSON file created after the page loads with the hardware stats information entered; clicking on that JSON link redirects you to another URL (https://whattomine.com/asic.json).

Goal

My goal is to access that JSON file directly after manipulating the values in the URL string via the terminal. For example, if I would like to change hashrate from 100 to 150 in this portion of the URL:

[sha256_hr]=100& ---> [sha256_hr]=150&

After the URL manipulations (like above, but not limited to), I would like to receive the JSON output so that I can pick-out the desired data.

My Code

Advisory: I started Python programming ~June 2017, please forgive.

import json
import pandas as pd
import urllib2
import requests


hashrate_ghs = float(raw_input('Hash Rate (TH/s): '))
power_W = float(raw_input('Power of Miner (W): '))
electric_cost = float(raw_input('Cost of Power ($/kWh): '))
hashrate_ths = hashrate_ghs * 1000

initial_request = ('https://whattomine.com/asic?utf8=%E2%9C%93&sha256f=true&factor[sha256_hr]={0}&factor[sha256_p]={1}&factor[cost]={2}&sort=Profitability24&volume=0&revenue=24h&factor[exchanges][]=&factor[exchanges][]=bittrex&dataset=Main&commit=Calculate'.format(hashrate_ths, power_W, electric_cost))
data_stream_mine = urllib2.Request(initial_request)

json_data = requests.get('https://whattomine.com/asic.json')
print json_data

Error from My Code

I am getting an HTTPS handshake error. This is where my Python freshness is second most blatantly visible:

Traceback (most recent call last):
  File "calc_1.py", line 16, in <module>
    s.get('https://whattomine.com/asic.json')
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='whattomine.com', port=443): Max retries exceeded with url: /asic.json (Caused by SSLError(SSLError(1, u'[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)'),))

Thank you for your help and time!

Please advise me of any changes or for more information concerning this question.

2 个答案:

答案 0 :(得分:1)

看起来其他一些人遇到了类似的问题。

虽然对于某些人来说它似乎像pyOpenSSL version issue,卸载并重新安装已经解决了问题。 SO中另一个较旧的答案是do the following

答案 1 :(得分:1)

这只是一个评论。以下方法就足够了(Python 3)。

import requests

initial_request = 'http://whattomine.com/asic.json?utf8=1&dataset=Main&commit=Calculate'

json_data = requests.get(initial_request)
print(json_data.json())

此部分的关键点 - 将.json放入initial_request,这就足够了。 您可以在?签名

之后添加所有参数,就像在查询部分中所做的那样