Python废料哭泣比较

时间:2018-07-26 13:24:44

标签: python web-scraping cryptography bitcoin

我正在尝试使用python取消crytocompare输出的“每月利润”。该网站使用javascript获取输出。

目前,我有:

from selenium import webdriver
driver = webdriver.PhantomJS()
driver.get("https://www.cryptocompare.com/mining/calculator/xmr?HashingPower=65&HashingUnit=H%2Fs&PowerConsumption=0&CostPerkWh=0.12&MiningPoolFee=.5")
p_element = driver.find_element_by_id(id_='ng-binding')
print(p_element.text)

错误:

Warning (from warnings module):
  File "C:\Python27\lib\site-packages\selenium\webdriver\phantomjs\webdriver.py", line 49
    warnings.warn('Selenium support for PhantomJS has been deprecated, please use headless '
UserWarning: Selenium support for PhantomJS has been deprecated, please use headless versions of Chrome or Firefox instead

我遇到的问题是,我将使用Python 3.6 AWS Lambda函数

https://www.fullstackpython.com/blog/aws-lambda-python-3-6.html。我不相信可以安装无头版本的Chrome / Firefox吗?

上面的矫kill过正获得我想要的输出吗?请帮助

1 个答案:

答案 0 :(得分:1)

是的,您要尝试的是彻底的过度杀伤。 对API的简单查询将为您提供所有必需的信息:

https://min-api.cryptocompare.com/data/coin/generalinfo?fsyms=%20BTC,ETH,ETC,XMR,ZEC,AION,PASC,DASH,LTC,DCR&tsym=USD

(您可以在浏览器中对其进行测试)

计算每种货币所需的数据为:NetHashesPerSecond, BlockTime, BlockReward

您可以在此javascript文件中找到该页面执行的所有计算:

https://www.cryptocompare.com/scripts/services/mining.js(Chrome浏览器会很好地显示其格式)

检查功能$scope.generateValues

将这些计算结果转换为python应该很容易。

希望对您有帮助!

PS:我如何找到相关的js文件?如果使用浏览器调试页面,则会在此文件https://www.cryptocompare.com/DependencyHandler.axd?s=L3NjcmlwdHMvVmVuZG9yL2pxdWVyeS0xLjExLjMubWluLmpzOy9zY3JpcHRzL1ZlbmRvci9hbmd1bGFyLjEuNS41Lm1pbi5qczsvc2NyaXB0cy9WZW5kb3IvZXh0cmEtYW5ndWxhci1tb2R1bGVzLmpzOy9zY3JpcHRzL2luaXRhbmd1bGFyYXBwLmpzOy9zY3JpcHRzL1ZlbmRvci9zb2NrZXQuaW8tMi4wLjMuanM7L3NjcmlwdHMvVmVuZG9yL2hvdmVyLWRyb3Bkb3duLmpzOy9zY3JpcHRzL3N0cmVhbWluZy11dGlsaXRpZXMuanM7L3NjcmlwdHMvc2VydmljZXMvZ2xvYmFsLmpzOy9zY3JpcHRzL2NvbnRyb2xsZXJzL2dsb2JhbC5qczsvc2NyaXB0cy9kaXJlY3RpdmVzL2dsb2JhbC5qczsvc2NyaXB0cy9nbG9iYWwuanM7L3NjcmlwdHMvcm91dGVzL21pbmluZy1jYWxjdWxhdG9yLXBhZ2UuanM7L3NjcmlwdHMvc2VydmljZXMvbWluaW5nLmpzOy9zY3JpcHRzL2NvbnRyb2xsZXJzL21pbmluZy5qczsvc2NyaXB0cy9kaXJlY3RpdmVzL21pbmluZy5qczs&t=Javascript

中找到所有代码

我不是专家,但这看起来是Angular下载javascript依赖项的方式,如果您对base64解码该参数,则会看到:

/scripts/Vendor/jquery-1.11.3.min.js;/scripts/Vendor/angular.1.5.5.min.js;/scripts/Vendor/extra-angular-modules.js;/scripts/initangularapp.js;/scripts/Vendor/socket.io-2.0.3.js;/scripts/Vendor/hover-dropdown.js;/scripts/streaming-utilities.js;/scripts/services/global.js;/scripts/controllers/global.js;/scripts/directives/global.js;/scripts/global.js;/scripts/routes/mining-calculator-page.js;/scripts/services/mining.js;/scripts/controllers/mining.js;/scripts/directives/mining.js;

所有这些文件都可以下载。

干杯。