Python:比较行和列之间的值

时间:2018-08-23 01:43:23

标签: python pandas beautifulsoup python-requests urllib

我想在行和列之间找到一种模式,并认为Panda可能有用,但是以某种方式我无法索引Pandas中的输出。它给出了错误,例如列表超出范围,数据框讲师错误地调用了诸如此类。我想在行之间找到一个变化,例如2018年9月和2018年10月或2019年2月和2019年3月。在代码末尾输出。

from urllib.request import urlopen
from bs4 import BeautifulSoup
import requests
import pandas as pd
url = "https://quotes.ino.com/exchanges/contracts.html?r=NYMEX_NG"
res = requests.get(url)
soup = BeautifulSoup(res.text, 'lxml')
column_headers = [th.getText() for th in soup.findAll('tr', limit=3)[2].findAll('th')]

print(column_headers)
data_rows = soup.findAll('tr')[3:]
for td in data_rows:
    Market = td.findAll('td')[0].text
    Contract = td.findAll('td')[1].text
    Open = td.findAll('td')[2].text
    High = td.findAll('td')[3].text
    Low = td.findAll('td')[4].text
    Last = td.findAll('td')[4].text
    Change = td.findAll('td')[4].text
    Pct = td.findAll('td')[4].text
    Time = td.findAll('td')[4].text

    print( Market, Contract, Open, High, Low, Last,Change, Pct, Time)

输出

仅部分复制,因为这会产生很多行


 ['Market', 'Contract', 'Open', 'High', 'Low', 'Last', 'Change', 'Pct', 'Time']
  NG.U18.E Sep 2018 (E) 2.958 2.960 2.945 2.945 2.945 2.945 2.945
  NG.V18.E Oct 2018 (E) 2.944 2.946 2.932 2.932 2.932 2.932 2.932
  NG.X18.E Nov 2018 (E) 2.975 2.977 2.964 2.964 2.964 2.964 2.964
  NG.Z18.E Dec 2018 (E) 3.068 3.071 3.058 3.058 3.058 3.058 3.058
  NG.F19.E Jan 2019 (E) 3.154 3.157 3.144 3.144 3.144 3.144 3.144
  NG.G19.E Feb 2019 (E) 3.117 3.118 3.110 3.110 3.110 3.110 3.110
  NG.H19.E Mar 2019 (E) 3.009 3.015 3.005 3.005 3.005 3.005 3.005
  NG.J19.E Apr 2019 (E) 2.698 2.698 2.698 2.698 2.698 2.698 2.698
  NG.K19.E May 2019 (E) 2.671 2.675 2.662 2.662 2.662 2.662 2.662
  NG.M19.E Jun 2019 (E) 2.697 2.701 2.692 2.692 2.692 2.692 2.692
  NG.N19.E Jul 2019 (E) 2.727 2.730 2.717 2.717 2.717 2.717 2.717
  NG.Q19.E Aug 2019 (E) 2.736 2.736 2.722 2.722 2.722 2.722 2.722

1 个答案:

答案 0 :(得分:1)

好的,所以这是将其转储到DataFrame中的方法,例如仅使用data_rows的前10行:

from pandas import DataFrame as DF

# the rest of your import statements...
# the rest of your code up until the `for td in data_rows` loop


table_data = [] # empty container for our table's data
for td in data_rows[:10]:
     table_data.append(list(e.text for e in td.findAll('td')))

# create the DataFrame:
df = DF(table_data, columns=column_headers)

print(df)

输出以下帧。此时,您该如何处理取决于您。

     Market      Contract   Open   High    Low   Last  Change     Pct   Time
0  NG.U18.E  Sep 2018 (E)  2.958  2.960  2.945  2.955  -0.001  -0.03%  21:53
1  NG.V18.E  Oct 2018 (E)  2.944  2.946  2.932  2.943  -0.001  -0.03%  21:53
2  NG.X18.E  Nov 2018 (E)  2.975  2.977  2.964  2.974  -0.001  -0.03%  21:48
3  NG.Z18.E  Dec 2018 (E)  3.068  3.071  3.058  3.068  -0.001  -0.03%  21:48
4  NG.F19.E  Jan 2019 (E)  3.154  3.157  3.144  3.155  +0.001  +0.03%  21:32
5  NG.G19.E  Feb 2019 (E)  3.117  3.118  3.110  3.118   0.000   0.00%  19:36
6  NG.H19.E  Mar 2019 (E)  3.009  3.015  3.005  3.015  +0.001  +0.03%  19:36
7  NG.J19.E  Apr 2019 (E)  2.698  2.698  2.698  2.698  -0.007  -0.26%  18:13
8  NG.K19.E  May 2019 (E)  2.671  2.675  2.662  2.670  -0.003  -0.11%  16:02
9  NG.M19.E  Jun 2019 (E)  2.697  2.701  2.692  2.695  -0.004  -0.15%  15:26