如何在python中使用硒将表剪贴到熊猫数据框?

时间:2019-03-21 23:36:43

标签: python pandas selenium web-scraping beautifulsoup

我想复制表(id = symbolMarket)并将其另存为此链接https://www.myfxbook.com/en/forex-market/currencies/US30-historical-data中的熊猫数据框

我应该如何以简单/美观的方式进行操作?

很显然,我可以一个一个地检索元素,但是我相信有更好的方法。

(如果有帮助,我正在使用硒来访问页面)

非常感谢您与我分享知识

1 个答案:

答案 0 :(得分:2)

由于它很基础,我非常犹豫,因为它很基础,并且有很多解决方案来说明如何将html表读入pandas数据框。让我想知道您是否甚至尝试过先查找它。

但是,只需使用.read_html()。这将返回数据帧列表。因此,您只需要找出列表中所需的数据框即可:

import pandas as pd

url = 'https://www.myfxbook.com/en/forex-market/currencies/US30-historical-data'
tables = pd.read_html(url)

输出:

table = tables[3]
print (table)
                      0      1     ...                  5           6
0                  Date   Open     ...      Change (Pips)  Change (%)
1    Mar 20, 2019 21:00  25737     ...             +253.0      +0.97%
2    Mar 19, 2019 21:00  25871     ...             -135.0      -0.52%
3    Mar 18, 2019 21:00  25935     ...              -63.0      -0.24%
4    Mar 17, 2019 21:00  25864     ...              +70.0      +0.27%
5    Mar 16, 2019 21:00  25864     ...              -20.0      -0.08%
6    Mar 14, 2019 21:00  25716     ...             +153.0      +0.59%
7    Mar 13, 2019 21:00  25756     ...              -40.0      -0.16%
8    Mar 12, 2019 21:00  25575     ...             +185.0      +0.72%
9    Mar 11, 2019 21:00  25686     ...              -93.0      -0.36%
10   Mar 10, 2019 21:00  25470     ...             +212.0      +0.83%
11   Mar 09, 2019 21:00  25470     ...              -29.0      -0.11%
12   Mar 07, 2019 21:00  25459     ...              +61.0      +0.24%
13   Mar 06, 2019 21:00  25673     ...             -197.0      -0.77%
14   Mar 05, 2019 21:00  25786     ...             -108.0      -0.42%
15   Mar 04, 2019 21:00  25805     ...               +3.0      +0.01%
16   Mar 03, 2019 21:00  26114     ...             -300.0      -1.16%
17   Feb 28, 2019 21:00  25911     ...             +138.0      +0.53%
18   Feb 27, 2019 21:00  26018     ...              -89.0      -0.34%
19   Feb 26, 2019 21:00  26005     ...              +31.0      +0.12%
20   Feb 25, 2019 21:00  26093     ...              -63.0      -0.24%
21   Feb 24, 2019 21:00  26094     ...               -3.0      -0.01%
22   Feb 21, 2019 21:00  25825     ...             +210.0      +0.81%
23   Feb 20, 2019 21:00  25962     ...             -120.0      -0.46%
24   Feb 19, 2019 21:00  25877     ...              +88.0      +0.34%
25   Feb 18, 2019 21:00  25894     ...               -9.0      -0.03%
26   Feb 17, 2019 21:00  25905     ...               +5.0      +0.02%
27   Feb 14, 2019 21:00  25404     ...             +500.0      +1.93%
28   Feb 13, 2019 21:00  25483     ...              -68.0      -0.27%
29   Feb 12, 2019 21:00  25418     ...             +102.0      +0.40%
..                  ...    ...     ...                ...         ...
71   Dec 11, 2018 21:00  24341     ...             +208.0      +0.85%
72   Dec 10, 2018 21:00  24490     ...             -152.0      -0.62%
73   Dec 09, 2018 21:00  24338     ...             +144.0      +0.59%
74   Dec 06, 2018 21:00  24921     ...             -517.0      -2.12%
75   Dec 05, 2018 21:00  25118     ...             -189.0      -0.76%
76   Dec 04, 2018 21:00  25033     ...             +134.0      +0.53%
77   Dec 03, 2018 21:00  25837     ...             -798.0      -3.19%
78   Dec 02, 2018 21:00  25897     ...              -55.0      -0.21%
79   Nov 29, 2018 21:00  25367     ...             +220.0      +0.86%
80   Nov 28, 2018 21:00  25327     ...              +62.0      +0.24%
81   Nov 27, 2018 21:00  24794     ...             +568.0      +2.24%
82   Nov 26, 2018 21:00  24546     ...             +253.0      +1.02%
83   Nov 25, 2018 21:00  24300     ...             +230.0      +0.94%
84   Nov 22, 2018 21:00  24367     ...              -80.0      -0.33%
85   Nov 21, 2018 21:00  24497     ...             -144.0      -0.59%
86   Nov 20, 2018 21:00  24461     ...              +38.0      +0.16%
87   Nov 19, 2018 21:00  25063     ...             -604.0      -2.47%
88   Nov 18, 2018 21:00  25410     ...             -342.0      -1.36%
89   Nov 15, 2018 21:00  25335     ...             +135.0      +0.53%
90   Nov 14, 2018 21:00  25085     ...             +256.0      +1.01%
91   Nov 13, 2018 21:00  25378     ...             -273.0      -1.09%
92   Nov 12, 2018 21:00  25422     ...              -65.0      -0.26%
93   Nov 11, 2018 21:00  25987     ...             -577.0      -2.27%
94   Nov 08, 2018 21:00  26184     ...             -202.0      -0.78%
95   Nov 07, 2018 21:00  26190     ...              +15.0      +0.06%
96   Nov 06, 2018 21:00  25663     ...             +572.0      +2.18%
97   Nov 05, 2018 21:00  25481     ...             +200.0      +0.78%
98   Nov 04, 2018 21:00  25267     ...             +221.0      +0.87%
99   Nov 01, 2018 21:00  25240     ...              +40.0      +0.16%
100  Oct 31, 2018 21:00  25090     ...             +229.0      +0.90%

[101 rows x 7 columns]