Python-数据框网址解析问题

时间:2019-05-01 19:42:08

标签: python-3.x urllib

我正在尝试将URL的域名从一列转换为另一列。它适用于像对象这样的字符串,当我应用于数据框时,它不起作用。如何将其应用于数据框?

尝试:

from urllib.parse import urlparse
import pandas as pd
id1 = [1,2,3]
ls = ['https://google.com/tensoflow','https://math.com/some/website',np.NaN]
df = pd.DataFrame({'id':id1,'url':ls})
df
# urlparse(df['url']) # ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
# df['url'].map(urlparse) # AttributeError: 'float' object has no attribute 'decode'

处理字符串:

string = 'https://google.com/tensoflow'
parsed_uri = urlparse(string)
result = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)
result

寻找一列:

col3
https://google.com/
https://math.com/
nan

错误

1 个答案:

答案 0 :(得分:1)

您可以尝试这样的事情。

  

在这里,我已经使用 pandas.Series.apply()来解决。

»初始化和导入

>>> from urllib.parse import urlparse
>>> import pandas as pd
>>> id1 = [1,2,3]
>>> import numpy as np
>>> ls = ['https://google.com/tensoflow','https://math.com/some/website',np.NaN]
>>> ls
['https://google.com/tensoflow', 'https://math.com/some/website', nan]
>>> 

»检查新创建的DataFrame。

>>> df = pd.DataFrame({'id':id1,'url':ls})
>>> df
   id                            url
0   1   https://google.com/tensoflow
1   2  https://math.com/some/website
2   3                            NaN
>>> 
>>> df["url"]
0     https://google.com/tensoflow
1    https://math.com/some/website
2                              NaN
Name: url, dtype: object
>>>

»使用 url 列上的pandas.Series.apply(func)应用功能。

>>> df["url"].apply(lambda url: "{uri.scheme}://{uri.netloc}/".format(uri=urlparse(url)) if not pd.isna(url) else np.nan)
0    https://google.com/
1      https://math.com/
2                    NaN
Name: url, dtype: object
>>> 
>>> df["url"].apply(lambda url: "{uri.scheme}://{uri.netloc}/".format(uri=urlparse(url)) if not pd.isna(url) else str(np.nan))
0    https://google.com/
1      https://math.com/
2                    nan
Name: url, dtype: object
>>> 
>>> 

»将以上结果存储在变量中(不是强制性的,只是为了简单起见)。

>>> s = df["url"].apply(lambda url: "{uri.scheme}://{uri.netloc}/".format(uri=urlparse(url)) if not pd.isna(url) else str(np.nan))
>>> s
0    https://google.com/
1      https://math.com/
2                    nan
Name: url, dtype: object
>>> 

»最后

>>> df2 = pd.DataFrame({"col3": s})
>>> df2
                  col3
0  https://google.com/
1    https://math.com/
2                  nan
>>> 

»为确保sdf2是什么,请检查类型(再次,不是强制性的)。

>>> type(s)
<class 'pandas.core.series.Series'>
>>> 
>>> 
>>> type(df2)
<class 'pandas.core.frame.DataFrame'>
>>> 

参考链接: