Question

我刚开始使用Pandas，并且正在使用域清理工具。本质上，我想删除所有子域并只保留主域+ tld。

以下内容在ipython中针对单个域起作用，但是我在针对多个域的数据框进行挣扎。

该脚本似乎可以正常运行，但是正则表达式导致行字符（如下所示）的结尾打印在域的结尾（例如com \ n1'）

我不确定这些字符是什么-我尝试过rstrip，但没有用。谁能告诉我这些字符是什么以及我如何摆脱它们以使脚本起作用？

输出

['0          graph', 'facebook', 'com\n1              news', 'bbc', 'co', 'uk\n2

预期输出

当我在ipython中运行相同代码时，我得到以下信息-使用df列时，拆分必须相同。

In [12]: re.split(r'\.(?!\d)', (str('domain.domain.com')))
Out[12]: ['domain', 'domain', 'com']

输入

In [1]: import pandas as pd

In [2]: import re

In [3]: path = "Desktop/domains.csv"

In [4]: df = pd.read_csv(path, delimiter=',', header='infer')

In [5]: df
Out[5]: 
                     Domain
0        graph.facebook.com
1            news.bbc.co.uk
2  news.more.news.bbc.co.uk
3       profile.username.co
4           offers.o2.co.uk
5     subdomain.pyspark.org
6       uds.data.domain.net

In [7]: for index, row in df.iterrows():
   ...:     tld = ['co.uk', 'com', 'org', 'co', 'net']
   ...:     index = re.split(r'\.(?!\d)', (str(df.Domain)))
   ...:     print(index)
   ...:     if str(index[len(index)-2]).rstrip()+'.'+ str(index[len(index)-1]).rstrip() in tld:
   ...:         print(str(index[len(index)-3])+'.'+str(index[len(index)-2])+'.'+ str(index[len(index)-1]))
   ...:     elif str(index[len(index)-1]) in tld:
   ...:         print(str(index[len(index)-2])+'.'+ str(index[len(index)-1]))

更新

谢谢大家提供的帮助。

下面的代码现在可以完全按预期工作，但是输出都重复了几次。

例如您可以看到Facebook.com在输出列表中已打印两次，我真的不明白为什么-有人可以建议吗？

输入

In [38]: for row in df.iterrows():
    ...:     tld = ['co.uk', 'com', 'org', 'co', 'net']
    ...:     index = df.Domain[df.Domain.str.strip().str.endswith(tuple(tld))].str.split('.').tolist()
    ...:     for x in index:
    ...:         if str(x[len(x)-2]).rstrip()+'.'+ str(x[len(x)-1]).rstrip() in tld:
    ...:             print(str(x[len(x)-3])+'.'+str(x[len(x)-2])+'.'+ str(x[len(x)-1]))
    ...:         elif str(x[len(x)-1]) in tld:
    ...:             print(str(x[len(x)-2])+'.'+ str(x[len(x)-1]))
    ...: 
    ...:                     
facebook.com
bbc.co.uk
bbc.co.uk
username.co
o2.co.uk
pyspark.org
domain.net
facebook.com

Answer 1

这是工作代码：

In [5]: import pandas as pd
In [6]: import re
#Define the path of the file & generate the dataframe from it
In [7]: path = "Desktop/domains.csv"
In [8]: df = pd.read_csv(path, delimiter=',', header='infer')
#Show the dataframe to validate input is correct
In [9]: df
Out[10]:
                     Domain
0        graph.facebook.com
1            news.bbc.co.uk
2  news.more.news.bbc.co.uk
3       profile.username.co
4           offers.o2.co.uk
5     subdomain.pyspark.org
6       uds.data.domain.net

#Iterate through the rows of the dataframe. If the domain ends with anything in the TLD list, then split the domains at the '.' into a list
In [11]: for row in df.iterrows():
    ...:     tld = ['co.uk', 'com', 'org', 'co', 'net']
    ...:     index = df.Domain[df.Domain.str.strip().str.endswith(tuple(tld))].str.split('.').tolist()
    ...:
#Now, let's look at the output of index. It's a list of lists. So, if we select index[1]. we will get  ['news', 'bbc', 'co', 'uk']
In [12]: index
Out[12]:
[['graph', 'facebook', 'com'],
 ['news', 'bbc', 'co', 'uk'],
 ['news', 'more', 'news', 'bbc', 'co', 'uk'],
 ['profile', 'username', 'co'],
 ['offers', 'o2', 'co', 'uk'],
 ['subdomain', 'pyspark', 'org'],
 ['uds', 'data', 'domain', 'net']]

#We therefore need to iterate through each item of the lists
#So, we need to go through index[0][0] (the sub lists - to see the breakdown of each domain)
#Now we take the length of the list e.g. graph, facebook, com = length of 3
#as it's a zero'd index, we need to start at 0 and hence a list with a length of 3, has a range of 0-2
#so first, we check, is the last but one character + the last character in the TLD list (e.g. co.uk)
#if yes, we take the last 3 elements of the list, to produce 'bbc.co.uk'
#if no, we go through and check the last character & see if its in the list (e.g. com)
#if it is, we print the last 2 characters (e.g. facebook.com)
    ...: for x in index:
    ...:     if str(x[len(x)-2]).rstrip()+'.'+ str(x[len(x)-1]).rstrip() in tld:
    ...:         print(str(x[len(x)-3])+'.'+str(x[len(x)-2])+'.'+ str(x[len(x)-1]))
    ...:     elif str(x[len(x)-1]) in tld:
    ...:         print(str(x[len(x)-2])+'.'+ str(x[len(x)-1]))
    ...:

#and here is the ouput
facebook.com
bbc.co.uk
bbc.co.uk
username.co
o2.co.uk
pyspark.org
domain.net

python / pandas中的正则表达式导致行尾字符奇怪

1 个答案: