我有df
2016-06-21 06:25:09 upi88@yandex.ru GET HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 application/json 2130 https://edge-chat.facebook.com/pull?channel=p_100006170407238&seq=27&clientid=1d67ca6e&profile=mobile&partition=-2&sticky_token=185&msgs_recv=27&qp=y&cb=1830997782&state=active&sticky_pool=frc3c09_chat-proxy&uid=100006170407238&viewer_uid=100006170407238&m_sess=&__dyn=1Z3p5wnE-4UpwDF3GAgy78qzoC6Erz8B0GxG9xu3Z0QwFzohxO3O2G2a1mwYxm48sxadwpVEy1qK78gwUx6&__req=79&__ajax__=AYlbtcBwGC2suZLI-J88V0PWa58vtQeG3YlQLydFRsAl6UwLSjsSpD7peu8mGl6NsHvd2zxfDcB6A0-XunBugUsYZ1lMYmUu97R43iV7XSfpyg&__user=100006170407238
2016-06-22 06:25:20 upi88@yandex.ru POST HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 application/x-javascript 20248 https://m.facebook.com/stories.php?aftercursor=MTQ2NjY2MzEwNToxNDY2NjYzMTA1Ojg6NzM0ODg0MDExMjAyNDY1MzA5NToxNDY2NjYyNzk1OjA%3D&tab=h_nor&__m_log_async__=1
2016-06-23 06:25:25 upi88@yandex.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
2016-06-23 06:25:25 upi88@yandex.ru GET HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 text/html 1105 https://m.facebook.com/xti.php?xt=2.qid.6299270070554694533%3Amf_story_key.343726573953754118%3Aei.AI%40ecf11fb3faf9c0b1f73ce2a74bc9f228
2016-06-24 06:25:25 lemuska@mail.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
2016-06-25 06:25:25 lemuska@mail.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
2016-06-25 06:25:25 lemuska@mail.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
我需要为每个ID
(仅年,月和日)获取唯一日期。
期望的输出:
upi88@yandex.ru - 2016-06-21, 2016-06-22, 2016-06-23
lemuska@mail.ru - 2016-06-24, 2016-06-25
我怎样才能得到这个日期?
答案 0 :(得分:2)
您可以先从日期中提取所需信息:
df['filtered date'] = [w[:10] for w in df['date']]
然后你使用`drop duplicates':
output = df[['id','filtered date']].drop_duplicates()
然后,您可以为了清晰起见对数据框重新排序:
output.sort_values(by['id','filtered date'],inplace = True)
你终于得到了这种输出:
id filtered date
0 lemuska@mail.ru 2016-06-24
1 lemuska@mail.ru 2016-06-25
3 upi88@yandex.ru 2016-06-21
4 upi88@yandex.ru 2016-06-22
5 upi88@yandex.ru 2016-06-23
答案 1 :(得分:1)
Pandas为DataFrames提供了groupby函数,它应该适合您的需要。
# Generate dataframe with random values
mail = ['alice@foo.com', 'bob@bar.com', 'me@baz.com']
stime = datetime.strptime('2016-07-01 00:00:00', '%Y-%m-%d %H:%M:%S')
etime = datetime.strptime('2016-07-30 00:00:00', '%Y-%m-%d %H:%M:%S')
tdelta = etime - stime
tdiff = tdelta.days * 24 * 60 * 60 + tdelta.seconds
df = pd.DataFrame({
'mail': [choice(mail) for _ in range(10)],
'time':[stime + timedelta(seconds=randrange(tdiff)) for _ in range(10)]
})
# Group dataframe by column 'mail' and apply the lambda expression to
# transform the grouped set of values into unique time values.
r = df.groupby(by='mail').apply(lambda x: set(x['time'].values))
然后,您应该能够使用结果:
print(r)
mail
alice@foo.com {2016-07-24T16:42:12.000000000, 2016-07-07T15:...
bob@bar.com {2016-07-13T18:53:07.000000000, 2016-07-04T06:...
me@baz.com {2016-07-10T07:37:19.000000000, 2016-07-09T07:...
dtype: object
答案 2 :(得分:1)
这是一个单行(假设date
和ID
作为相关列的名称)
df.groupby('ID').apply(lambda x: (x['date'].str[:10]).unique())
及其输出
ID
lemuska@mail.ru [2016-06-24, 2016-06-25]
upi88@yandex.ru [2016-06-21, 2016-06-22, 2016-06-23]
dtype: object
答案 3 :(得分:1)
让我们在下面阅读您的样本数据:
import pandas as pd
import StringIO
df = pd.read_table(StringIO.StringIO("""2016-06-21 06:25:09 upi88@yandex.ru GET HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 application/json 2130 https://edge-chat.facebook.com/pull?channel=p_100006170407238&seq=27&clientid=1d67ca6e&profile=mobile&partition=-2&sticky_token=185&msgs_recv=27&qp=y&cb=1830997782&state=active&sticky_pool=frc3c09_chat-proxy&uid=100006170407238&viewer_uid=100006170407238&m_sess=&__dyn=1Z3p5wnE-4UpwDF3GAgy78qzoC6Erz8B0GxG9xu3Z0QwFzohxO3O2G2a1mwYxm48sxadwpVEy1qK78gwUx6&__req=79&__ajax__=AYlbtcBwGC2suZLI-J88V0PWa58vtQeG3YlQLydFRsAl6UwLSjsSpD7peu8mGl6NsHvd2zxfDcB6A0-XunBugUsYZ1lMYmUu97R43iV7XSfpyg&__user=100006170407238
2016-06-22 06:25:20 upi88@yandex.ru POST HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 application/x-javascript 20248 https://m.facebook.com/stories.php?aftercursor=MTQ2NjY2MzEwNToxNDY2NjYzMTA1Ojg6NzM0ODg0MDExMjAyNDY1MzA5NToxNDY2NjYyNzk1OjA%3D&tab=h_nor&__m_log_async__=1
2016-06-23 06:25:25 upi88@yandex.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
2016-06-23 06:25:25 upi88@yandex.ru GET HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 text/html 1105 https://m.facebook.com/xti.php?xt=2.qid.6299270070554694533%3Amf_story_key.343726573953754118%3Aei.AI%40ecf11fb3faf9c0b1f73ce2a74bc9f228
2016-06-24 06:25:25 lemuska@mail.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
2016-06-25 06:25:25 lemuska@mail.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
2016-06-25 06:25:25 lemuska@mail.ru CONNECT HTTP/1.1 Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53 200 - 0 scontent.xx.fbcdn.net:443
"""), delim_whitespace=True, header=None)
您感兴趣的是第一个(index:0)列,即日期和第三个(索引:2),即电子邮件地址。纯粹出于可见性的原因,让我们将它们隔离在新的数据框中:
df2 = df[[0, 2]]
现在是:
0 2
0 2016-06-21 upi88@yandex.ru
1 2016-06-22 upi88@yandex.ru
2 2016-06-23 upi88@yandex.ru
3 2016-06-23 upi88@yandex.ru
4 2016-06-24 lemuska@mail.ru
5 2016-06-25 lemuska@mail.ru
6 2016-06-25 lemuska@mail.ru
我们现在需要对它们进行分组并使用自定义函数进行聚合,这会将聚合日期转换为列表(如您所需的输出):
df2.groupby(2).agg(lambda x: x.unique().tolist()).reset_index()
reset_index()
修复索引,以便得到以下数据框:
2 0
0 lemuska@mail.ru [2016-06-24, 2016-06-25]
1 upi88@yandex.ru [2016-06-21, 2016-06-22, 2016-06-23]