将函数中返回的键值作为新列追加到数据框

Question

我有一个带有URL列表的数据框，我想为其提取几个值。然后应将返回的键/值添加到原始数据帧中，并将键作为新列和相应的值。

我认为这会神奇发生 result_type='expand'，显然不是。当我尝试

df5["data"] = df5.apply(lambda x: request_function(x['url']),axis=1, result_type='expand')

我最终将结果全部放在一个数据列中

[{'title': ['Python Notebooks: Connect to Google Search Console API and Extract Data - Adapt'], 'description': []}]

我想要的结果是一个包含以下3列的数据框：

| URL|      Title      |  Description|

这是我的代码：

import requests
from requests_html import HTMLSession
import pandas as pd
from urllib import parse

ex_dic = {'url': ['https://www.searchenginejournal.com/reorganizing-xml-sitemaps-python/295539/', 'https://searchengineland.com/check-urls-indexed-google-using-python-259773', 'https://adaptpartners.com/technical-seo/python-notebooks-connect-to-google-search-console-api-and-extract-data/']}

df5 = pd.DataFrame(ex_dic)
df5

def request_function(url):
    try:
        found_results = []
        r = session.get(url)
        title = r.html.xpath('//title/text()')
        description = r.html.xpath("//meta[@name='description']/@content")
        found_results.append({ 'title': title, 'description': description})
        return found_results


    except requests.RequestException:
        print("Connectivity error")      
    except (KeyError):
        print("anoter error")

df5.apply(lambda x: request_function(x['url']),axis=1, result_type='expand')

Answer 1

ex_dic应该是字典列表，以便您可以更新所应用的属性。

import requests
from requests_html import HTMLSession
import pandas as pd
from urllib import parse

ex_dic = {'url': ['https://www.searchenginejournal.com/reorganizing-xml-sitemaps-python/295539/', 'https://searchengineland.com/check-urls-indexed-google-using-python-259773', 'https://adaptpartners.com/technical-seo/python-notebooks-connect-to-google-search-console-api-and-extract-data/']}

ex_dic['url'] = [{'url': item} for item in ex_dic['url']]

df5 = pd.DataFrame(ex_dic)
session = HTMLSession()

def request_function(url):
    try:
        print(url)
        r = session.get(url['url'])
        title = r.html.xpath('//title/text()')
        description = r.html.xpath("//meta[@name='description']/@content")
        url.update({ 'title': title, 'description': description})
        return url


    except requests.RequestException:
        print("Connectivity error")      
    except (KeyError):
        print("anoter error")

df6 = df5.apply(lambda x: request_function(x['url']),axis=1, result_type='expand')
print df6

Answer 2

如果您的函数仅返回字典，而不返回字典列表，则它实际上可以按您期望的方式工作。此外，密钥内部仅提供一个字符串，而不是列表。然后，它会按您的预期工作。参见我的示例代码：

import requests
import pandas as pd
from urllib import parse

ex_dic = {'url': ['https://www.searchenginejournal.com/reorganizing-xml-sitemaps-python/295539/', 'https://searchengineland.com/check-urls-indexed-google-using-python-259773', 'https://adaptpartners.com/technical-seo/python-notebooks-connect-to-google-search-console-api-and-extract-data/']}

df5 = pd.DataFrame(ex_dic)
#rint(df5)

def request_function(url):
    return {'title': 'Python Notebooks: Connect to Google Search Console API and Extract Data - Adapt', 
            'description': ''}


df6 = df5.apply(lambda x: request_function(x['url']), axis=1, result_type='expand')
df7 = pd.concat([df5,df6],1)


df7

给你这个：

您还可以调整Lambda函数：

df6 = df5.apply(lambda x: request_function(x['url'])[0], axis=1, result_type='expand')

但是您仍然需要确保键值是字符串，而不是列表。