Question

我的循环首先不按照我想要的方式工作。它逐步为特定网站提供字典内的链接。我想立刻填写它。我的输出就像：

{'Banks – Assets': {'link': 'https://data.gov.au/dataset/banks-assets'}, 'Consolidated Exposures – Immediate and Ultimate Risk Basis': {}, 'Foreign Exchange Transactions and Holdings of Official Reserve Assets': {}, 'Finance Companies and General Financiers – Selected Assets and Liabilities': {}, 'Liabilities and Assets – Monthly': {}, 'Consolidated Exposures – Immediate Risk Basis – International Claims by Country': {}, 'Consolidated Exposures – Ultimate Risk Basis': {}, 'Banks – Consolidated Group off-balance Sheet Business': {}, 'Liabilities of Australian-located Operations': {}, 'Building Societies – Selected Assets and Liabilities': {}, 'Consolidated Exposures – Immediate Risk Basis – Foreign Claims by Country': {}, 'Banks – Consolidated Group Impaired Assets': {}, 'Assets and Liabilities of Australian-Located Operations': {}, 'Managed Funds': {}, 'Daily Net Foreign Exchange Transactions': {}, 'Consolidated Exposures-Immediate Risk Basis': {}, 'Public Unit Trust': {}, 'Securitisation Vehicles': {}, 'Assets of Australian-located Operations': {}, 'Banks – Consolidated Group Capital': {}}
{'Banks – Assets': {'link': 'https://data.gov.au/dataset/banks-assets'}, 'Consolidated Exposures – Immediate and Ultimate Risk Basis': {'link': 'https://data.gov.au/dataset/consolidated-exposures-immediate-and-ultimate-risk-basis'}, 'Foreign Exchange Transactions and Holdings of Official Reserve Assets': {}, 'Finance Companies and General Financiers – Selected Assets and Liabilities': {}, 'Liabilities and Assets – Monthly': {}, 'Consolidated Exposures – Immediate Risk Basis – International Claims by Country': {}, 'Consolidated Exposures – Ultimate Risk Basis': {}, 'Banks – Consolidated Group off-balance Sheet Business': {}, 'Liabilities of Australian-located Operations': {}, 'Building Societies – Selected Assets and Liabilities': {}, 'Consolidated Exposures – Immediate Risk Basis – Foreign Claims by Country': {}, 'Banks – Consolidated Group Impaired Assets': {}, 'Assets and Liabilities of Australian-Located Operations': {}, 'Managed Funds': {}, 'Daily Net Foreign Exchange Transactions': {}, 'Consolidated Exposures-Immediate Risk Basis': {}, 'Public Unit Trust': {}, 'Securitisation Vehicles': {}, 'Assets of Australian-located Operations': {}, 'Banks – Consolidated Group Capital': {}}

我要做的第二件事就是：

Titles                                                  Links
Banks - Assets                      https://data.gov.au/dataset/banks-assets
Consolidated Exposures – Immediate and Ultimate Risk Basis   https://data.gov.au/dataset/consolidated-exposures-immediate-and-ultimate-risk-basis

依旧...... 我的代码是：

webpage4_urls = ["https://data.gov.au/dataset?q=&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&organization=departmentofagriculturefisheriesandforestry&_groups_limit=0",
                 "https://data.gov.au/dataset?q=&organization=commonwealthscientificandindustrialresearchorganisation&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&_groups_limit=0",
                 "https://data.gov.au/dataset?q=&organization=bureauofmeteorology&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&_groups_limit=0",
                 "https://data.gov.au/dataset?q=&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&organization=tasmanianmuseumandartgallery&_groups_limit=0",
                 "https://data.gov.au/dataset?q=&organization=department-of-industry&sort=extras_harvest_portal+asc%2C+score+desc%2C+metadata_modified+desc&_organization_limit=0&groups=sciences&_groups_limit=0"]
for i in webpage4_urls:
    wiki2 = i
    page= urllib.request.urlopen(wiki2)
    soup = BeautifulSoup(page)

    lobbying = {}
    data2 = soup.find_all('h3', class_="dataset-heading")
    for element in data2:
        lobbying[element.a.get_text()] = {}
    data2[0].a["href"]
    prefix = "https://data.gov.au"
    for element in data2:
        lobbying[element.a.get_text()]["link"] = prefix + element.a["href"]
        print(lobbying)

Answer 1

我认为您需要DataFrame.from_dict + DataFrame.rename_axis + DataFrame.reset_index：

Mark as Deployable

编辑：

for element in data2:
    lobbying[element.a.get_text()]["link"] = prefix + element.a["href"]
    #print(lobbying)
    df = pd.DataFrame.from_dict(lobbying, orient='index').rename_axis('Titles').reset_index()
    print (df)

在python中将字典转换为数据框架

1 个答案: