Question

我正在尝试使用ensembl genome browser API获取一些基因组信息。挑战在于每个网络请求可能要花费几秒钟，因此我一直在尝试使用asyncio等待这些网络请求，同时处理我拥有的数据。

这是我正在使用的示例输入DataFrame：

import pandas as pd
df = pd.DataFrame({'Gene Name': {0: 'A1CF', 1: 'A1CF', 2: 'A1CF'},
 'Sample Name': {0: 'ATL045', 1: 'QC2-20-T2', 2: 'GHE0624'},
 'CDS Mutation': {0: 'c.234A>C', 1: 'c.492C>T', 2: 'c.490G>A'},
 'AA Mutation': {0: 'p.K78N', 1: 'p.V164V', 2: 'p.V164I'}})

目标是使用上述Gene Name中的CDS Mutation和df信息来获取其他一些基因组信息。

第一个方法旨在调用ensembl_calls方法，该方法将发出网络请求并返回一些已解析的输出。理想情况下，将解析后的输出组合成类似大熊猫主数据帧的内容。

async def concurrent_location_info(df):
    import pandas as pd
    import asyncio

    full_df = pd.DataFrame()

    # iterate through DataFrame
    dfs = [asyncio.ensure_future(ensembl_calls(row)) for index, row in df.iterrows()]

    print(dfs)

通过这种方法，我试图发出网络请求并解析我得到的信息。

# this makes the network ensembl call asynchronously
async def ensembl_calls(row):
    new_df = {}

    try: # sometimes ensembl can't find what i'm looking for
        # this can take a while
        await info = Ensembl(row['Gene Name'], row['CDS Mutation']).info().split(',')

        # parse the output
        new_df['Gene'] = row['Gene Name']
        new_df['Chrom'] = info[0]
        new_df['Start'] = info[1]
        new_df['End'] = info[2]
        new_df['WT'] = info[3]
        new_df['Var'] = info[4]
        new_df['Sift_Index'] = info[5]
    except:
        pass

    return new_df # ideally somehow gets added to a master pd dataframe

我在这里正确吗？有办法使它起作用吗？

Answer 1

尝试一下：

import pandas as pd
import asyncio


async def concurrent_location_info(df):
    # iterate through DataFrame
    tasks = [ensembl_calls(row) for _, row in df.iterrows()]
    df = pd.concat(asyncio.gather(*tasks))
    print(df)

并像这样修改第二个功能

info = await Ensembl(...)

这假设Ensembl具有异步的 init 方法，这是一种罕见的模式。

正确实现异步

1 个答案: