用线程烧瓶获取数据

时间:2020-03-15 18:05:18

标签: python multithreading

我有一个很大的数据文件,其中包含1000多个记录。 对于该文件中的每条记录,我正在重建对象并构建另一个对象。该对象是在另一个向该对象添加信息的端点的帮助下构建的。最后,我附加了所有对象并将它们发送回去。此操作需要花费很多时间,因为我在每个条目上都执行此操作(读取对象详细信息并从另一个端点获取其他数据,以最终构造最终的对象。我考虑过将对象的每种构造拆分为单独的线程,并以这种方式更快地检索所需的结果。 我不知道如何将该代码拆分为线程代码,如果这是适合这种情况的正确解决方案,那么我就不知道了。

这是我的工作,需要很多时间: 该函数从Flask API的/ api / all-cards端点运行:

def get_all_players_full_data(ea_players_json):
all_full_data_players = []
# Legends or Players
for player_data in ea_players_json.values():
    for index in range(len(player_data)):
        if player_data[index].get("c") is not None:
            player_full_name = player_data[index].get("c")
        else:
            player_full_name = player_data[index].get("f") + " " + player_data[index].get("l")
        player_id = player_data[index].get("id")
        # go to futhead to find all cards of that player
        futhead_url_player_data = f'{FUTHEAD_PLAYER}{player_full_name}'
        details_of_specific_player = json.loads(requests.get(futhead_url_player_data).content)

        for player_in_json_futhead in details_of_specific_player:
            if player_in_json_futhead["player_id"] == player_id:
                rating = player_in_json_futhead["rating"]
                specific_card_id = player_in_json_futhead["def_id"]
                revision = player_in_json_futhead["revision_type"]
                name = player_in_json_futhead["full_name"]
                nation = player_in_json_futhead["nation_name"]
                position = player_in_json_futhead["position"]
                club = player_in_json_futhead["club_name"]

                all_full_data_players.append(Player(specific_card_id, name, rating, revision, nation, 
                position, club))
return all_full_data_players

0 个答案:

没有答案