BeautifulSoup花费太长时间来解析页面

时间:2019-06-23 20:38:37

标签: python python-3.x web-scraping beautifulsoup html-parsing

我试图解析My Anime List page以获得“ ID”字段,以后需要进行API调用。函数prepare_soup花费的时间太长(超过2分钟),这是一个问题。如何加快此过程?

def prepare_soup():
    print("Fetching the pre requisites")
    anime_list_path = "https://myanimelist.net/info.php?search=%25%25%25&go=relationids&divname=relationGen1"
    anime_page = urllib.request.urlopen(anime_list_path)
    print("Preparing the soup. This may take a while")
    soup = BeautifulSoup(anime_page,"lxml")
    print("All done")
    return soup

0 个答案:

没有答案