Question

我正在用Selenium和BeautifulSoup抓取视频源。我想问问是否有办法用Firefox或Chrome提取m3u8文件（HLS内容）而不是blob文件？

以下代码使用Selenium Safari Web驱动程序将视频源作为播放列表字符串抓取。

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium import webdriver
from bs4 import BeautifulSoup
import re
import urllib.request


def get_all_channels(base: str="https://www.telewebion.com/channels"):
    channels_url = urllib.request.urlopen(f"{base}")
    soup_channels_url = BeautifulSoup(channels_url, "lxml")

    # create a list of all channels
    all_channels_list = []
    for a in soup_channels_url.select('.no-featured a'):
        all_channels_list.append(a['href'])
        # all_channels_list.append(a['href'], a.get_text(strip=True))

    # return the list
    return all_channels_list


def get_video_src(url: str, base: str="https://www.telewebion.com"):
    channel_url = f"{base}{url}"

    wd = webdriver.Safari()
    # wd = webdriver.Chrome()
    # wd = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')

    wd.get(channel_url)
    WebDriverWait(wd, 6000).until(EC.visibility_of_element_located(
        (By.CLASS_NAME, "position-relative")))

    html_page = wd.page_source

    # Now use html_page
    soup = BeautifulSoup(html_page, "lxml")

    video = soup.find_all("video", class_="rmp-object-fit-contain")
    video_src = video[0]['src']

    wd.quit()

    return video_src

for channel in get_all_channels():
    print(get_video_src(channel))

结果是我感兴趣的m3u8播放列表（HLS内容）字符串，但是它不是可伸缩的解决方案，因为它仅在安装Safari时有效。 Selenium的Firefox / Chrome Web驱动程序将返回Blob字符串。我的最终目标是下载扩展的M3U（m3u8）播放列表（或任何其他类型的视频流），而不是下载视频流的一部分，以便用作Kodi附加视频源。

P.S。视频源是动态的，并由JavaScript渲染以加载其内容；因此，我使用Selenium来调用浏览器。

Answer 1

我认为您不需要使用硒来获取频道或频道链接的列表。

STEPS：您可以使用任何喜欢的编程语言。

 1. Get All channels:
Make a get request to this url to get all the channels.
https://wa1.telewebion.com/v2/channels/getChannels?logo_version=4&thumb_size=240

If you look at the response. "data" is an array of channel that has attribute called "descriptor" which gives us value of "channel_desc" for next request

 2. Get channel links:
Make a get request to using link below to get all links of channel from first request
https://wa1.telewebion.com/v2/channels/getChannelLinks?channel_desc=tv1&device=desktop&logo_version=4

The channel desc value "tv1" was received from first call.
On the response if you look at the links on data you will see all the m3u8 urls to for the tv1 channel. 

 3. Now you can use https://github.com/carlanton/m3u8-parser 
 to parse the m3u8 file to get the playlist urls or segment urls on the master or media manifests.

您可以在此处阅读有关m3u8规范的信息：https://tools.ietf.org/html/draft-pantos-http-live-streaming-08

有没有办法通过Firefox / Chrome获得HTTP Live Streaming（HLS）内容？

1 个答案: