Question

我有以下似乎无法正常工作的代码：

import pathlib
import requests
from bs4 import BeautifulSoup as bs
import re
import sys
import os
import lxml.html
url = sys.argv[1]
page = requests.get(url)
tree = lxml.html.fromstring(page.content)
names = tree.xpath('//div[@class="cd-timeline-block"]/text()')
names = filter(lambda n: n.strip(), names)
table = str.maketrans(dict.fromkeys('?:/'))
for index, name in enumerate(names, start = 1):
print('/{}.{}'.format(index, name.strip().translate(table)))

所以我想提取每个主题的名称和该主题中每个视频的名称，并从print命令获取此输出。格式应如下：

/0.Project Tools & Documentation/1.Organizational Change
/0.Project Tools & Documentation/2.Project Management Tools
/0.Project Tools & Documentation/3.Project Documentation
/0.Project Tools & Documentation/4.Vendor Documentation

在完成第一个主题之后，转到下一个主题，并输出下一个主题和视频：

 /1.Glossary/1.Review of Terms & Acroynms
 /1.Glossary/2.Review of Formulas

在我提取了这个之后，我也想提取“ href”链接：

and print it like that :

  https://streaming.ine.com/play/dfdf64b8-30a5-4bce-8ade- 
  a09ec56bcd6d/vendor-documentation

The page from where i want to extract this informations is :

https://streaming.ine.com/c/ine-comptia-pk0-004-project-plus

谢谢！

Answer 1

url = "https://streaming.ine.com/c/ine-comptia-pk0-004-project-plus"
import requests
page = requests.get(url)
import re

from bs4 import BeautifulSoup
soup = BeautifulSoup(page.content, 'html.parser') 
listed = soup.find_all('div',class_="cd-timeline-block")

for i,sth in enumerate(listed):
    soup1 = BeautifulSoup(sth.encode().decode('utf-8'))
    main_title = soup1.find_all('div',class_="cd-timeline-topic")[0].contents[0]

    sub_list = soup1.find_all('div',class_="cd-timeline-level")
    for j,elem in enumerate(sub_list): 
        temp = elem.contents[2].rstrip().strip('\n') 
        temp = re.sub(' +', ' ', temp)
        print("/%s.%s/%s.%s"%(str(i),main_title,str(j),temp[1:]))

重要的是要了解BeautifulSoup如何构造HTML页面，然后使用列表等标准数据结构。

输出为：

/0.Overview/0.Course Introduction
/1.Project Basics/0.What is a Project?
/1.Project Basics/1.Project Roles & Responsibilities
/1.Project Basics/2.Project Phases
/1.Project Basics/3.Cost Control
/1.Project Basics/4.Organizational Structures
/1.Project Basics/5.Project Schedules
/1.Project Basics/6.Agile
/1.Project Basics/7.Project Resources
/2.Project Constraints/0.Contraints
/2.Project Constraints/1.Risk Management
/3.Communication & Change Management/0.Communication Methods :: Overview
/3.Communication & Change Management/1.Use of Communication Methods
/3.Communication & Change Management/2.Communication Triggers
/3.Communication & Change Management/3.Change Control Processes
/4.Project Tools & Documentation/0.Organizational Change
/4.Project Tools & Documentation/1.Project Management Tools
/4.Project Tools & Documentation/2.Project Documentation
/4.Project Tools & Documentation/3.Vendor Documentation
/5.Glossary/0.Review of Terms & Acroynms
/5.Glossary/1.Review of Formulas
/6.Exam/0.Exam Preparation

我想提取每个主题的名称和每个视频的名称

1 个答案: