提取URL的特定部分

时间:2018-04-10 15:07:52

标签: python python-3.x beautifulsoup scrapy python-requests

对于给定的网址

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&p=1&u=%2Fnetahtml%2Fsearch-bool.html&r=1&f=G&l=50&d=pall&s1=706%2F20.CCLS.&OS=CCL/706/20&RS=CCL/706/20

如何通过在python中导入BeautifulSoup和请求库来提取Abstract和Claims部分,因为没有给出特定的div和class属性?对于摘要,我可以使用 p 标记,但可以用于声明部分。

2 个答案:

答案 0 :(得分:3)

看起来页面中的第一个<p>标记包含 Abstract 。因此,正如您所说,您可以使用soup.find('p').text直接获取它。

要从 Claims 部分获取文字,我使用了与xtratic has used in their answer类似的逻辑。首先,使用<i>Claims</i>找到soup.find('i', text='Claims')标记。然后使用find_all_next()函数和text=True来获取此标记之后的所有文字。

但是,这将获得所有文本直到页面结束,因此,如果text == 'Description',则打破循环。

claims_tag = soup.find('i', text='Claims')
for text in claims_tag.find_all_next(text=True):
    if text == 'Description':
        break
    print(text)

输出:

Claims


What is claimed is: 
 1.  A computer implemented method for providing lossless compression of an enumeration space for genetic founder lines, the computer implemented method comprising obtaining
an input comprising a set of genetic founder lines and a maximum number of generations G, wherein the following is iteratively performed for h=0, 1, .  . . , G: generating a set of genetic crossing templates of a height h, wherein each of the set of
genetic crossing templates represents a binary tree, the binary tree comprising h levels representing a given generation, each of the h levels comprising a set of nodes wherein when h>0 one or more of the h levels of the binary tree correspond to at
least one cross between at least one pair of nodes in the set of nodes, wherein each of the set of genetic crossing templates comprises an array of h entries, wherein a position of an entry within the array corresponds to a level in the binary tree
represented by the genetic crossing template, each of the h entries in the array comprising a value indicating a number of leaf nodes in the set of nodes for the level in the binary tree;  and determining if at least a first genetic crossing template in
the set of genetic crossing templates is redundant with respect to a second genetic crossing template in the set of genetic crossing templates;  and based on the at least first genetic crossing template being redundant, removing the at least first
genetic crossing template from the set of genetic crossing templates, the removing creating an updated set of genetic crossing templates.

 2.  The computer implemented method of claim 1, wherein the determining comprises: comparing the array of the at least first genetic crossing template with the array of the second genetic crossing template;  determining, based on the comparing,
if the value within each entry of the array for the at least first genetic crossing template matches the value within each corresponding entry of the array for the second genetic crossing template;  and determining that the at least first genetic
crossing template is redundant with respect to the second genetic crossing template based on the value within each entry of the array for the at least first genetic crossing template matching the value within each corresponding entry of the array for the
second genetic crossing template.

 3.  The computer implemented method of claim 1, wherein at least one of the set of genetic crossing templates of height h is generated by combining a previously generated genetic crossing template of a height less than h with another previously
generated genetic crossing template of a height less than h.

 4.  The computer implemented method of claim 1, further comprising: generating a set of genetic crossing instances for each genetic crossing template in the updated set of genetic crossing templates based on the set of founder lines, wherein
each genetic crossing instance in the set of genetic crossing instances is the binary tree represented by the genetic crossing template in the updated set of genetic crossing templates with leaf nodes at each of the h levels labeled with one of the set
of genetic founder lines, wherein the set of genetic crossing instances comprises a genetic crossing instance for each of a plurality of different leaf node labeling variations based on the set of genetic founder lines.

 5.  The computer implemented method of claim 4, further comprising: determining if at least a first genetic crossing instance in the set of genetic crossing instances is redundant with respect to a second genetic crossing instance in the set of
genetic crossing instances;  and based on the at least first genetic crossing instance being redundant, removing the at least first genetic crossing instance from the set of genetic crossing instances, the removing creating an updated set of genetic
crossing instances.

 6.  The computer implemented method of claim 5, wherein the determining comprises: comparing the binary tree of the at least a first genetic crossing instance with the binary tree of the second genetic crossing instance;  determining, based on
the comparing, if the label of each leaf node in the binary tree of the at least first genetic crossing instance matches a label of a corresponding leaf node in the binary tree of the second genetic crossing instance;  and determining that the at least
first genetic crossing instance is redundant with respect to the second genetic crossing instance based on the label of each leaf node in the binary tree of the at least first genetic crossing instance matching a label of a corresponding leaf node in the
binary tree of the second genetic crossing instance.

 7.  The computer implemented method of claim 4, wherein at least one of the set of genetic crossing instances is generated by combining each of a previously generated set of crossing instances of a given genetic crossing template in the update
set of genetic crossing templates with each of a previously generated set of crossing instances of at least one other genetic crossing template in the update set of genetic crossing templates. 

答案 1 :(得分:2)

我之前从未使用过BeautifulSoup并且在很长时间内都没有使用过Python,但是这样的事情应该可行(尽管你应该添加错误检查):

# Get the content of the claims section
# input:
#   soup - the root soup document
def getClaimsContent(soup):

    # the text content of the Claims section
    claimsContent = "";

    # first horizontal line starting the Claims section
    # might have to be "soup.find_all(isClaimsHeader)"
    firstHR = soup.find(isClaimsHeader).find('hr');

    # for every sibling, until the next horizontal line...
    node = firstHR.next_sibling;
    while(node.name.lower() != 'hr'):
        claimsContent += node.get_text();
        node = node.next_sibling;

    return claimsContent;


# Returns true if '<CENTER>... Claims ...</CENTER>' is the current tag
def isClaimsHeader(tag):
    # can be simplified, just expanded for debugging
    if (tag.name.lower() == 'center'):
        text = tag.get_text().lower();
        # for debugging, make sure the text actually equals 'claims'
        print('center text: ' + text);
        return (text == 'claims');

我希望我把一切都搞定了......如果这确实有用,请告诉我。

相关问题