公共框链接的可下载链接

Question

我希望能够通过python从公共Box链接中检索pdf，但我不太确定如何做到这一点。这是我希望能够下载的pdf类型的示例： https://fnn.app.box.com/s/ho73v0idqauzda1r477kj8g8okh72lje

我可以点击下载按钮或点击按钮在我的浏览器上获得可打印链接，但我还没有找到源html中此页面的链接。有没有办法以编程方式找到此链接？也许通过selenium或请求甚至通过盒子API？

非常感谢您的帮助！

Answer 1

这是获取pdf：

的下载链接的代码

from urllib.request import Request, urlopen
from bs4 import BeautifulSoup

def get_html(url, timeout = 15):
    ''' function returns html of url
    usually html = urlopen(url) is enough but sometimes it doesn't work
    also instead urllib.request you can use any other method to get html
    code of url like urllib or urllib2 (just search it online), but I
    think urllib.request comes with python installation'''

    html = ''
    try:
        html = urlopen(url, None, timeout)
    except:
        url = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
        try:
            html = urlopen(url, None, timeout)
        except:
            pass
    return html

def get_soup(html):
    ''' function returns soup of html code
    Beautiful Soup is a Python library for pulling data out of HTML
    and XML files. It works with your favorite parser to provide idiomatic
    ways of navigating, searching, and modifying the parse tree. It
    commonly saves programmers hours or days of work.
    more at https://www.crummy.com/software/BeautifulSoup/bs4/doc/'''

    soup = BeautifulSoup(html, "lxml")
    ## if it doesn't work instead of using "lxml"
    ## you can use any of these options:
    ##  soup = BeautifulSoup(html, "html.parser")
    ##  soup = BeautifulSoup(html, "lxml-xml")
    ##  soup = BeautifulSoup(html, "xml")
    ##  soup = BeautifulSoup(markup, "html5lib")
    return soup

def get_data_file_id(html):
    '''function returns data_file_id which is found in html code'''

    ## to scrape website i suggest using BeautifulSoup,
    ## you can do it manually using html.read() which will give you
    ## html code as string and then you need to do some string searching
    soup = get_soup(html)
    ## part of html code we are interested in is:
    ## <div class="preview" data-module="preview" data-file-id="69950302561" data-file-version-id="">
    ## we want to extract this data-file-id
    ## first we find div in which it's located in
    classifier = {"class": 'preview'}   ## classifier specifies div we are looking for
    div = soup.find('div', classifier)  ## we will get div which has class 'preview'
    ## now we can easily get data-file-id by using
    data_file_id = div.get('data-file-id')
    return data_file_id

    ## you can install BeautifulSoup from:
    ##      on windows http://www.lfd.uci.edu/~gohlke/pythonlibs/
    ##      or from https://pypi.python.org/pypi/beautifulsoup4/4.4.1
    ##      official page is https://www.crummy.com/software/BeautifulSoup/

    ## if you don't want to use BeautifulSoup than you should do smotehing like this:
    ##
    ##html_str = str(html.read())
    ##search_for = 'div class="preview" data-module="preview" data-file-id="'
    ##start = html_str.find(search_for) + len(search_for)
    ##end = html_str.find('"', start)
    ##data_file_id = html_str[start : end]
    ##
    ## it may seem easier  to do it than to use BeautifulSoup, but the problem is that
    ## if there is one more space in search_for or the order of div attributes is different
    ## or there sign " is used instead of ' and and vice versa this string searching
    ## won't work while BeautifulSoup will so I recommend  using BeautifulSoup

def get_url_id(url):
    ''' function returns url_id which is last part of url'''

    reverse_url = url[::-1]
    start = len(url) - reverse_url.find('/')    # start is position of last '/' in url
    url_id = url[start:]
    return url_id

def get_download_url(url_id, data_file_id):
    ''' function returns download_url'''

    start = 'https://fnn.app.box.com/index.php?rm=box_download_shared_file&shared_name='
    download_url = start + url_id + '&file_id=f_' + data_file_id
    return download_url 

url = 'https://fnn.app.box.com/s/ho73v0idqauzda1r477kj8g8okh72lje'
url = 'https://fnn.app.box.com/s/n74mnmrwyrmtiooqwppqjkrd1hhf3t3j'
html = get_html(url)
data_file_id = get_data_file_id(html)   ## we need data_file_id to create download url
url_id = get_url_id(url)                ## we need url_id to create download url
download_url = get_download_url(url_id, data_file_id)
## this actually isn't real download url
## you can get real url by using:
## real_download_url = get_html(download_url).geturl()
## but you will get really long url for your example it would be
## https://dl.boxcloud.com/d/1/4vx9ZWYeeQikW0KHUuO4okRjjQv3t6VGFTbMkh7weWQQc_tInOFR_1L_FuqVFovLqiycOLUDHu4o2U5EdZjmwnSmVuByY5DhpjmmdlizjaVjk6RMBbLcVhSt0ewtusDNL5tA8aiUKD1iIDlWCnXHJlcVzBc4aH3BXIEU65Ki1KdfZIlG7_jl8wuwP4MQG_yFw2sLWVDZbddJ50NLo2ElBthxy4EMSJ1auyvAWOp6ai2S4WPdqUDZ04PjOeCxQhvo3ufkt3og6Uw_s6oVVPryPUO3Pb2M4-Li5x9Cki882-WzjWUkBAPJwscVxTbDbu1b8GrR9P-5lv2I_DC4uPPamXb07f3Kp2kSJDVyy9rKbs16ATF3Wi2pOMMszMm0DVSg9SFfC6CCI0ISrkXZjEhWa_HIBuv_ptfQUUdJOMm9RmteDTstW37WgCCjT2Z22eFAfXVsFTOZBiaFVmicVAFkpB7QHyVkrfxdqpCcySEmt-KOxyjQOykx1HiC_WB2-aEFtEkCBHPX8BsG7tm10KRbSwzeGbp5YN1TJLxNlDzYZ1wVIKcD7AeoAzTjq0Brr8du0Vf67laJLuBVcZKBUhFNYM54UuOgL9USQDj8hpl5ew-W__VqYuOnAFOS18KVUTDsLODYcgLMzAylYg5pp-2IF1ipPXlbBOJgwNpYgUY0Bmnl6HaorNaRpmLVQflhs0h6wAXc7DqSNHhSnq5I_YbiQxM3pV8K8IWvpejYy3xKED5PM9HR_Sr1dnO0HtaL5PgfKcuiRCdCJjpk766LO0iNiRSWKHQ9lmdgA-AUHbQMMywLvW71rhIEea_jQ84elZdK1tK19zqPAAJ0sgT7LwdKCsT781sA90R4sRU07H825R5I3O1ygrdD-3pPArMf9bfrYyVmiZfI_yE_XiQ0OMXV9y13daMh65XkwETMAgWYwhs6RoTo3Kaa57hJjFT111lQVhjmLQF9AeqwXb0AB-Hu2AhN7tmvryRm7N2YLu6IMGLipsabJQnmp3mWqULh18gerlve9ZsOj0UyjsfGD4I0I6OhoOILsgI1k0yn8QEaVusHnKgXAtmi_JwXLN2hnP9YP20WjBLJ/download
## and we don't really care about real download url so i will use just download_url
print(download_url)

我也编写了下载pdf的代码：

from urllib.request import Request, urlopen

def get_html(url, timeout = 15):
    ''' function returns html of url
    usually html = urlopen(url) is enough but sometimes it doesn't work
    also instead urllib.request you can use any other method to get html
    code of url like urllib or urllib2 (just search it online), but I
    think urllib.request comes with python installation'''

    html = ''
    try:
        html = urlopen(url, None, timeout)
    except:
        url = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
        try:
            html = urlopen(url, None, timeout)
        except:
            pass
    return html

def get_current_path():
    ''' function returns path of folder in which python program is saved'''

    try:
        path = __file__
    except:
        try:
            import sys
            path = sys.argv[0]
        except:
            path = ''
    if path:
        if '\\' in path:
            path = path.replace('\\', '/')
        end = len(path) - path[::-1].find('/')
        path = path[:end]
    return path

def check_if_name_already_exists(name, path):
    ''' function checks if there is already existing pdf file
    with same name in folder given by path.'''

    try:
        file = open(path+name+'.pdf', 'r')
        file.close()
        return True
    except:
        return False

def get_new_name(old_name, path):
    ''' functions ask user to enter new name for file and returns inputted name.'''

    print('File with name "{}" already exist.'.format(old_name))
    answer = input('Would you like to replace it (answer with "r")\nor create new one (answer with "n") ? ')
    while answer not in 'rRnN':
        print('Your answer is inconclusive')
        print('Please answer again:')
        print('if you would like to replece the existing file answer with "r"')
        print('if you would like to create new one answer with "n"')
        answer = input('Would you like to replace it (answer with "r")\n or create new one (answer with "n") ? ')
    if answer in 'nN':
        new_name = input('Enter new name for file: ')
        if check_if_name_already_exists(new_name, path):
            return get_new_name(new_name, path)
        else:
            return new_name
    if answer in 'rR':
        return old_name



def download_pdf(url, name = 'document1', path = None):
    '''function downloads pdf file from its url
    required argument is url of pdf file and
    optional argument is name for saved pdf file and
    optional argument path if you want to choose where is your file saved
    variable path must look like:
        'C:\\Users\\Computer name\\Desktop' or
        'C:/Users/Computer name/Desktop' '''
    # and not like
    #   'C:\Users\Computer name\Desktop'

    pdf = get_html(url)

    name = name.replace('.pdf', '')
    if path == None:
        path = get_current_path()
    if '\\' in path:
        path = path.replace('\\', '/')
    if path[-1] != '/':
        path += '/'
    if path:
        check = check_if_name_already_exists(name, path)
        if check:
            if name == 'document1':
                i = 2
                name = 'document' + str(i)
                while check_if_name_already_exists(name, path):
                    i += 1
                    name = 'document' + str(i)
            else:
                name = get_new_name(name, path)
        file = open(path+name + '.pdf', 'wb')
    else:
        file = open(name + '.pdf', 'wb')

    file.write(pdf.read())
    file.close()
    if path:
        print(name + '.pdf file downloaded in folder "{}".'.format(path))
    else:
        print(name + '.pdf file downloaded.')
    return


download_url = 'https://fnn.app.box.com/index.php?rm=box_download_shared_file&shared_name=n74mnmrwyrmtiooqwppqjkrd1hhf3t3j&file_id=f_53868474893'
download_pdf(download_url)

希望它有所帮助，让我知道它是否有效。