如何使用我已知道其URL地址的Python在本地保存图像?

时间:2011-11-27 14:46:38

标签: python web-scraping

我知道互联网上图片的网址。

e.g。 http://www.digimouth.com/news/media/2011/09/google-logo.jpg,其中包含Google徽标。

现在,如何使用Python下载此图像,而无需在浏览器中实际打开URL并手动保存文件。

16 个答案:

答案 0 :(得分:250)

Python 2

如果您只想将其保存为文件,这是一种更直接的方式:

import urllib

urllib.urlretrieve("http://www.digimouth.com/news/media/2011/09/google-logo.jpg", "local-filename.jpg")

第二个参数是应保存文件的本地路径。

Python 3

正如SergO建议的,下面的代码应该适用于Python 3。

import urllib.request

urllib.request.urlretrieve("http://www.digimouth.com/news/media/2011/09/google-logo.jpg", "local-filename.jpg")

答案 1 :(得分:26)

import urllib
resource = urllib.urlopen("http://www.digimouth.com/news/media/2011/09/google-logo.jpg")
output = open("file01.jpg","wb")
output.write(resource.read())
output.close()

file01.jpg将包含您的图片。

答案 2 :(得分:15)

我写了a script that does just this,我可以在我的github上找到它供你使用。

我利用BeautifulSoup允许我解析任何网站的图像。如果您要进行大量网页抓取(或打算使用我的工具),我建议您sudo pip install BeautifulSoup。有关BeautifulSoup的信息可用here

为方便起见,这是我的代码:

from bs4 import BeautifulSoup
from urllib2 import urlopen
import urllib

# use this image scraper from the location that 
#you want to save scraped images to

def make_soup(url):
    html = urlopen(url).read()
    return BeautifulSoup(html)

def get_images(url):
    soup = make_soup(url)
    #this makes a list of bs4 element tags
    images = [img for img in soup.findAll('img')]
    print (str(len(images)) + "images found.")
    print 'Downloading images to current working directory.'
    #compile our unicode list of image links
    image_links = [each.get('src') for each in images]
    for each in image_links:
        filename=each.split('/')[-1]
        urllib.urlretrieve(each, filename)
    return image_links

#a standard call looks like this
#get_images('http://www.wookmark.com')

答案 3 :(得分:6)

适用于Python 2和Python 3的解决方案:

try:
    from urllib.request import urlretrieve  # Python 3
except ImportError:
    from urllib import urlretrieve  # Python 2

url = "http://www.digimouth.com/news/media/2011/09/google-logo.jpg"
urlretrieve(url, "local-filename.jpg")

或者,如果requests的额外要求是可接受的,并且是http(s)网址:

def load_requests(source_url, sink_path):
    """
    Load a file from an URL (e.g. http).

    Parameters
    ----------
    source_url : str
        Where to load the file from.
    sink_path : str
        Where the loaded file is stored.
    """
    import requests
    r = requests.get(source_url, stream=True)
    if r.status_code == 200:
        with open(sink_path, 'wb') as f:
            for chunk in r:
                f.write(chunk)

答案 4 :(得分:5)

我制作了一个扩展Yup。脚本的脚本。我修了一些东西。它现在将绕过403:禁止的问题。当图像无法检索时,它不会崩溃。它试图避免损坏的预览。它获得了正确的绝对网址。它提供了更多信息。它可以使用命令行中的参数运行。

# getem.py
# python2 script to download all images in a given url
# use: python getem.py http://url.where.images.are

from bs4 import BeautifulSoup
import urllib2
import shutil
import requests
from urlparse import urljoin
import sys
import time

def make_soup(url):
    req = urllib2.Request(url, headers={'User-Agent' : "Magic Browser"}) 
    html = urllib2.urlopen(req)
    return BeautifulSoup(html, 'html.parser')

def get_images(url):
    soup = make_soup(url)
    images = [img for img in soup.findAll('img')]
    print (str(len(images)) + " images found.")
    print 'Downloading images to current working directory.'
    image_links = [each.get('src') for each in images]
    for each in image_links:
        try:
            filename = each.strip().split('/')[-1].strip()
            src = urljoin(url, each)
            print 'Getting: ' + filename
            response = requests.get(src, stream=True)
            # delay to avoid corrupted previews
            time.sleep(1)
            with open(filename, 'wb') as out_file:
                shutil.copyfileobj(response.raw, out_file)
        except:
            print '  An error occured. Continuing.'
    print 'Done.'

if __name__ == '__main__':
    url = sys.argv[1]
    get_images(url)

答案 5 :(得分:5)

Python 3

urllib.request — Extensible library for opening URLs

from urllib.error import HTTPError
from urllib.request import urlretrieve

try:
    urlretrieve(image_url, image_local_path)
except FileNotFoundError as err:
    print(err)   # something wrong with local path
except HTTPError as err:
    print(err)  # something wrong with url

答案 6 :(得分:3)

这可以通过请求来完成。加载页面并将二进制内容转储到文件中。

import os
import requests

url = 'https://apod.nasa.gov/apod/image/1701/potw1636aN159_HST_2048.jpg'
page = requests.get(url)

f_ext = os.path.splitext(url)[-1]
f_name = 'img{}'.format(f_ext)
with open(f_name, 'wb') as f:
    f.write(page.content)

答案 7 :(得分:3)

使用请求库

import requests
import shutil,os

headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'
}
currentDir = os.getcwd()
path = os.path.join(currentDir,'Images')#saving images to Images folder

def ImageDl(url):
    attempts = 0
    while attempts < 5:#retry 5 times
        try:
            filename = url.split('/')[-1]
            r = requests.get(url,headers=headers,stream=True,timeout=5)
            if r.status_code == 200:
                with open(os.path.join(path,filename),'wb') as f:
                    r.raw.decode_content = True
                    shutil.copyfileobj(r.raw,f)
            print(filename)
            break
        except Exception as e:
            attempts+=1
            print(e)


ImageDl(url)

答案 8 :(得分:1)

这是一个非常简短的答案。

import urllib
urllib.urlretrieve("http://photogallery.sandesh.com/Picture.aspx?AlubumId=422040", "Abc.jpg")

答案 9 :(得分:1)

Python 3版本

我为Python 3调整了@madprops的代码

# getem.py
# python2 script to download all images in a given url
# use: python getem.py http://url.where.images.are

from bs4 import BeautifulSoup
import urllib.request
import shutil
import requests
from urllib.parse import urljoin
import sys
import time

def make_soup(url):
    req = urllib.request.Request(url, headers={'User-Agent' : "Magic Browser"}) 
    html = urllib.request.urlopen(req)
    return BeautifulSoup(html, 'html.parser')

def get_images(url):
    soup = make_soup(url)
    images = [img for img in soup.findAll('img')]
    print (str(len(images)) + " images found.")
    print('Downloading images to current working directory.')
    image_links = [each.get('src') for each in images]
    for each in image_links:
        try:
            filename = each.strip().split('/')[-1].strip()
            src = urljoin(url, each)
            print('Getting: ' + filename)
            response = requests.get(src, stream=True)
            # delay to avoid corrupted previews
            time.sleep(1)
            with open(filename, 'wb') as out_file:
                shutil.copyfileobj(response.raw, out_file)
        except:
            print('  An error occured. Continuing.')
    print('Done.')

if __name__ == '__main__':
    get_images('http://www.wookmark.com')

答案 10 :(得分:1)

使用Requests对于Python 3来说是新鲜的东西

代码中的注释。随时可用的功能。


import requests
from os import path

def get_image(image_url):
    """
    Get image based on url.
    :return: Image name if everything OK, False otherwise
    """
    image_name = path.split(image_url)[1]
    try:
        image = requests.get(image_url)
    except OSError:  # Little too wide, but work OK, no additional imports needed. Catch all conection problems
        return False
    if image.status_code == 200:  # we could have retrieved error page
        base_dir = path.join(path.dirname(path.realpath(__file__)), "images") # Use your own path or "" to use current working directory. Folder must exist.
        with open(path.join(base_dir, image_name), "wb") as f:
            f.write(image.content)
        return image_name

get_image("https://apod.nasddfda.gov/apod/image/2003/S106_Mishra_1947.jpg")

答案 11 :(得分:1)

如果您还没有图片的网址,则可以使用gazpacho进行抓取:

from gazpacho import Soup
base_url = "http://books.toscrape.com"

soup = Soup.get(base_url)
links = [img.attrs["src"] for img in soup.find("img")]

然后按照所述用urllib下载资产:

from pathlib import Path
from urllib.request import urlretrieve as download

directory = "images"
Path(directory).mkdir(exist_ok=True)

link = links[0]
name = link.split("/")[-1]

download(f"{base_url}/{link}", f"{directory}/{name}")

答案 12 :(得分:0)

最新答案,但是对于python>=3.6,您可以使用dload,即:

import dload
dload.save("http://www.digimouth.com/news/media/2011/09/google-logo.jpg")

如果您需要将图像作为bytes,请使用:

img_bytes = dload.bytes("http://www.digimouth.com/news/media/2011/09/google-logo.jpg")

使用pip3 install dload

安装

答案 13 :(得分:0)

使用简单的python wget模块下载链接。用法如下:

import wget
wget.download('http://www.digimouth.com/news/media/2011/09/google-logo.jpg')

答案 14 :(得分:-1)

img_data=requests.get('https://apod.nasa.gov/apod/image/1701/potw1636aN159_HST_2048.jpg')

with open(str('file_name.jpg', 'wb') as handler:
    handler.write(img_data)

答案 15 :(得分:-1)

下载图片文件,避免所有可能的错误:

import requests
import validators
from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError


def is_downloadable(url):
  valid=validators. url(url)
  if valid==False:
    return False
  req = Request(url)
  try:
    response = urlopen(req)
  except HTTPError as e:
    return False
  except URLError as e:
    return False
  else:
    return True



for i in range(len(File_data)):   #File data Contain list of address for image 
                                                      #file
  url = File_data[i][1]
  try:
    if (is_downloadable(url)):
      try:
        r = requests.get(url, allow_redirects=True)
        if url.find('/'):
          fname = url.rsplit('/', 1)[1]
          fname = pth+File_data[i][0]+"$"+fname #Destination to save 
                                                   #image file
          open(fname, 'wb').write(r.content)
      except Exception as e:
        print(e)
  except Exception as e:
    print(e)