Question

我正在开发一个允许人们上传图片的网站。它主要是一种学习体验，我用python编写它。它应该能够做的一件事是获取图像所在的URL，然后将其重新上载到自己的服务器。所以这就是问题：

一个人给我的网站一个网址，然后我的服务器开始下载图像。当它正在下载该图像时，会有另一个请求进来。有没有办法让服务器给出响应（就像说服务器很忙），即使它正在做其他事情？有没有更好的方法来解决这个问题？

提前致谢！

Answer 1

我写了一个例子，希望你能找到帮助。它包含一个列出要处理的作业的urls.txt文件和一个使用多个进程下载的xdownloader.py文件。我在运行Python 2.7的Linux机器上测试过它。

<强> xdownloader.py

from multiprocessing import Process
import urllib2

def main():

  try:

    # Read URLs/Destinations from a file
    jobs = []
    with open("urls.txt","r") as ifile:
      for line in ifile:
        jobs.append(line.split(" "))

    # Create a process list to keep track of running processes
    process_list = []
    # Iterate through our jobs list
    for url, save_to in jobs:
      # Create a new process that runs function 'download'
      p = Process(target=download, args=(url, save_to))
      # Save it to a list
      process_list.append(p)
      # Start the process
      p.start()
  except KeyboardInterrupt:
    print("Received keyboard interrupt (ctrl+c). Exiting...")
  finally:
    # Wait for all processes to finish before exiting
    for process in process_list:
       # Wait for this process to finish
       process.join()
    print("All processes finished successfully!")


def download(url, destination):
  # Open a request
  request = urllib2.urlopen(url)
  # Read and save the webpage data to a file
  with open(destination, "w+") as save_file:
    print("Downloading {0}".format(url))
    save_file.write(request.read())

if __name__=="__main__":
  main()

<强> urls.txt

http://google.com google.html
http://yahoo.com yahoo.html
http://news.google.com news.google.html
http://reddit.com reddit.html
http://news.ycombinator.com news.ycombinator.html

正在运行python xdownloader.py：

mike@localhost ~ $ python xdownloader.py 
Downloading http://news.ycombinator.com
Downloading http://reddit.com
Downloading http://news.google.com
Downloading http://google.com
Done downloading http://google.com
Done downloading http://news.ycombinator.com
Done downloading http://reddit.com
Downloading http://yahoo.com
Done downloading http://news.google.com
Done downloading http://yahoo.com

您可以看到作业异步运行...某些作业较早开始，但比其他作业晚。（我正在看你news.google.com！）如果这个例子不符合你的需要，请在评论中告诉我。

服务器忙时返回响应

1 个答案: