Question

Python的软件包管理器允许通过安装脚本的dependency_links参数定义非PyPI依赖项。但是也可以以类似的方式定义镜像吗？（所以，相同的包，相同的版本，一切都相同， - 只是在两个不同的存储库中， - 我希望第二个作为后备，以防第一个失败 - 例如主机无法访问，身份验证失败等）。

UPD

我已经尝试在dependency_links中为同一个包定义更多的网址。不幸的是，这并不像我希望的那样有效。如果因为某种原因而被解析为所请求的包/版本的正确匹配的第一个存储库失败，那么整个安装脚本就会失败（即它不会尝试迭代所有其他纠正匹配，直到找到一个好的，或者全部失败）。

Answer 1

查看setuptools个来源，默认代码仅对dependency_links中的URL进行语法验证，但实际文件下载尝试中发生的所有错误都会停止脚本执行。在下面的示例中，将覆盖默认包索引impl，因此除了URL语法验证之外，还尝试下载目标文件。如果失败，则流程切换到下一个URL。

from distutils.errors import DistutilsError
import os
import tempfile

from setuptools import setup
from setuptools.command.easy_install import easy_install
from setuptools.package_index import PackageIndex as PackageIndexOrig
import faker


class PackageIndex(PackageIndexOrig):

    def url_ok(self, url, fatal=False):
        if super().url_ok(url, fatal):
            try:
                tmpfile = os.path.join(tempfile.mkdtemp(prefix='url-ok-check-'), 'file.out')
                self._attempt_download(url, tmpfile)
                return True
            except Exception as ex:
                msg = 'Download error for %s: %s' % (url, ex)
                if fatal:
                    raise DistutilsError(msg)
                self.warn(msg)
        return False


f = faker.Faker()
fake_urls = [f.url() + '#egg=django' for i in range(10)]
urls = [fake_urls + ['https://github.com/django/django/archive/stable/2.0.x.zip#egg=django']

easy_install.create_index = PackageIndex

setup(
    name='spam',
    packages=['spam'],
    install_requires=['django'],
    dependency_links=urls,
)

测试出来：

$ pip install faker
$ python setup.py install --verbose
running install
...
Installed /Users/hoefling/.virtualenvs/stackoverflow/lib/python3.6/site-packages/spam-0.0.0-py3.6.egg
Downloading https://harris.com#egg=django
Download error for https://harris.com#egg=django: Unexpected HTML page found at https://harris.com#egg=django
Downloading https://www.torres.com#egg=django
Download error for https://www.torres.com#egg=django: Download error for https://www.torres.com#egg=django: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:777)
Downloading https://smith.com/#egg=django
Download error for https://smith.com/#egg=django: Download error for https://smith.com/#egg=django: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
Downloading http://tucker.info/#egg=django
Download error for http://tucker.info/#egg=django: Unexpected HTML page found at http://tucker.info/#egg=django
Downloading https://christian-murphy.org/#egg=django
Download error for https://christian-murphy.org/#egg=django: Download error for https://christian-murphy.org/#egg=django: [Errno 8] nodename nor servname provided, or not known
Downloading http://www.ramirez.com/#egg=django
Download error for http://www.ramirez.com/#egg=django: Unexpected HTML page found at http://www.ramirez.com/#egg=django
Downloading http://www.perez-davis.com/#egg=django
Download error for http://www.perez-davis.com/#egg=django: Download error for http://www.perez-davis.com/#egg=django: [Errno 8] nodename nor servname provided, or not known
Downloading https://ramirez.org/#egg=django
Download error for https://ramirez.org/#egg=django: Download error for https://ramirez.org/#egg=django: [Errno 8] nodename nor servname provided, or not known
Downloading https://www.bridges.com/#egg=django
Download error for https://www.bridges.com/#egg=django: Download error for https://www.bridges.com/#egg=django: [Errno 61] Connection refused
Downloading https://bryant.org/#egg=django
Download error for https://bryant.org/#egg=django: Download error for https://bryant.org/#egg=django: [Errno 61] Connection refused
Downloading http://porter-griffith.com/#egg=django
Download error for http://porter-griffith.com/#egg=django: Download error for http://porter-griffith.com/#egg=django: [Errno 8] nodename nor servname provided, or not known
Downloading https://www.hooper.net/#egg=django
Download error for https://www.hooper.net/#egg=django: Download error for https://www.hooper.net/#egg=django: [Errno 61] Connection refused
Downloading https://github.com/django/django/archive/stable/2.0.x.zip#egg=django
Processing dependencies for spam==0.0.0
Searching for django
Best match: django [unknown version]
Downloading https://github.com/django/django/archive/stable/2.0.x.zip#egg=django
Downloading https://github.com/django/django/archive/stable/2.0.x.zip#egg=django
Processing 2.0.x.zip
...

重写PackageIndex的缺点是，一旦找到有效链接，脚本就会下载两次分发 - 首先验证链接，然后实际下载以进行安装。通过编写比仅使用_attempt_download()方法更精细的检查，可以避免这种情况。或者您可以存储对下载文件的引用，然后在_download_url()方法中重复使用它：

class PackageIndex(PackageIndexOrig):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._attempted = None

    def url_ok(self, url, fatal=False):
        if super().url_ok(url, fatal):
            try:
                tmpfile = os.path.join(tempfile.mkdtemp(prefix='url-ok-check-'), 'file.out')
                self._attempted = self._attempt_download(url, tmpfile)
                return True
            except Exception as ex:
                msg = 'Download error for %s: %s' % (url, ex)
                if fatal:
                    raise DistutilsError(msg)
                self.warn(msg)
        return False

    def _download_url(self, scheme, url, tmpdir):
        if self._attempted is not None:
            (result, self._attempted) = (self._attempted, None)
            return result
        return super()._download_url(scheme, url, tmpdir)

在python包

1 个答案: