在Alpine上安装crcmod CRC32C C扩展

时间:2017-05-24 11:21:18

标签: python gcloud gsutil alpine

我正在尝试将我的CI版本迁移到基于阿尔卑斯山的泊坞窗图像。

部分构建是使用gsutil命令从Firebase testlab检索工件:

gsutil -m cp -r -U $BUCKET_DIR* $OUTPUT_DIR

此命令失败,因为它无法对下载的工件执行CRC校验:

CommandException: 
Downloading this composite object requires integrity checking with CRC32c, but your crcmod installation isn't using the module's C extension, so the hash computation will likely throttle download performance. For help installing the extension, please see "gsutil help crcmod".

To download regardless of crcmod performance or to skip slow integrity checks, see the "check_hashes" option in your boto config file.

NOTE: It is strongly recommended that you not disable integrity checks. 
Doing so could allow data corruption to go undetected during uploading/downloading.
CommandException: 1 file/object could not be transferred.

gsutil help crcmod未提供有关如何在Alpine上安装C扩展程序的说明。

我已经尝试安装以下软件包,虽然安装成功但是没有安装C扩展,仍会导致gsutil命令失败并出现同样的错误。

apk add --update --no-cache python py-pip gcc python-dev
pip install -U crcmod

任何线索?

2 个答案:

答案 0 :(得分:1)

非常确定必须有更好的选择,但这是我的解决方法

# Install Alpine Python dependencies
apk add --update --no-cache python python-dev gcc musl-dev 

# Compile CRC32c
# See for more information: https://cloud.google.com/storage/docs/gsutil/addlhelp/CRC32CandInstallingcrcmod
#
curl -L -o crcmod.tar.gz "https://downloads.sourceforge.net/project/crcmod/crcmod/crcmod-1.7/crcmod-1.7.tar.gz"
tar -xzf crcmod.tar.gz
cd crcmod-1.7/
python setup.py install
cd ..

答案 1 :(得分:0)

这看起来像py-crcmod包中的问题。有一些.so文件(_crcfunext.so),但也许它不能用于某些原因?请在https://bugs.alpinelinux.org/projects/alpine/issues上打开一个问题。