Question

我正在设置一个数据流作业，对于此作业，工作人员需要访问私有的bitbucket存储库以安装库来处理数据。为了授予对数据流工作者的访问权限，我设置了一对SSH密钥（公共和私有）。我设法将私钥添加到我的数据流工作者上。尝试通过git + ssh点子安装软件包时，出现错误Host key verification failed。

我试图在数据流工作程序上寻找.ssh/known_hosts文件，但这并不像在常规VM上那样简单。

或者，我通过以下命令自己设置了它，但是效果不佳：

mkdir -p ~/.ssh
chmod 0700 ~/.ssh
ssh-keyscan bitbucket.org > ~/.ssh/known_hosts

我仍然遇到Host key verification failed错误。

针对此问题的另一种建议的解决方案是运行ssh-keygen -R bitbucket.org，但随后出现以下错误： mkstemp: No such file or directory

对于Dataflow Python SDK，您需要使用setup.py打包代码。工人启动时要执行的所有命令都用subprocess.Popen编写。命令列表如下：

CUSTOM_COMMANDS = [
    # decrypt key encrypted key in repository via gcloud kms
    ['gcloud', '-v'],
    ['gcloud', 'kms', 'decrypt', '--location', 'global', '--keyring',
     'bitbucketpackages', '--key', 'package', '--plaintext-file',
     'bb_package_key_decrypted', '--ciphertext-file', 'bb_package_key'],
    ['chmod', '700', 'bb_package_key_decrypted'],
    # install git & ssh
    ['apt-get', 'update'],
    ['apt-get', 'install', '-y', 'openssh-server'],
    ['apt-get', 'install', '-y', 'git'],
    # add bitbucket.org as known host
    ['mkdir', '-p', '~/.ssh'],
    ['chmod', '0700', '~/.ssh'],
    ['ssh-keyscan', 'bitbucket.org', '>', '~/.ssh/known_hosts'],
    # other attempts to fix it
    # ['ssh-keygen', '-R', 'bitbucket.org']
    # pip install
    ['sh', '-c', 'GIT_SSH_COMMAND="ssh -i ./bb_package_key_decrypted" pip install git+ssh://git@bitbucket.org/team/repo.git'],
]

Answer 1

尝试更新ssh-keyscan以写入一些临时路径，然后将已知的主机文件位置作为GIT_SSH_COMMAND的一部分传递。例如，我将您的脚本更新为：

CUSTOM_COMMANDS = [
    # decrypt key encrypted key in repository via gcloud kms
    ['gcloud', '-v'],
    ['gcloud', 'kms', 'decrypt', '--location', 'global', '--keyring',
     'bitbucketpackages', '--key', 'package', '--plaintext-file',
     'bb_package_key_decrypted', '--ciphertext-file', 'bb_package_key'],
    ['chmod', '700', 'bb_package_key_decrypted'],
    # install git & ssh
    ['apt-get', 'update'],
    ['apt-get', 'install', '-y', 'openssh-server'],
    ['apt-get', 'install', '-y', 'git'],
    # add bitbucket.org as known host
    ['mkdir', '-p', '~/.ssh'],
    ['chmod', '0700', '~/.ssh'],
    ['ssh-keyscan', 'bitbucket.org', '>', '/tmp/bit_bucket_known_hosts'],
    # other attempts to fix it
    # ['ssh-keygen', '-R', 'bitbucket.org']
    # pip install
    ['sh', '-c', 'GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/tmp/bit_bucket_known_hosts -i ./bb_package_key_decrypted" pip install git+ssh://git@bitbucket.org/team/repo.git'],
]

在Dataflow Worker上使用SSH密钥提取私有库

1 个答案: