使用AWS Sagemaker Notebook的生命周期配置升级ML库时的问题

时间:2018-11-12 09:55:25

标签: python tensorflow keras amazon-sagemaker amazon-machine-learning

我使用以下脚本来自动升级库。

我的脚本(启动笔记本):

#!/bin/bash

set -e

echo 'Before:'
echo $PATH

export PATH=/home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin:/home/ec2-user/anaconda3/bin/:/usr/libexec/gcc/x86_64-amazon-linux/4.8.5:/usr/local/cuda/bin:/usr/local/bin:/opt/aws/bin:/usr/local/mpi/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:$PATH

echo 'After:'
echo $PATH

echo `pwd`

source /home/ec2-user/anaconda3/bin/activate tensorflow_p36

pip install pandas --upgrade

pip install tensorflow-gpu --upgrade

错误:

出现以下错误,我该如何指向keras的正确位置(/ home / ec2-user)而不是 / root

cp: cannot stat ‘/root/.keras/keras_tensorflow.json’: No such file or directory 

完整日志:

Before:
/sbin:/bin:/usr/sbin:/usr/bin
After:
/home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin:/home/ec2-user/anaconda3/bin/:/usr/libexec/gcc/x86_64-amazon-linux/4.8.5:/usr/local/cuda/bin:/usr/local/bin:/opt/aws/bin:/usr/local/mpi/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/sbin:/bin:/usr/sbin:/usr/bin
/home/ec2-user
cp: cannot stat ‘/root/.keras/keras_tensorflow.json’: No such file or directory

没有生命周期配置:

以上脚本中的所有命令均有效。

当我删除具有以下值的生命周期配置时,实际的keras.json文件正在/ home / ec2-user下。

sh-4.2$ cat .keras/keras.json
{
    "backend": "tensorflow"
}

1 个答案:

答案 0 :(得分:2)

在您的情况下,看起来像pip install在virtualenv的外部执行

尝试从以下位置更改:

source /home/ec2-user/anaconda3/bin/activate tensorflow_p36

至:

source /home/ec2-user/anaconda3/bin/activate tensorflow_p36 && pip install pandas tensorflow-gpu --upgrade

并删除多余的行