Question

我是Python SageMaker的新手（我的背景是C＃）。目前，我有一个问题，因为最后一个方法调用（我的意思是拟合方法）导致“NoCredentialsError”。我不明白这个。 AWS凭证已设置，我确实使用它们与AWS通信，例如与S3通信。我该如何防止此错误？

import io
import os
import gzip
import pickle
import urllib.request
import boto3
import sagemaker
import sagemaker.amazon.common as smac

DOWNLOADED_FILENAME = 'C:/Users/Daan/PycharmProjects/downloads/mnist.pkl.gz'
if not os.path.exists(DOWNLOADED_FILENAME):
    urllib.request.urlretrieve("http://deeplearning.net/data/mnist/mnist.pkl.gz", DOWNLOADED_FILENAME)

with gzip.open(DOWNLOADED_FILENAME, 'rb') as f:
    train_set, valid_set, test_set = pickle.load(f, encoding='latin1')
vectors = train_set[0].T
buf = io.BytesIO()
smac.write_numpy_to_dense_tensor(buf, vectors)
buf.seek(0)
key = 'recordio-pb-data'
bucket_name = 'SOMEKINDOFBUCKETNAME'
prefix = 'sagemaker/pca'
path = os.path.join(prefix, 'train', key)
print(path)

session = boto3.session.Session(aws_access_key_id='SECRET',aws_secret_access_key='SECRET',region_name='eu-west-1')
client = boto3.client('sagemaker',region_name='eu-west-1',aws_access_key_id='SECRET',aws_secret_access_key='SECRET')
region='eu-west-1'
sagemakerSession= sagemaker.Session(sagemaker_client=client,boto_session=session)
s3_resource=session.resource('s3')
bucket = s3_resource.Bucket(bucket_name)
current_bucket = bucket.Object(path)

train_data = 's3://{}/{}/train/{}'.format(bucket_name, prefix, key)
print('uploading training data location: {}'.format(train_data))
current_bucket.upload_fileobj(buf)

output_location = 's3://{}/{}/output'.format('SOMEBUCKETNAME', prefix)
print('training artifacts will be uploaded to: {}'.format(output_location))

region='eu-west-1'

containers = {'us-west-2': 'SOMELOCATION',
              'us-east-1': 'SOMELOCATION',
              'us-east-2': 'SOMELOCATION',
              'eu-west-1': 'SOMELOCATION'}
container = containers[region]

role='AmazonSageMaker-ExecutionRole-SOMEVALUE'
pca = sagemaker.estimator.Estimator(container,
                                    role,
                                    train_instance_count=1,
                                    train_instance_type='ml.c4.xlarge',
                                    output_path=output_location,
                                    sagemaker_session=sagemakerSession)


pca.set_hyperparameters(feature_dim=50000,
                        num_components=10,
                        subtract_mean=True,
                        algorithm_mode='randomized',
                        mini_batch_size=200)

pca.fit(inputs=train_data)

print('END')

Answer 1

我不确定您是否屏蔽了实际的访问ID和密钥，或者这就是您正在运行的内容。

session = boto3.session.Session(aws_access_key_id='SECRET',aws_secret_access_key='SECRET',region_name='eu-west-1')
client = boto3.client('sagemaker',region_name='eu-west-1',aws_access_key_id='SECRET',aws_secret_access_key='SECRET')

我希望您在上面的代码行中提供实际的aws_access_key_id和aws_secret_access_key。

在代码中指定相同而非硬编码的另一种方法是在配置文件目录中创建凭证文件，即

在Mac~ / .aws /

中

并在Windows "%UserProfile%\.aws"

中

该文件是纯文本文件，名称为“凭据”（不带引号）。文件包含

[default]
aws_access_key_id=XXXXXXXXXXXXXX
aws_secret_access_key=YYYYYYYYYYYYYYYYYYYYYYYYYYY

AWS CLI将从上述位置获取并使用。

希望这会有所帮助。

Answer 2

我总是解决这个问题

os.environ["AWS_PROFILE"] = "profile-name"

我觉得它并不漂亮，很高兴听到另一个建议。不知何故，我不确定我们如何将凭据传递给 sagemaker 包。

如何在SageMaker中调用fit方法时阻止NoCredentialsError？

2 个答案: