Python只读文件系统错误使用S3和Lambda打开文件进行读取

时间:2016-09-08 05:56:22

标签: python amazon-s3 lambda io

当我将file.csv放入S3存储桶时,我从lambda函数中看到了以下错误。该文件不大,我甚至在打开文件进行读取之前添加了60秒的睡眠,但由于某种原因,该文件具有额外的" .6CEdFe7C"附加到它。那是为什么?

[Errno 30] Read-only file system: u'/file.csv.6CEdFe7C': IOError
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 75, in lambda_handler
s3.download_file(bucket, key, filepath)
File "/var/runtime/boto3/s3/inject.py", line 104, in download_file
extra_args=ExtraArgs, callback=Callback)
File "/var/runtime/boto3/s3/transfer.py", line 670, in download_file
extra_args, callback)
File "/var/runtime/boto3/s3/transfer.py", line 685, in _download_file
self._get_object(bucket, key, filename, extra_args, callback)
File "/var/runtime/boto3/s3/transfer.py", line 709, in _get_object
extra_args, callback)
File "/var/runtime/boto3/s3/transfer.py", line 723, in _do_get_object
with self._osutil.open(filename, 'wb') as f:
File "/var/runtime/boto3/s3/transfer.py", line 332, in open
return open(filename, mode)
IOError: [Errno 30] Read-only file system: u'/file.csv.6CEdFe7C'

代码:

def lambda_handler(event, context):

    s3_response = {}
    counter = 0
    event_records = event.get("Records", [])

    s3_items = []
    for event_record in event_records:
        if "s3" in event_record:
            bucket = event_record["s3"]["bucket"]["name"]
            key = event_record["s3"]["object"]["key"]
            filepath = '/' + key
            print(bucket)
            print(key)
            print(filepath)
            s3.download_file(bucket, key, filepath)

以上结果是:

mytestbucket
file.csv
/file.csv
[Errno 30] Read-only file system: u'/file.csv.6CEdFe7C'

如果密钥/文件是" file.csv",那么为什么s3.download_file方法会尝试下载" file.csv.6CEdFe7C"?我猜测该函数何时被触发,该文件是file.csv.xxxxx但是当它到达第75行时,该文件被重命名为file.csv?

3 个答案:

答案 0 :(得分:86)

AWS Lambda中只有/** * Relative path to author photo. * If there is no photo loaded it contains default author image path ({@see DEFAULT_AVATAR_PATH}). * * @var string */ public $photo_url; 似乎可写。

因此这可行:

/tmp

答案 1 :(得分:3)

根据http://boto3.readthedocs.io/en/latest/guide/s3-example-download-file.html

该示例显示如何使用云名称的第一个参数和要下载的本地路径的第二个参数。

enter image description here

另一方面,amazaon docsenter image description here

因此,我们有512 MB的创建文件。 这是我在lambda aws中的代码,对我来说就像魅力一样。

<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.6.4/angular.min.js"></script>

<body ng-app="myApp">

<div class="container" ng-controller="candidateCtrl">
<h2>Bootstrap inline Radio Buttons</h2>
<div class="row"> 
<div class="col-lg-4">
  <p>Angular JS Filter by Radio Button</p>
  <form>
    <label class="radio-inline">
      <input type="radio" name="optradio" ng-model="searchText.role" value="Engineer">Engineer
    </label>
    <label class="radio-inline">
      <input type="radio" name="optradio" ng-model="searchText.role" value="Doctor">Doctor
    </label>
    <label class="radio-inline">
      <input type="radio" name="optradio" ng-model="searchText.role" value="Teacher">Teacher
    </label>
  </form>

</div>
</div>

<div class="row"> 
<div class="col-lg-4">
  
  <table class="table table-hover">
    <thead>
      <tr>
        <th>Name</th>
        <th>Profession</th>
        <th>Country</th>
      </tr>
    </thead>
    <tbody>
      <tr ng-repeat="candidate in candidates | filter:searchText:strict">
        <td>{{candidate.name}}</td>
        <td>{{candidate.role}}</td>
        <td>{{candidate.country}}</td>
      </tr>
    </tbody>
  </table>
  
</div>
</div>

</div>

答案 2 :(得分:0)

我注意到当我上传lambda directly as a zip file的代码时,我只能写入/tmp文件夹,但是当从S3上传代码时,我能够写入{{ 1}}。