gsutil无法上传“codec无法解码字节”错误的文件

时间:2014-06-21 12:14:25

标签: google-cloud-storage gsutil

我正在使用命令

将静态网站上传到云端存储
gsutil cp -v -R -c * gs://www.gennadiy-dolzhenko.ru

中间过程失败并显示以下输出

...SNIP...
Copying file://tourism_studies/models/articles/54-moskva/slideshow/resized/sk576.jpg [Content-Type=image/jpeg]...
Created: gs://www.gennadiy-dolzhenko.ru/tourism_studies/models/articles/54-moskva/slideshow/resized/sk576.jpg#1403352412336000
Copying file://tourism_studies/models/articles/54-moskva/slideshow/resized/7.jpg [Content-Type=image/jpeg]...
Created: gs://www.gennadiy-dolzhenko.ru/tourism_studies/models/articles/54-moskva/slideshow/resized/7.jpg#1403352412725000
'ascii' codec can't decode byte 0xdd in position 1: ordinal not in range(128)

不清楚哪个文件有问题,-c标志不会使工具继续上传,如上所述。如何确定导致错误的原因或至少使工具上传尽可能多的文件?

更新:使用-D开关我能够将其追溯到以下错误

Comparing local vs cloud md5-checksum for tourism_studies/models/articles/54-moskva/slideshow/resized/7.jpg. (O8RgYfpRe8ylUbJh4UPfWg==/O8RgYfpRe8ylUbJh4UPfWg==)
Created: gs://www.gennadiy-dolzhenko.ru/tourism_studies/models/articles/54-moskva/slideshow/resized/7.jpg#1403367381161000
'ascii' codec can't decode byte 0xdd in position 1: ordinal not in range(128)


Encountered exception while copying:
Traceback (most recent call last):
  File "/home/evgeniy/gsutil/gslib/command.py", line 1105, in _SequentialApply
    args = args_iterator.next()
  File "/home/evgeniy/gsutil/gslib/plurality_checkable_iterator.py", line 57, in _PopulateHead
    e = self.base_iterator.next()
  File "/home/evgeniy/gsutil/gslib/name_expansion.py", line 294, in __iter__
    for (names_container, blr) in post_step3_iter:
  File "/home/evgeniy/gsutil/gslib/plurality_checkable_iterator.py", line 57, in _PopulateHead
    e = self.base_iterator.next()
  File "/home/evgeniy/gsutil/gslib/name_expansion.py", line 521, in __iter__
    for (names_container, blr) in self.tuple_iter:
  File "/home/evgeniy/gsutil/gslib/plurality_checkable_iterator.py", line 57, in _PopulateHead
    e = self.base_iterator.next()
  File "/home/evgeniy/gsutil/gslib/name_expansion.py", line 580, in __iter__
    for exp_blr in implicit_subdir_iterator:
  File "/home/evgeniy/gsutil/gslib/plurality_checkable_iterator.py", line 57, in _PopulateHead
    e = self.base_iterator.next()
  File "/home/evgeniy/gsutil/gslib/wildcard_iterator.py", line 560, in IterAll
    for bucket_listing_ref in self.__iter__():
  File "/home/evgeniy/gsutil/gslib/wildcard_iterator.py", line 519, in __iter__
    for filepath in filepaths:
  File "/home/evgeniy/gsutil/gslib/wildcard_iterator.py", line 530, in _IterDir
    for dirpath, unused_dirnames, filenames in os.walk(directory):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 294, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 284, in walk
    if isdir(join(top, name)):
  File "/usr/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xdd in position 1: ordinal not in range(128)

2 个答案:

答案 0 :(得分:1)

这里的问题是包含无效Unicode字符的文件名。我们更改了gsutil,为此案例提供了更有用的错误消息。此更改将包含在gsutil 4.4中。

答案 1 :(得分:0)

我在使用以下命令时遇到问题:

gsutil mv "台北.html" gs://my.bucket.com

甚至

gsutil mv * gs://mybucket.com

赢了工作并且(我想phyton?)总是告诉我:

Failure: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)

但是有效的是,将本地文件重命名为生成的名称并执行以下操作:     gsutil mv" generatedName.html" " GS://my.bucket.com/台北的.html"

我已经读过另一个stackoverflow question它可能会被重新发送?