作业因奇怪的消息而失败

时间:2018-02-28 16:06:34

标签: google-cloud-dataflow google-cloud-dataprep

我的工作在消息一开始就失败了:

"@*" and "@N" are reserved sharding specs. Filepattern must not contain any of them.

我已将目标位置更改为默认值(电子邮件地址)以外的其他内容,其中包含@符号,但我仍然可以看到它正在使用我无法编辑的路径中的临时目标。

以前有没有人遇到过这个问题?我有一个只有65k行的文件,我可以预览Data Prep中的所有完整数据,但是当我运行它失败时,这是非常繁琐的,并且如果不能运行那么〜3小时清理排水管。 (我很欣赏它不是为此设计的,但Excel是一个母马,所以它似乎是一个很好的解决方案!)

编辑 - 添加日志:

 2018-03-10 (13:47:34) Value "PTableLoadTransformGCS/Shuffle/GroupByKey/Session" materialized.
 2018-03-10 (13:47:34) Executing operation PTableLoadTransformGCS/SumQuoteAndDelimiterCounts/GroupByKey/Read+PTableLoadTran...
 2018-03-10 (13:47:38) Executing operation PTableLoadTransformGCS/Shuffle/GroupByKey/Close
 2018-03-10 (13:47:38) Executing operation PTableStoreTransformGCS/WriteFiles/GroupUnwritten/Create
 2018-03-10 (13:47:39) Value "PTableStoreTransformGCS/WriteFiles/GroupUnwritten/Session" materialized.
 2018-03-10 (13:47:39) Executing operation PTableLoadTransformGCS/Shuffle/GroupByKey/Read+PTableLoadTransformGCS/Shuffle/Gr...
 2018-03-10 (13:47:39) Executing failure step failure49
 2018-03-10 (13:47:39) Workflow failed. Causes: (c759db2a23a80ea): "@*" and "@N" are reserved sharding specs. Filepattern m...
(c759db2a23a8c5b): Workflow failed. Causes: (c759db2a23a80ea): "@*" and "@N" are reserved sharding specs. Filepattern must not contain any of them. 
 2018-03-10 (13:47:39) Cleaning up.
 2018-03-10 (13:47:39) Starting worker pool teardown.
 2018-03-10 (13:47:39) Stopping worker pool...

和StackDriver警告或更高:

W  ACPI: RSDP 0x00000000000F23A0 000014 (v00 Google) 
W  ACPI: RSDT 0x00000000BFFF3430 000038 (v01 Google GOOGRSDT 00000001 GOOG 00000001) 
W  ACPI: FACP 0x00000000BFFFCF60 0000F4 (v02 Google GOOGFACP 00000001 GOOG 00000001) 
W  ACPI: DSDT 0x00000000BFFF3470 0017B2 (v01 Google GOOGDSDT 00000001 GOOG 00000001) 
W  ACPI: FACS 0x00000000BFFFCF00 000040 
W  ACPI: FACS 0x00000000BFFFCF00 000040 
W  ACPI: SSDT 0x00000000BFFF65F0 00690D (v01 Google GOOGSSDT 00000001 GOOG 00000001) 
W  ACPI: APIC 0x00000000BFFF5D10 00006E (v01 Google GOOGAPIC 00000001 GOOG 00000001) 
W  ACPI: WAET 0x00000000BFFF5CE0 000028 (v01 Google GOOGWAET 00000001 GOOG 00000001) 
W  ACPI: SRAT 0x00000000BFFF4C30 0000B8 (v01 Google GOOGSRAT 00000001 GOOG 00000001) 
W  ACPI: 2 ACPI AML tables successfully acquired and loaded 
W  ACPI: Executed 2 blocks of module-level executable AML code 
W  acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. 
W  ACPI: Enabled 16 GPEs in block 00 to 0F 
W  ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 
W  ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10 
W  i8042: Warning: Keylock active 
W  GPT:Primary header thinks Alt. header is not at the end of the disk. 
W  GPT:41943039 != 524287999 
W  GPT:Alternate GPT header not at the end of the disk. 
W  GPT:41943039 != 524287999 
W  GPT: Use GNU Parted to correct GPT errors. 
W  device-mapper: verity: Argument 0: 'payload=PARTUUID=245B0EEC-6404-8744-AAF2-E8C6BF78D7B2' 
W  device-mapper: verity: Argument 1: 'hashtree=PARTUUID=245B0EEC-6404-8744-AAF2-E8C6BF78D7B2' 
W  device-mapper: verity: Argument 2: 'hashstart=2539520' 
W  device-mapper: verity: Argument 3: 'alg=sha1' 
W  device-mapper: verity: Argument 4: 'root_hexdigest=244007b512ddbf69792d485fdcbc3440531f1264' 
W  device-mapper: verity: Argument 5: 'salt=5bacc0df39d2a60191e9b221ffc962c55e251ead18cf1472bf8d3ed84383765b' 
E  EXT4-fs (dm-0): couldn't mount as ext3 due to feature incompatibilities 
W  [/usr/lib/tmpfiles.d/var.conf:12] Duplicate line for path "/var/run", ignoring. 
W  Could not stat /dev/pstore: No such file or directory 
W  Kernel does not support crash dumping 
W  Could not load the device policy file. 
W  [CLOUDINIT] cc_write_files.py[WARNING]: Undecodable permissions None, assuming 420 
W  [CLOUDINIT] cc_write_files.py[WARNING]: Undecodable permissions None, assuming 420 
W  [CLOUDINIT] cc_write_files.py[WARNING]: Undecodable permissions None, assuming 420 
W  [CLOUDINIT] cc_write_files.py[WARNING]: Undecodable permissions None, assuming 420 
W  [WARNING:persistent_integer.cc(75)] cannot open /var/lib/metrics/version.cycle for reading: No such file or directory 
W  No API client: no api servers specified 
W  Unable to update cni config: No networks found in /etc/cni/net.d 
W  unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused 
W  No api server defined - no events will be sent to API server. 
W  Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth" 
W  Unable to update cni config: No networks found in /etc/cni/net.d 
E  Image garbage collection failed once. Stats initialization may not have completed yet: unable to find data for container / 
W  No api server defined - no node status update will be sent. 
E  Failed to check if disk space is available for the runtime: failed to get fs info for "runtime": unable to find data for container / 
E  Failed to check if disk space is available on the root partition: failed to get fs info for "root": unable to find data for container / 
E  [ContainerManager]: Fail to get rootfs information unable to find data for container / 
W  Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused 
E  Could not find capacity information for resource storage.kubernetes.io/scratch 
W  eviction manager: no observation found for eviction signal allocatableNodeFs.available 
W  Profiling Agent not found. Profiles will not be available from this worker. 
E  debconf: delaying package configuration, since apt-utils is not installed 
W  [WARNING:metrics_daemon.cc(598)] cannot read /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq 
E    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current 
E                                   Dload  Upload   Total   Spent    Left  Speed 
E  
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  3698  100  3698    0     0  64248      0 --:--:-- --:--:-- --:--:-- 64877 

0 个答案:

没有答案