我将DataFlow应用程序从Eclipse导出到Runnable JAR文件。 当我运行命令Desktop $ java -jar mariadevconn.jar时,我得到以下异常 为什么命令行的当前工作目录已添加到Google存储路径?谢谢你的帮助。
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFO: No stagingLocation provided, falling back to gcpTempLocation
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:52)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:142)
at com.masterconcept.cloud.dataflow.connector.MariaDBConnector.main(MariaDBConnector.java:98)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222)
... 4 more
Caused by: java.lang.IllegalArgumentException: Expected a valid 'gs://' path but was given '/Users/henry/Desktop/gs:/hellodataflowfromhenry/temp/staging/'
at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.getGcsPath(GcsPathValidator.java:101)
at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.verifyPath(GcsPathValidator.java:75)
at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.validateOutputFilePrefixSupported(GcsPathValidator.java:60)
at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:237)
... 9 more
Caused by: java.lang.IllegalArgumentException: Invalid GCS URI: /Users/henry/Desktop/gs:/hellodataflowfromhenry/temp/staging/
at org.apache.beam.sdks.java.extensions.google.cloud.platform.core.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:191)
at org.apache.beam.sdk.util.gcsfs.GcsPath.fromUri(GcsPath.java:116)
at org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.getGcsPath(GcsPathValidator.java:99)
... 12 more
部分代码:
DataflowPipelineOptions options = PipelineOptionsFactory.fromArgs(argv).withValidation().as(DataflowPipelineOptions.class);
options.setGcpTempLocation("gs://hellodataflowfromhenry/temp");
options.setRunner(DataflowRunner.class);
....
的pom.xml:
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>2.2.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.2.0</version>
</dependency>
工作目录:
Desktop$pwd
/Users/henry/Desktop