发布测试GCP数据存储

时间：2018-11-07 10:41:19

标签： java unit-testing google-cloud-datastore

我有另一个spring boot项目，能够在测试用例中填充数据存储，但是我刚遇到该项目的运行时问题，因此看不到我在做些什么：（

OS: XUbuntu 18.04.1
Java: 8 Updated 191
Spring boot: 2.1.0.RELEASE
GCP bom: 0.69.0-alpha
gcloud cloud-datastore-emulator: 224.0.0

堆栈跟踪为：

com.google.cloud.datastore.DatastoreException: I/O error
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:129) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:114) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.lookup(HttpDatastoreRpc.java:164) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl$3.call(DatastoreImpl.java:384) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl$3.call(DatastoreImpl.java:380) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) ~[gax-1.34.0.jar:1.34.0]
    at com.google.cloud.RetryHelper.run(RetryHelper.java:74) ~[google-cloud-core-1.51.0.jar:1.51.0]
    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:51) ~[google-cloud-core-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl.lookup(DatastoreImpl.java:379) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl$ResultsIterator.loadResults(DatastoreImpl.java:355) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl$ResultsIterator.<init>(DatastoreImpl.java:351) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl.get(DatastoreImpl.java:341) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl.get(DatastoreImpl.java:305) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreHelper.get(DatastoreHelper.java:46) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.google.cloud.datastore.DatastoreImpl.get(DatastoreImpl.java:290) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    at com.mycompany.data.repo.impl.ConstantsRepositoryImpl.setProperty(ConstantsRepositoryImpl.java:51) ~[mycompany-data-repository-0.0.1-SNAPSHOT.jar:na]
    at com.mycompany.merchantrouting.MerchantRoutingApplicationTest.setup(MerchantRoutingApplicationTest.java:60) [test-classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:515) [junit-platform-commons-1.3.1.jar:1.3.1]
    at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115) [junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeMethodInExtensionContext(ClassTestDescriptor.java:436) [junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$14(ClassTestDescriptor.java:424) [junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:136) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:156) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:135) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:110) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:59) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:105) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_191]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_191]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) ~[junit-platform-engine-1.3.1.jar:1.3.1]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170) ~[junit-platform-launcher-1.2.0.jar:1.2.0]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154) ~[junit-platform-launcher-1.2.0.jar:1.2.0]
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90) ~[junit-platform-launcher-1.2.0.jar:1.2.0]
    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:142) ~[surefire-junit-platform-2.22.0.jar:2.22.0]
    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:117) ~[surefire-junit-platform-2.22.0.jar:2.22.0]
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383) ~[surefire-booter-2.22.0.jar:2.22.0]
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344) ~[surefire-booter-2.22.0.jar:2.22.0]
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125) ~[surefire-booter-2.22.0.jar:2.22.0]
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417) ~[surefire-booter-2.22.0.jar:2.22.0]
Caused by: com.google.datastore.v1.client.DatastoreException: I/O error
    at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:226) ~[datastore-v1-proto-client-1.6.0.jar:na]
    at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:195) ~[datastore-v1-proto-client-1.6.0.jar:na]
    at com.google.datastore.v1.client.Datastore.lookup(Datastore.java:95) ~[datastore-v1-proto-client-1.6.0.jar:na]
    at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.lookup(HttpDatastoreRpc.java:162) ~[google-cloud-datastore-1.51.0.jar:1.51.0]
    ... 56 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_191]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_191]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_191]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_191]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_191]
    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_191]
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[na:1.8.0_191]
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~[na:1.8.0_191]
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~[na:1.8.0_191]
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[na:1.8.0_191]
    at sun.net.www.http.HttpClient.New(HttpClient.java:339) ~[na:1.8.0_191]
    at sun.net.www.http.HttpClient.New(HttpClient.java:357) ~[na:1.8.0_191]
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220) ~[na:1.8.0_191]
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156) ~[na:1.8.0_191]
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050) ~[na:1.8.0_191]
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984) ~[na:1.8.0_191]
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334) ~[na:1.8.0_191]
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309) ~[na:1.8.0_191]
    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:77) ~[google-http-client-1.24.1.jar:1.24.1]
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981) ~[google-http-client-1.24.1.jar:1.24.1]
    at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:183) ~[datastore-v1-proto-client-1.6.0.jar:na]
    ... 58 common frames omitted

我对数据存储区进行了以下测试配置：

package com.mycompany.merchantrouting;

import com.mycompany.data.repo.DatastoreRegistry;
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.testing.LocalDatastoreHelper;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

@Configuration
@Profile("test")
public class TestConfiguration {

    @Bean(destroyMethod = "stop")
    public LocalDatastoreHelper dsHelper() throws Exception {
        final LocalDatastoreHelper helper = LocalDatastoreHelper.create();
        helper.start();

        return helper;
    }

    @Bean
    public DatastoreRegistry testDatastore(@Value("${mycompany.namespace}") final String namespace,
            final LocalDatastoreHelper helper) {
        final Datastore ds = helper.getOptions(namespace).getService();

        return (s) -> ds;
    }
}

在调用Datastore.add或Datastore.put时，我看到了堆栈跟踪。同样，我看不到任何明显的不同之处。我正在填充相同的实体“种类”。该端口似乎是在BaseEmulatorHelper中随机分配的，因此它似乎不是端口冲突

编辑2018-11-08

我已打开跟踪日志记录。似乎有一个请求，正在发送响应。

09:40:00.506 [main] DEBUG c.g.datastore.v1.client.RemoteRpc - remote datastore call lookup
09:40:00.507 [main] INFO  c.g.api.client.http.HttpTransport - -------------- REQUEST  --------------
POST http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup
Accept-Encoding: gzip
User-Agent: gcloud-java/1.49.0 Google-HTTP-Java-Client/1.24.1 (gzip)
x-goog-api-format-version: 2
x-goog-api-client: gl-java/1.8.0_191 gccl/1.49.0 gax/1.32.0
Content-Type: application/x-protobuf
Content-Length: 70

09:40:00.507 [main] INFO  c.g.api.client.http.HttpTransport - curl -v --compressed -X POST -H 'Accept-Encoding: gzip' -H 'User-Agent: gcloud-java/1.49.0 Google-HTTP-Java-Client/1.24.1 (gzip)' -H 'x-goog-api-format-version: 2' -H 'x-goog-api-client: gl-java/1.8.0_191 gccl
/1.49.0 gax/1.32.0' -H 'Content-Type: application/x-protobuf' -d '@-' -- 'http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup' << $$$
09:40:00.507 [main] TRACE s.n.w.p.http.HttpURLConnection - ProxySelector Request for http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup
09:40:00.508 [main] TRACE s.n.w.p.http.HttpURLConnection - HttpClient.available(): SocketTimeout: its available
09:40:00.508 [main] TRACE s.n.w.p.http.HttpURLConnection - KeepAlive stream retrieved from the cache, sun.net.www.http.HttpClient(http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup)
09:40:00.508 [main] TRACE s.n.w.p.http.HttpURLConnection - Proxy used: DIRECT
09:40:00.508 [main] DEBUG s.n.w.p.http.HttpURLConnection - sun.net.www.MessageHeader@3f1ed06810 pairs: {POST /v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup HTTP/1.1: null}{Accept-Encoding: gzip}{User-Agent: gcloud-java/1.49.0 Google-HTTP-Java-Clien
t/1.24.1 (gzip)}{x-goog-api-format-version: 2}{x-goog-api-client: gl-java/1.8.0_191 gccl/1.49.0 gax/1.32.0}{Content-Type: application/x-protobuf}{Host: localhost:43639}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}{Content-Length: 
70}
09:40:00.522 [main] INFO  c.g.api.client.http.HttpTransport - Total: 70 bytes
09:40:00.523 [main] INFO  c.g.api.client.http.HttpTransport -  D
3 1test-project-3de2413c-dceb-4506-8121-aeaa75e97486 
 Constants  
09:40:00.524 [main] TRACE s.n.w.p.http.HttpURLConnection - KeepAlive stream used: http://localhost:43639/v1/projects/test-project-3de2413c-dceb-4506-8121-aeaa75e97486:lookup
09:40:00.524 [main] DEBUG s.n.w.p.http.HttpURLConnection - sun.net.www.MessageHeader@753aca853 pairs: {null: HTTP/1.1 200 OK}{content-type: application/x-protobuf}{content-length: 76}
09:40:00.525 [main] INFO  c.g.api.client.http.HttpTransport - -------------- RESPONSE --------------
HTTP/1.1 200 OK
content-length: 76
content-type: application/x-protobuf

09:40:00.526 [main] DEBUG c.g.datastore.v1.client.RemoteRpc - remote datastore call lookup took 20 ms
09:40:00.529 [main] INFO  c.g.api.client.http.HttpTransport - Total: 76 bytes
09:40:00.532 [main] INFO  c.g.api.client.http.HttpTransport -  J
F
D
3 1test-project-3de2413c-dceb-4506-8121-aeaa75e97486 
 Constants

1 个答案:

答案 0 :(得分：0)

好像和Dataflow + Datastore = DatastoreException: I/O error一样。 我假设您正在使用开源的google-cloud-java库？是的，LocalDatastoreHelper有点令人困惑，但这仅用于测试目的。根据{{​​3}}，建议的构造数据存储区对象的方法是：

import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
Datastore datastore = DatastoreOptions.getDefaultInstance().getService();

那么您可以做：

datastore.put(stuff);

如果仍有问题，请回复。

相关问题
最新问题