运行单元测试时嵌入式ES实例失败

时间：2015-12-21 22:39:03

标签： scala unit-testing elasticsearch specs2 elastic4s

我目前有以下代码。 以下是使用elastic4s库

设置本地弹性搜索实例 
val essettings = Settings
  .settingsBuilder().put("cluster.name", "elasticsearch")
  .put("path.home", "/tmp/v").build()

ElasticsearchConnection.setInstance(ElasticClient.local(essettings))

val mockESItem1 = Map("title" -> "Worth It",
  "artist" -> "Fifth Harmony",
  "countries" -> Seq("AI", "CA", "IT", "AU", "GB", "IO", "IE", "GI", "US", "SH", "MS", "KY"),
  "thumbnails" -> Seq(Map("default" -> "http://cache.vevo.com/Content/VevoImages/video/WRONGVEVOPICTURE.jpg")),
  "videoId" -> "VEVOID1", 
  "videoType" -> "type", 
  "ytVideoId" -> "YTID1", 
  "features" -> Seq(),
  "duration" -> 230)

这里我创建一个新索引，然后插入上面的mockES项目。

client.execute( create index "videos" shards 1 replicas 5 mappings(
  "video" as (
    "artist" typed StringType,
    "title" typed StringType,
    "countries" typed StringType,
    "thumbnails" typed ObjectType,
    "videoId" typed StringType,
    "videoType" typed StringType,
    "ytVideoId" typed StringType,
    "features" typed StringType,
    "duration" typed IntegerType
    )
  )
)

client.execute(
  bulk(
    index into "videos"/"video" id 1 fields mockESItem1
  )
).await

但是，如果我然后运行查询以在我的任何测试中找到该项目，则类似于以下内容：

es.execute {
  search in "videos" / "video" limit 5 query bool {
    must(
      queryStringQuery("Worth It").field("title"),
      queryStringQuery("Fifth Harmony").field("artist").field("features"),
      matchQuery("videoType","type"),
      matchQuery("countries","US")
    )
  }
}.await

该程序因以下错误而中断：

org.elasticsearch.action.NoShardAvailableActionException: null
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:151) [elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:64) [elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:53) [elasticsearch-2.0.1.jar:2.0.1]
...
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:228) ~[elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:151) ~[elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:64) ~[elasticsearch-2.0.1.jar:2.0.1]
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:53) ~[elasticsearch-2.0.1.jar:2.0.1]

我一直试图找到解决这个问题的解决方案几天，并且找不到任何可以帮助我解决这个问题的东西，所以现在想看看是否有人在SE遇到了类似的问题并找到了解决办法。

更新

我已经重写了我所拥有的内容，因此它是一个更简单的示例，因为之前我使用的代码中仍然有一些类。

import com.sksamuel.elastic4s.ElasticClient
import com.sksamuel.elastic4s.ElasticDsl._
import com.sksamuel.elastic4s.mappings.FieldType.{IntegerType, ObjectType, StringType}
import org.elasticsearch.action.delete.DeleteRequest
import org.elasticsearch.common.settings.Settings
import org.scalatest._
class ESTest extends FlatSpec with Matchers {
  val essettings = Settings
    .settingsBuilder().put("cluster.name", "elasticsearch")
    .put("path.home", "/tmp/v").build()

  val client = ElasticClient.local(essettings)

  val mockESItem1 = Map("title" -> "Worth It",
    "artist" -> "Fifth Harmony",
    "countries" -> Seq("AI", "CA", "IT", "AU", "GB", "IO", "IE", "GI", "US", "SH", "MS", "KY"),
    "thumbnails" -> Seq(Map("default" -> "http://cache.vevo.com/Content/VevoImages/video/WRONGVEVOPICTURE.jpg")),
    "videoId" -> "VEVOID1",
    "videoType" -> "type",
    "ytVideoId" -> "YTID1",
    "features" -> Seq(),
    "duration" -> 230)

  client.execute(create index "videos" shards 1 replicas 5 mappings (
    "video" as(
      "artist" typed StringType,
      "title" typed StringType,
      "countries" typed StringType,
      "thumbnails" typed ObjectType,
      "videoId" typed StringType,
      "videoType" typed StringType,
      "ytVideoId" typed StringType,
      "features" typed StringType,
      "duration" typed IntegerType
      )
    )
  ).await


  val exists = client.execute {
    index exists "videos"
  }.await.isExists

  println(exists)
  println(mockESItem1.get("videoId").get)

  client.execute(
    index into "videos" / "video" id mockESItem1.get("videoId").get fields mockESItem1
  ).await
  println("hi")

  val resp = client.execute {
    search in "videos" / "video" limit 5 query bool {
      must(
        matchQuery("videoType", "type"),
        matchQuery("countries", "US")
      )
    }
  }.await

  println(resp.hits.head.sourceAsString)
  client.client.delete(new DeleteRequest("videos"))
  client.close
}

目前，如果你运行它，它将无法执行批量插入，然后在＆＃34;测试＆＃34;时接收NoShardAvailableActionException。尝试运行搜索。这可能是specs2引起的问题吗？

2 个答案:

答案 0 :(得分：0)

这适用于1.7.4（如果使用2.0+将ImmutableSettings更改为设置）：

[2015-12-22 07:58:43.324 UTC] 8:55 8:55 i: 0    divisor: 3  acc_val: 0  j: 0
[2015-12-22 07:58:43.326 UTC] 8:56 8:55 i: 1    divisor: 3  acc_val: 0  j: 0
[2015-12-22 07:58:43.326 UTC] 8:57 8:55 i: 2    divisor: 3  acc_val: 0  j: 0
[2015-12-22 07:58:43.327 UTC] 8:58 8:56 i: 3    divisor: 3  acc_val: 0  j: 1
[2015-12-22 07:58:43.327 UTC] 8:59 8:56 i: 4    divisor: 3  acc_val: 0  j: 1
[2015-12-22 07:58:43.327 UTC] 9:00 8:56 i: 5    divisor: 3  acc_val: 0  j: 1
[2015-12-22 07:58:43.327 UTC] 9:01 8:57 i: 6    divisor: 3  acc_val: 0  j: 2
[2015-12-22 07:58:43.328 UTC] 9:02 8:57 i: 7    divisor: 3  acc_val: 0  j: 2
[2015-12-22 07:58:43.328 UTC] 9:03 8:57 i: 8    divisor: 3  acc_val: 0  j: 2
[2015-12-22 07:58:43.328 UTC] 9:04 8:58 i: 9    divisor: 3  acc_val: 0  j: 3
[2015-12-22 07:58:43.328 UTC] 9:05 8:58 i: 10   divisor: 3  acc_val: 0  j: 3
[2015-12-22 07:58:43.328 UTC] 9:06 8:58 i: 11   divisor: 3  acc_val: 0  j: 3
[2015-12-22 07:58:43.329 UTC] 9:07 8:59 i: 12   divisor: 3  acc_val: 0  j: 4
[2015-12-22 07:58:43.329 UTC] 9:08 8:59 i: 13   divisor: 3  acc_val: 0  j: 4
[2015-12-22 07:58:43.330 UTC] 9:09 8:59 i: 14   divisor: 3  acc_val: 0  j: 4
[2015-12-22 07:58:43.330 UTC] 9:10 9:00 i: 15   divisor: 3  acc_val: 0  j: 5
[2015-12-22 07:58:43.330 UTC] 9:11 9:00 i: 16   divisor: 2  acc_val: 5  j: 5
[2015-12-22 07:58:43.330 UTC] 9:12 9:01 i: 17   divisor: 2  acc_val: 5  j: 6
[2015-12-22 07:58:43.330 UTC] 9:13 9:01 i: 18   divisor: 2  acc_val: 5  j: 6
[2015-12-22 07:58:43.331 UTC] 9:14 9:02 i: 19   divisor: 2  acc_val: 5  j: 7
[2015-12-22 07:58:43.331 UTC] 9:15 9:02 i: 20   divisor: 2  acc_val: 5  j: 7
[2015-12-22 07:58:43.331 UTC] 9:16 9:03 i: 21   divisor: 2  acc_val: 5  j: 8
[2015-12-22 07:58:43.331 UTC] 9:17 9:03 i: 22   divisor: 2  acc_val: 5  j: 8
[2015-12-22 07:58:43.331 UTC] 9:18 9:04 i: 23   divisor: 2  acc_val: 5  j: 9
[2015-12-22 07:58:43.332 UTC] 9:19 9:04 i: 24   divisor: 2  acc_val: 5  j: 9
[2015-12-22 07:58:43.332 UTC] 9:20 9:05 i: 25   divisor: 2  acc_val: 5  j: 10
[2015-12-22 07:58:43.332 UTC] 9:21 9:06 i: 26   divisor: 1  acc_val: 10 j: 11
[2015-12-22 07:58:43.332 UTC] 9:22 9:07 i: 27   divisor: 1  acc_val: 10 j: 12
[2015-12-22 07:58:43.333 UTC] 9:23 9:08 i: 28   divisor: 1  acc_val: 10 j: 13
[2015-12-22 07:58:43.333 UTC] 9:24 9:09 i: 29   divisor: 1  acc_val: 10 j: 14
[2015-12-22 07:58:43.333 UTC] 9:25 9:10 i: 30   divisor: 1  acc_val: 10 j: 15
[2015-12-22 07:58:43.333 UTC] 9:26 9:11 i: 31   divisor: 1  acc_val: 10 j: 16
[2015-12-22 07:58:43.334 UTC] 9:27 9:12 i: 32   divisor: 1  acc_val: 10 j: 17
[2015-12-22 07:58:43.334 UTC] 9:28 9:13 i: 33   divisor: 1  acc_val: 10 j: 18
[2015-12-22 07:58:43.334 UTC] 9:29 9:14 i: 34   divisor: 1  acc_val: 10 j: 19
[2015-12-22 07:58:43.334 UTC] 9:30 9:15 i: 35   divisor: 1  acc_val: 10 j: 20
[2015-12-22 07:58:43.334 UTC] 9:31 9:17 i: 36   divisor: 1  acc_val: 10 j: 22
[2015-12-22 07:58:43.340 UTC] 9:32 9:19 i: 37   divisor: 1  acc_val: 10 j: 24
[2015-12-22 07:58:43.342 UTC] 9:33 9:21 i: 38   divisor: 1  acc_val: 10 j: 26
[2015-12-22 07:58:43.343 UTC] 9:34 9:23 i: 39   divisor: 1  acc_val: 10 j: 28
[2015-12-22 07:58:43.344 UTC] 9:35 9:25 i: 40   divisor: 1  acc_val: 10 j: 30
[2015-12-22 07:58:43.344 UTC] 9:36 9:27 i: 41   divisor: 1  acc_val: 10 j: 32
[2015-12-22 07:58:43.345 UTC] 9:37 9:29 i: 42   divisor: 1  acc_val: 10 j: 34
[2015-12-22 07:58:43.345 UTC] 9:38 9:31 i: 43   divisor: 1  acc_val: 10 j: 36
[2015-12-22 07:58:43.346 UTC] 9:39 9:33 i: 44   divisor: 1  acc_val: 10 j: 38
[2015-12-22 07:58:43.346 UTC] 9:40 9:35 i: 45   divisor: 1  acc_val: 10 j: 40
[2015-12-22 07:58:43.347 UTC] 9:41 9:37 i: 46   divisor: 1  acc_val: 10 j: 42
[2015-12-22 07:58:43.348 UTC] 9:42 9:39 i: 47   divisor: 1  acc_val: 10 j: 44
[2015-12-22 07:58:43.351 UTC] 9:43 9:41 i: 48   divisor: 1  acc_val: 10 j: 46
[2015-12-22 07:58:43.351 UTC] 9:44 9:43 i: 49   divisor: 1  acc_val: 10 j: 48
[2015-12-22 07:58:43.352 UTC] 9:45 9:45 i: 50   divisor: 1  acc_val: 10 j: 50

由于这是您错过的部分的代码，我只能假设您的错误出现在您未显示的某些代码中（例如object Test extends App { import ElasticDsl._ val essettings = ImmutableSettings .settingsBuilder().put("cluster.name", "elasticsearch") .put("path.home", "/tmp/v").build() val client = ElasticClient.local(essettings) val mockESItem1 = Map("title" -> "Worth It", "artist" -> "Fifth Harmony", "countries" -> Seq("AI", "CA", "IT", "AU", "GB", "IO", "IE", "GI", "US", "SH", "MS", "KY"), "thumbnails" -> Seq(Map("default" -> "http://cache.vevo.com/Content/VevoImages/video/WRONGVEVOPICTURE.jpg")), "videoId" -> "VEVOID1", "videoType" -> "type", "ytVideoId" -> "YTID1", "features" -> Seq(), "duration" -> 230) client.execute(create index "videos" shards 1 replicas 5 mappings ( "video" as( "artist" typed StringType, "title" typed StringType, "countries" typed StringType, "thumbnails" typed ObjectType, "videoId" typed StringType, "videoType" typed StringType, "ytVideoId" typed StringType, "features" typed StringType, "duration" typed IntegerType ) ) ) client.execute( bulk( index into "videos" / "video" id 1 fields mockESItem1 ) ).await val resp = client.execute { search in "videos" / "video" limit 5 query bool { must( matchQuery("videoType", "type"), matchQuery("countries", "US") ) } }.await println(resp.hits.head.sourceAsString) sys.exit(0) } ）。我也稍微调整了你的查询。

答案 1 :(得分：0)

我遇到了类似的问题：当我在内存中的Elasticsearch节点上进行测试时，后续的测试运行失败并出现NoShardAvailableActionException。当我意识到这只发生在测试运行2次或更多次时，我的解决方案是删除旧索引。粗略地说，我做了：

if (testsDoRunInInternalMode()) {
  client.admin().indices().delete(new DeleteIndexRequest(myIndex)).actionGet();
}

也许这有帮助。

相关问题
最新问题