Question

我们创建了一个作业，以使用SnappyData中的采样执行聚合。

生成的记录数-1200万
时间跨度-5分钟
单节点-64GB RAM，32芯Leads
铅记忆配置-memory-size = 16g -heap-size = 8g
服务器内存配置-memory-size = 24g -heap-size = 12g

开始采样时，我们看到以下错误

18/08/20 18：13：52.412 IST执行器任务启动worker-165 错误SnappyExecutor：阶段37.0的任务31.0中的异常（TID 3790） java.lang.RuntimeException：java.sql.SQLException（24000）：无效光标状态-没有当前行。在org.apache.spark.sql.catalyst.expressions.GeneratedClass $ GeneratedIterator.processNext（未知资源）在org.apache.spark.sql.execution.BufferedRowIterator.hasNext（BufferedRowIterator.java:43）
    at org.apache.spark.sql.execution.WholeStageCodegenRDD$$anon$1.hasNext(WholeStageCodegenExec.scala:510)
    at org.apache.spark.sql.execution.StratifiedSamplerCached$$anonfun$sample$1.apply(StratifiedSamplerCached.scala:381)
    at org.apache.spark.sql.execution.StratifiedSamplerCached$$anonfun$sample$1.apply(StratifiedSamplerCached.scala:373)
    at org.apache.spark.sql.collection.GenerateFlatIterator.hasNext(GenerateFlatIterator.scala:41)
    at org.apache.spark.sql.execution.columnar.InMemoryRelation$$anonfun$1$$anon$1.next(InMemoryRelation.scala:106)
    at org.apache.spark.sql.execution.columnar.InMemoryRelation$$anonfun$1$$anon$1.next(InMemoryRelation.scala:98)
    at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:232)
    at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:935)
    at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:926)
    at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:866)

SnappyData聚合错误

0 个答案: