Question

我想将对象下方追加到流

中的每个对象

{"index":{"_index":"tvseries","_type":"internindex"}}

我的信息流看起来像这样

[
  {"showname":"The X Files","episode":"04","content":"Before what?","season":"1"},
  {"showname":"The X Files","episode":"04","content":"Before what?","season":"1"},
  {"showname":"The X Files","episode":"01","content":"What?","season":"1"}
]

我的视频流应该是什么样的！

> -> POST http://localhost:9200/_bulk   {"index":{"_index":"tvseries","_type":"internindex"}}  
> {"showname":"The X Files","episode":"04","content":"Before
> what?","season":"1"}  
> {"index":{"_index":"tvseries","_type":"internindex"}}  
> {"showname":"The X
> Files","episode":"04","content":"Great.","season":"1"}  
> {"index":{"_index":"tvseries","_type":"internindex"}}  
> {"showname":"The X
> Files","episode":"01","content":"What?","season":"1"}

如何在我现有的代码库

中使用jsonstream实现这一目标

var stream = new ElasticsearchWritableStream(client, {
  highWaterMark: 256,
  flushTimeout: 500
});

pg.connect(connectionString,function(err, client, done) {
  if(err) throw err;
  var query = new QueryStream('SELECT * FROM srt limit 2')
  var streams = client.query(query)

  //release the client when the stream is finished
  streams.on('end', done)
  streams.pipe(JSONStream.stringify()).pipe(stream)
})

目前正在使用的npm包

对于elasticsearch中的批量插入！

elasticsearch-writable-stream

从postgres获取数据到流！

pg-query-stream

缺少的部分是将postgres流转换为弹性可写流！任何建议，指针，如何实现这一点的建议！

Answer 1

基本上，没有太多代码更改的唯一可行选项是从postgres本身而不是在node.js对象中批量插入弹性搜索所需的构建格式！

get_sells

Nodejs对弹性搜索批量进行流修改

1 个答案: