Question

我正在编写一个小脚本来流式下载并处理来自url的多个顺序命名的文件。我正在使用highlandjs，并使其一个接一个地完美工作。我试图将其重构为一个循环，在此循环中，我将等待一个高原流的完成，然后再开始另一个流：

// my attempt to abstract the highland into a promise I can await  
const processFile = async (url, path) => {
      const writeStream = fs.createWriteStream(path);

      return hl(request(url))
        .split()
         // various working transforms
        .map(splitDelim)
        .filter(filterSchedAOnly)
        .map(appendIdentity)
        .filter(filterOnlyIdentified)
        .map(convertToCSVsafeString)
        // end result should write to a file
        .pipe(writeStream)
        .toPromise();

   // also tried this
  // return new Promise((resolve, reject) => {
  //   writeStream.on("end", () => resolve());
  //   writeStream.on("error", () => reject());
  // });
};

(async () => {
  let i = 1;
  // infinite loop
  while (true) {
    const url = `http://.../${i}.txt`;
    const filePath = `${i}.processed.csv`;
    try {
      // does not work!
      await processFile(url, filePath);
    } catch (err) {
      console.log(err);
    }
    i++;
  }
})();

我应该如何包装processFile函数，以便在进行下一次迭代之前可以等待其完成？

Answer 1

这似乎有效：

function streamToPromise(stream) {
  return new Promise(function(resolve, reject) {
    // resolve with location of saved file
    stream.on("finish", () => resolve());
    // stream.on("end", () => resolve());
    stream.on("error", () => reject());
  });
}

const processFile = async (url, path, i) => {

  const stream = hl(request(url))
    .split()
    // ......
    .pipe(fs.createWriteStream(path));

  return streamToPromise(stream);

};

等待高原流

1 个答案: