Question

我有一个shell命令来下载和解压缩从javascript执行的文件。当将卷曲用管道输送到焦油中时，这有时会失败：

const child_process = require('child_process');
var cmd = `curl -s ${file} | tar -xj --strip-components=1 --exclude=README.txt -C ${directory}`;
child_process.exec(cmd, function commandCallback(error, stdout, stderr) {
      console.log('done downloading ' + type + ' bundle');
      if (error) {
        console.error('error downloading ' + type + ' bundle: ' + error);
        console.log(stderr);
      }

有

之类的错误

Error: Command failed: curl -s https://dist.whosonfirst.org/bundles/whosonfirst-data-locality-latest.tar.bz2 | tar -xj --strip-components=1 --exclude=README.txt -C /mnt/storage-proc2/users/jeremy/pelias_metal/data/whosonfirst    
bzip2: Compressed file ends unexpectedly;
        perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
        Input file = (stdin), output file = (stdout)

但是如果命令是这样完成的

var cmd=`wget -q ${file} && tar -xj --strip-components=1 --exclude=README.txt -C ${directory} -f latest.file `;

然后运行正常。有谁知道是什么原因导致第一个命令失败？将管道直接连接到tar可以节省大量时间和磁盘空间，因此是可取的。

如果-s标志没有传递给curl，Node.js显然在长时间下载时会出现缓冲区填充问题，但这似乎不是问题，因为存在-s标志。

编辑-看起来javascript不是罪魁祸首：

jeremyr@w6:~/pelias_metal/whosonfirst$ curl -s https://dist.whosonfirst.org/bundles/whosonfirst-data-postalcode-jp-latest.tar.bz2 | tar -xj --strip-components=1 --exclude=README.txt -C /mnt/data_science/pelias_w6/data/whosonfirst && mv /mnt/data_science/pelias_w6/data/whosonfirst/whosonfirst-data-postalcode-jp-latest.csv /mnt/data_science/pelias_w6/data/whosonfirst/meta 

bzip2: Compressed file ends unexpectedly;
        perhaps it is corrupted?  *Possible* reason follows.

wget -q -O | tar也可以工作-

jeremyr@w6:~/pelias_metal/whosonfirst$ wget -q -O - https://dist.whosonfirst.org/bundles/whosonfirst-data-postalcode-jp-latest.tar.bz2|tar -xj --strip-components=1 --exclude=README.txt -C /mnt/data_science/pelias_w6/data/whosonfirst
jeremyr@w6:~/pelias_metal/whosonfirst$

curl传递给tar的javascript exec命中错误

0 个答案: