Grisome构建在本地成功,但在AWS Amplify上失败

时间:2020-04-23 14:53:40

标签: aws-amplify aws-codebuild gridsome enomem

我有一个要尝试部署到AWS Amplify的Gridsome网站。

该站点可以正常运行,并且在gridsome build上本地没有错误,但是在AWS Amplify上,它失败了。

最初,我认为该错误与我正在使用必需的<ClientOnly>标签的非SSR组件有关,因为这在我的本地版本中引发了警告。但是,此后我删除了该组件,问题仍然存在。

我还认为该问题可能是由于与Webpack相关的软件包需要一些过时的依赖关系,但是经过数小时的尝试来解决这些依赖关系后,我相对相信情况并非如此。

我不太了解错误日志,但是我认为问题可能与我网站上的大量图像有关,并且AWS Amplify在处理它们时内存不足(ENOMEM)不足。我什至找不到原因来增加AWS Amplify的内存分配。

AWS Amplify日志提供以下信息:

2020-04-23T14:21:23.331Z [WARNING]: Gridsome v0.7.14
2020-04-23T14:21:23.867Z [WARNING]: Initializing plugins...
2020-04-23T14:21:24.053Z [WARNING]: Load sources - 0.1s
2020-04-23T14:21:24.166Z [WARNING]: Create GraphQL schema - 0.11s
2020-04-23T14:21:24.200Z [WARNING]: Create pages and templates - 0.03s
2020-04-23T14:21:24.224Z [WARNING]: Generate temporary code - 0.02s
2020-04-23T14:21:24.224Z [WARNING]: Bootstrap finish - 0.89s
2020-04-23T14:21:24.226Z [WARNING]: Compiling assets...
2020-04-23T14:28:43.461Z [WARNING]: Compile assets - 439.23s
2020-04-23T14:28:43.487Z [WARNING]: Execute GraphQL (14 queries) - 0.03s
2020-04-23T14:28:43.492Z [WARNING]: Write out page data (14 files) - 0s
2020-04-23T14:28:45.196Z [WARNING]: Render HTML (14 files) - 1.7s
2020-04-23T14:28:45.198Z [WARNING]: Process files (0 files) - 0s
2020-04-23T14:28:45.563Z [INFO]: [2K
2020-04-23T14:28:45.563Z [INFO]: [1GProcessing images (7675 images) - 0%
2020-04-23T14:28:47.487Z [INFO]: [2K
2020-04-23T14:28:47.489Z [INFO]: [1GProcessing images (7675 images) - 0%
2020-04-23T14:28:47.579Z [INFO]: [2K[1G
2020-04-23T14:28:47.579Z [INFO]: Processing images (7675 images) - 1%
2020-04-23T14:28:48.834Z [INFO]: [2K[1GProcessing images (7675 images) - 1%
...
2020-04-23T14:33:25.168Z [INFO]: [2K[1GProcessing images (7675 images) - 41%
2020-04-23T14:33:26.901Z [WARNING]: internal/child_process.js:366
                                    throw errnoException(err, 'spawn');
                                    ^
                                    Error: spawn ENOMEM
                                    at ChildProcess.spawn (internal/child_process.js:366:11)
                                    at spawn (child_process.js:551:9)
                                    at Object.fork (child_process.js:113:10)
                                    at ChildProcessWorker.initialize (/codebuild/output/src926123643/src/website/node_modules/jest-worker/build/workers/ChildProcessWorker.js:105:44)
                                    at ChildProcessWorker.onExit (/codebuild/output/src926123643/src/website/node_modules/jest-worker/build/workers/ChildProcessWorker.js:204:12)
                                    at ChildProcess.emit (events.js:198:13)
                                    at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
2020-04-23T14:33:27.254Z [WARNING]: npm
2020-04-23T14:33:27.254Z [WARNING]: ERR! code ELIFECYCLE
                                    npm ERR! errno 1
2020-04-23T14:33:27.255Z [WARNING]: npm
2020-04-23T14:33:27.259Z [WARNING]: ERR! website@ build: `gridsome build`
                                    npm ERR! Exit status 1
                                    npm ERR!
                                    npm ERR! Failed at the website@ build script.
                                    npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2020-04-23T14:33:27.270Z [WARNING]: 
2020-04-23T14:33:27.270Z [WARNING]: npm ERR! A complete log of this run can be found in:
                                    npm ERR!     /root/.npm/_logs/2020-04-23T14_33_27_259Z-debug.log
2020-04-23T14:33:27.270Z [HELP]: Outputting the npm debug log
                                 0 info it worked if it ends with ok
                                 1 verbose cli [ '/root/.nvm/versions/node/v10.16.0/bin/node',
                                 1 verbose cli   '/root/.nvm/versions/node/v10.16.0/bin/npm',
                                 1 verbose cli   'run',
                                 1 verbose cli   'build' ]
                                 2 info using npm@6.9.0
                                 3 info using node@v10.16.0
                                 4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
                                 5 info lifecycle website@~prebuild: website@
                                 6 info lifecycle website@~build: website@
                                 7 verbose lifecycle website@~build: unsafe-perm in lifecycle true
                                 8 verbose lifecycle website@~build: PATH: /root/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/codebuild/output/src926123643/src/website/node_modules/.bin:/usr/local/rvm/gems/ruby-2.4.6/bin:/usr/local/rvm/gems/ruby-2.4.6@global/bin:/usr/local/rvm/rubies/ruby-2.4.6/bin:/usr/local/rvm/bin:/root/.yarn/bin:/root/.config/yarn/global/node_modules/.bin:/root/.nvm/versions/node/v10.16.0/bin:/root/.local/bin:/usr/local/rvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                                 9 verbose lifecycle website@~build: CWD: /codebuild/output/src926123643/src/website
                                 10 silly lifecycle website@~build: Args: [ '-c', 'gridsome build' ]
                                 11 silly lifecycle website@~build: Returned: code: 1  signal: null
                                 12 info lifecycle website@~build: Failed to exec build script
                                 13 verbose stack Error: website@ build: `gridsome build`
                                 13 verbose stack Exit status 1
                                 13 verbose stack     at EventEmitter.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
                                 13 verbose stack     at EventEmitter.emit (events.js:198:13)
                                 13 verbose stack     at ChildProcess.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
                                 13 verbose stack     at ChildProcess.emit (events.js:198:13)
                                 13 verbose stack     at maybeClose (internal/child_process.js:982:16)
                                 13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
                                 14 verbose pkgid website@
                                 15 verbose cwd /codebuild/output/src926123643/src/website
                                 16 verbose Linux 4.14.165-103.209.amzn1.x86_64
                                 17 verbose argv "/root/.nvm/versions/node/v10.16.0/bin/node" "/root/.nvm/versions/node/v10.16.0/bin/npm" "run" "build"
                                 18 verbose node v10.16.0
                                 19 verbose npm  v6.9.0
                                 20 error code ELIFECYCLE
                                 21 error errno 1
                                 22 error website@ build: `gridsome build`
                                 22 error Exit status 1
                                 23 error Failed at the website@ build script.
                                 23 error This is probably not a problem with npm. There is likely additional logging output above.
                                 24 verbose exit [ 1, true ]
2020-04-23T14:33:27.291Z [ERROR]: !!! Build failed
2020-04-23T14:33:27.292Z [ERROR]: !!! Non-Zero Exit Code detected
2020-04-23T14:33:27.292Z [INFO]: # Starting environment caching...
2020-04-23T14:33:27.296Z [INFO]: # Environment caching completed
Terminating logging...

我的package.json包含以下内容:

{
  "name": "website",
  "private": true,
  "scripts": {
    "build": "gridsome build",
    "develop": "gridsome develop",
    "explore": "gridsome explore"
  },
  "dependencies": {
    "@gridsome/source-filesystem": "^0.6.2",
    "@gridsome/transformer-remark": "^0.5.0",
    "gridsome": "^0.7.14",
    "gridsome-plugin-gtm": "^0.1.1",
    "marked": "^0.8.0",
    "vue-burger-menu": "^2.0.4",
    "vue-easy-lightbox": "^0.11.0",
    "vue-slick-carousel": "^1.0.5"
  },
  "devDependencies": {
    "node-sass": "^4.13.1",
    "normalize.css": "^8.0.1",
    "sass-loader": "^8.0.2",
    "style-resources-loader": "^1.3.3"
  },
  "browserslist": [
    "defaults"
  ]
}

如果能帮助您解决此问题,我将不胜感激。

谢谢。

更新#1

我现在非常确定这是由于在我的网站上通过<g-image>处理了大量图像所致。

我决定尝试使用AWS CodeBuild进行构建,以查看是否有更多信息。它也失败了,但确实给了我一些有用的错误。

当我删除了几乎所有的图像时,构建工作正常。一旦我获得超过300张以不同尺寸显示的图像,事情就会开始失败。错误消息:

Process files (0 files) - 0s
...
[2K·[1GProcessing images (4671 images) - 58%Error: Failed to process image src/assets/media/jobs/1.jpg. spawn ENOMEM

    at pMap.concurrency (/codebuild/output/src132267748/src/github.com/xxxxx/website/node_modules/gridsome/lib/workers/image-processor.js:113:13)

(sharp:31795): GLib-CRITICAL **: 04:50:22.507: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:31795): GLib-CRITICAL **: 04:50:22.516: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:31795): GLib-CRITICAL **: 04:50:22.516: g_hash_table_insert_internal: assertion 'hash_table != NULL' failed

(sharp:31795): GLib-CRITICAL **: 04:50:22.516: g_hash_table_lookup: assertion 'hash_table != NULL' failed

所以,是的,我想这是一个Gridsome错误?

更新#2

node-options=--max_old_space_size=4096文件中添加.npmrc会导致构建挂起并最终超时。

更新#3

批量调整大小的图像,因此最大宽度/高度为2000px(有效最大文件大小为〜2MB)。构建现已成功。

希望有人可以提供解决方案,以便我们可以从源文件进行构建而无需预处理图像。

1 个答案:

答案 0 :(得分:0)

这似乎是一个错误,其中Gridsome的内存不足,无法处理大量大文件。

此变通办法会在Gridsome尝试处理所有图像文件之前对其进行预处理,以减轻每个图像的负担。

使用夏普预处理图像

// image-preprocessor.js
const fs = require("fs");
const path = require("path");
const sharp = require("sharp");

const dirPath = "./src/assets/";
const maxWidth = 2000;
const maxHeight = 2000;

const getAllFiles = function(dirPath, arrayOfFiles) {
  files = fs.readdirSync(dirPath);

  arrayOfFiles = arrayOfFiles || [];

  files.forEach(function(file) {
    if (fs.statSync(`${dirPath}/${file}`).isDirectory()) {
      arrayOfFiles = getAllFiles(`${dirPath}/${file}`, arrayOfFiles);
    } else {
      arrayOfFiles.push(path.join(__dirname, dirPath, "/", file));
    }
  });
  return arrayOfFiles;
};

const result = getAllFiles(dirPath);

result.forEach(async (file) => {
  await sharp(`${file}`)
    .resize(maxWidth, maxHeight, {
      fit: "inside",
    })
    .toBuffer(function(err, buffer) {
      fs.writeFile(`${file}`, buffer, function(err) {
        if (err) {
          console.log(err);
        }
      });
    });
});

将图像预处理器添加到builspec

// buildspec.yml
  ...
  build:
    commands:
      - node image-preprocessor.js
      - npm run build
  ...

请注意,由于每次通过buildpsec构建Gridsome时都会进行预处理,因此源映像在存储库中保持不变(并最终等待对此问题进行适当的修复)。