GITLAB CI/CD 使用缓存或工件提高流水线速度?

时间:2021-02-11 15:02:58

标签: caching gitlab devops gitlab-ci

我真的不明白缓存是如何工作的,以及我如何才能真正加快我的 CI/CD。我想让它在不到 2 分钟的时间内运行,但我得到的只是一条 30m/40m/1h 的管道。

我已经尝试了很多缓存技术(全局缓存,每个作业缓存) 我不会在这里写下所有内容(我现在正在这 3 天里工作,所以我尝试了很多东西,也阅读了很多东西......)。

该项目是一个 .NET MVC 项目,通过 NPM 进行了一些静态捆绑。

CI/CD 在 Windows Server 2008 R2 上运行。 没有泊坞窗图像。只是 C:\gitlab_runner 文件夹中的 gitlab-runner,所有工具都本地安装在标记为 deploy-vmdev

的机器上

(项目名为Iom.Site,保存在Iom.Site文件夹内)

我能做的最好的就是这个(但它平均运行 30m,有时它会因为依赖缓存、node_modules 和包不存在而中断。):

stages:
  - install_dependencies
  - build_app
  - deploy

#Execute dependencies only on package-lock.json e packages.config changes
install_dependencies:
  stage: install_dependencies
  only:
    changes:
      - Iom.Site/package-lock.json
      - Iom.Site/packages.config
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - Iom.Site/node_modules
      - packages/
  tags:
    - deploy-vmdev
  script:
    - if(!((nuget sources) -like "*Promoscience*").Length -gt 0) { nuget sources add -Name Promoscience -Source http://nupromo.promoscience.com/nuget };
    - nuget restore
    - cd Iom.Site
    - npm install

#Bundling through gulp npm packages static files (SCSS, JS,..)
#Copying .\site\content inside .\dist folder at root for packaging
build_static_files:
  stage: build_app
  only:
    changes:
      - Iom.Site\scripts\bundle\**\*
      - Iom.Site\scripts\react\**\*
      - Iom.Site\scss\**\*
      - ".gitlab-ci.yml"
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - Iom.Site/node_modules
    policy: pull
  tags:
    - deploy-vmdev
  script:
    - gulp build
    - cd ..
    - Copy-Item Iom.Site\content .\dist\
  artifacts:
    paths:
      - .\dist
    when: on_success
    expire_in: 30 days

#Building .NET App through MSBUILD. Msbuild profile saves the output in .\dist folder
#for packaging the whole app
build_app:
  stage: build_app
  only:
    changes:
      - "**/*.cs"
      - "**/*.cshtml"
      - "**/*.config"
      - ".gitlab-ci.yml"
  tags:
    - deploy-vmdev
  script:
    - msbuild Iom.Site\Iom.Site.csproj /p:Configuration=test /p:PublishProfile=FolderProfile /p:transformConfigFiles=true /p:deployOnBuild=true
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - packages/
    policy: pull
  artifacts:
    paths:
      - .\dist
    when: on_success
    expire_in: 30 days

#Deploying: Downloading artifacts and merging them. 
#Copying the content of dist folder inside the IIS Site
deploy_test:
  stage: deploy
  dependencies:
    - build_static_files
    - build_app
  tags:
    - deploy-vmdev
  only:
    - master
    - dev
  cache: {} 
  variables:
    SiteAppPath: "E:\\websites\\CnrIom2020"
  script:
    - echo $env:SiteAppPath
    - dir .\dist
    - Copy-Item .\dist\* $env:SiteAppPath -Recurse -Force

更新 1:

最后我在 gitlab ci wiki 上找到了这个:

<块引用>

缓存是一种优化,但不能保证总是有效。您需要准备好在需要它们的每个作业中重新生成任何缓存文件。

并以这种方式改变了我的管道:

stages:
  - build_app
  - package
  - deploy

build_static_files:
  stage: build_app
  before_script:
    - cd Iom.Site
    - if(-Not(Test-Path node_modules)){npm install} #Installiamo le dipendenze di npm solo se non presenti
  only:
    changes:
      - Iom.Site\scripts\bundle\**\*
      - Iom.Site\scripts\react\**\*
      - Iom.Site\scss\**\*
      - ".gitlab-ci.yml"
  tags:
    - deploy-vmdev
  cache:
    key: static_${CI_COMMIT_REF_SLUG}
    paths:
      - Iom.Site/node_modules
    policy: pull-push
  script:
    - gulp build
    - cd ..
    - Copy-Item Iom.Site\content .\dist\content -Recurse -Force #Copiamo i file statici in dist
  artifacts:
    paths:
      - .\dist
    when: on_success
    expire_in: 2 hours

build_app:
  stage: build_app
  before_script: #Installiamo le dipendenze di Nuget solo se già non presenti
    - if(-Not(Test-Path packages)){ nuget restore } 
  only:
    changes:
      - "**/*.cs"
      - "**/*.cshtml"
      - "**/*.config"
      - ".gitlab-ci.yml"
  tags:
    - deploy-vmdev
  cache:
    key: app_${CI_COMMIT_REF_SLUG}
    paths:
      - packages
    policy: pull-push
  script:
    - msbuild Iom.Site\Iom.Site.csproj /p:Configuration=test /p:PublishProfile=FolderProfile /p:transformConfigFiles=true /p:deployOnBuild=true
  artifacts:
    paths:
      - .\dist
    when: on_success
    expire_in: 2 hours

package_app:
  stage: package
  dependencies:
    - build_static_files
    - build_app
  tags:
    - deploy-vmdev
  only:
    - master
    - dev
    - release
  cache: { }
  script:
    - dir .\dist
  artifacts:
    paths:
      - .\dist
    expire_in: never

deploy_test:
  stage: deploy
  dependencies:
    - package_app
  tags:
    - deploy-vmdev
  only: #TODO: When: manual
    - master
    - dev
    - release
  cache: {} 
  variables:
    SiteAppPath: "E:\\websites\\CnrIom2020"
  script:
    - echo $env:SiteAppPath
    - dir .\dist
    - Copy-Item .\dist\* $env:SiteAppPath -Recurse -Force

# deploy_release: To define

0 个答案:

没有答案