gitlab-ci缓存如何在dockerRunner上工作?什么是/ cache目录?什么是cache_dir?

时间:2019-02-24 19:06:51

标签: gitlab-ci gitlab-ci-runner

  1. gitlab-ci缓存如何在dockerRunner上运行?

  2. 什么是/ cache目录?

  3. 什么是cache_dir?

  4. "paths" in "cache" gitlab-ci.yml匹配的文件存储在哪里以及如何存储?

1 个答案:

答案 0 :(得分:4)

安装到/cache目录的卷是在gitlab-runner安装时自动创建的,并通过cache_dir设置进​​行管理

有关cache_dir的更多信息:

If you modify the /cache storage path, you also need to make sure to mark this
directory as persistent by defining it in volumes = ["/my/cache/"] under the
[runners.docker] section in config.toml.

TLDR

/cache目录与gitlab-ci.yml中的cache配置不同

    作业容器中的
  1. /cache目录是存储缓存文件的位置
  2. 与gitlab-ci.yml中的cache配置匹配的文件在工作结束时被复制到/cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>
  3. 项目“管道”页面中的“清除流道缓存”按钮计划不要将/cache/CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>/cache.zip提取到gitlab-ci.yml中cache配置中指定的目录中(而不是“删除/就像我一开始那样缓存文件夹)

P.S。

在具有gitlab-runner(https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382)的计算机上创建了名为gitlab-runner-cache的容器

(似乎类似)此容器用于创建存储/ cache数据的匿名卷。创建匿名卷后,该容器将停止。

作业容器(意味着您的测试通常在容器中运行)挂载此匿名卷


证明

具有gitlab-ci.yml

image: srghma/docker-nixos-with-git-crypt

cache:
  key: "test00000" # to reset cache - change this key OR clear cache in project settings page
  paths:
    - .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically)

testtest:
  script:
    - nix-env -i tree

    - tree --dirsfirst -L 4 /cache
    - ls -al ./.mycache || true

    - echo "test" > /cache/test
    - mkdir -p ./.mycache
    - echo "test" > ./.mycache/test

    - tree --dirsfirst -L 4 /cache
    - ls -al ./.mycache || true

输出:

  1. 第一次运行
Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 675caa7 feat: cache update
From https://gitlab.com/srghma/myproject
   675caa7..3d1e223  nix        -> origin/nix
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache

0 directories, 0 files
$ ls -al ./.mycache || true
$ echo "test" > /cache/test
ls: ./.mycache: No such file or directory
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
`-- test

0 directories, 1 file
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded
  1. 第二次跑步
Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded
  1. 在项目“管道”页面上单击“清除流道缓存”清除缓存后,
Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-12...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
ls: ./.mycache: No such file or directory
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- myproject
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:45 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:45 ..
-rw-r--r--    1 root     root             5 Feb 24 11:45 test
Creating cache test00000-12...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded