AWS CodeBuild本地缓存无法实际缓存?

时间:2019-11-10 22:59:30

标签: aws-codebuild

我一直在尝试使AWS CodeBuild的本地缓存正常工作,并且我一生都无法获得最基本的缓存。我的最终目标是缓存Gradle工件,如here所述。

但是由于无法正常工作,我尝试了一个更简单的测试,在该测试中,我尝试使用文件>>> addToInventory(inventory, dragonLoot) >>> displayInventory(inventory) Inventory: 45 gold coin 1 rope 1 dagger 1 ruby Total of number of items: 48 来缓存目录/root/foo,并在每次构建时增加该文件。我的期望是,如果我彼此之间在几分钟之内运行后续的构建,我将在日志中看到“ 2”,“ 3”等。但是现实是,尽管建立了符号链接,但是下一个构建版本却看不到前一个counter.txt文件,这对我来说表明有些问题。

谁能确认他们的本地缓存实际上在CodeBuild中工作?我开始怀疑该功能当前是否损坏!还是我完全误解了该怎么办?

buildspec.yml:

counter.txt

cache-test.sh:

version: 0.2

phases:
  install:
    runtime-versions:
      java: corretto8
  build:
    commands:
      - pwd
      - ls -l /root/
      - ls -l /root/foo/
      - ./cache-test.sh
      - ls -l /root/
      - ls -l /root/foo/

cache:
  paths:
    - '/root/foo/*'

CodeBuild输出 :(即使快速连续执行,输出也相同)

#!/bin/bash
if [ -d "/root/foo" ]; then
  C=$(cat /root/foo/count.txt)
  C=$((C + 1))
  echo "*********************************"
  echo "*********************************"
  echo "Incrementing counter to $C"
  echo $C > /root/foo/count.txt
  echo "*********************************"
  echo "*********************************"
else
  mkdir /root/foo
  echo "*********************************"
  echo "*********************************"
  echo "File not found, starting count at 1"
  echo "*********************************"
  echo "*********************************"
  echo 1 > /root/foo/count.txt
fi

CodeBuild项目JSON:

[Container] 2019/11/10 22:35:08 Waiting for agent ping 
[Container] 2019/11/10 22:35:10 Waiting for DOWNLOAD_SOURCE 
[Container] 2019/11/10 22:35:10 Phase is DOWNLOAD_SOURCE 
[Container] 2019/11/10 22:35:10 CODEBUILD_SRC_DIR=/codebuild/output/src905503483/src 
[Container] 2019/11/10 22:35:10 YAML location is /codebuild/output/src905503483/src/buildspec.yml 
[Container] 2019/11/10 22:35:10 No commands found for phase name: INSTALL 
[Container] 2019/11/10 22:35:10 Processing environment variables 
[Container] 2019/11/10 22:35:10 Moving to directory /codebuild/output/src905503483/src 
[Container] 2019/11/10 22:35:10 MkdirAll: /codebuild/local-cache/custom/de68c9f22ae028d4e4dfb0d11bbb481053d28b1373db0d6a56ebee0416bf13b2/root/foo 
[Container] 2019/11/10 22:35:10 Symlinking: /root/foo => /codebuild/local-cache/custom/de68c9f22ae028d4e4dfb0d11bbb481053d28b1373db0d6a56ebee0416bf13b2/root/foo 
[Container] 2019/11/10 22:35:10 Registering with agent 
[Container] 2019/11/10 22:35:10 Phases found in YAML: 2 
[Container] 2019/11/10 22:35:10  BUILD: 6 commands 
[Container] 2019/11/10 22:35:10  INSTALL: 0 commands 
[Container] 2019/11/10 22:35:10 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED 
[Container] 2019/11/10 22:35:10 Phase context status code:  Message:  
[Container] 2019/11/10 22:35:11 Entering phase INSTALL 
[Container] 2019/11/10 22:35:11 Running command echo "Installing corretto(OpenJDK) version 8 ..." 
Installing corretto(OpenJDK) version 8 ... 

[Container] 2019/11/10 22:35:11 Running command export JAVA_HOME="$JAVA_8_HOME" 

[Container] 2019/11/10 22:35:11 Running command export JRE_HOME="$JRE_8_HOME" 

[Container] 2019/11/10 22:35:11 Running command export JDK_HOME="$JDK_8_HOME" 

[Container] 2019/11/10 22:35:11 Running command for tool_path in "$JAVA_8_HOME"/bin/* "$JRE_8_HOME"/bin/*; 
 do tool=`basename "$tool_path"`; 
  if [ $tool != 'java-rmi.cgi' ]; 
  then 
   rm -f /usr/bin/$tool /var/lib/alternatives/$tool \ 
    && update-alternatives --install /usr/bin/$tool $tool $tool_path 20000; 
  fi; 
done 

[Container] 2019/11/10 22:35:11 Phase complete: INSTALL State: SUCCEEDED 
[Container] 2019/11/10 22:35:11 Phase context status code:  Message:  
[Container] 2019/11/10 22:35:11 Entering phase PRE_BUILD 
[Container] 2019/11/10 22:35:11 Phase complete: PRE_BUILD State: SUCCEEDED 
[Container] 2019/11/10 22:35:11 Phase context status code:  Message:  
[Container] 2019/11/10 22:35:11 Entering phase BUILD 
[Container] 2019/11/10 22:35:11 Running command pwd 
/codebuild/output/src905503483/src 

[Container] 2019/11/10 22:35:11 Running command ls -l /root/ 
total 4 
lrwxrwxrwx 1 root root 103 Nov 10 22:35 foo -> /codebuild/local-cache/custom/de68c9f22ae028d4e4dfb0d11bbb481053d28b1373db0d6a56ebee0416bf13b2/root/foo 

[Container] 2019/11/10 22:35:11 Running command ls -l /root/foo/ 
total 0 

[Container] 2019/11/10 22:35:11 Running command ./cache-test.sh 
cat: /root/foo/count.txt: No such file or directory 
********************************* 
********************************* 
Incrementing counter to 1 
********************************* 
********************************* 

[Container] 2019/11/10 22:35:11 Running command ls -l /root/ 
total 4 
lrwxrwxrwx 1 root root 103 Nov 10 22:35 foo -> /codebuild/local-cache/custom/de68c9f22ae028d4e4dfb0d11bbb481053d28b1373db0d6a56ebee0416bf13b2/root/foo 

[Container] 2019/11/10 22:35:11 Running command ls -l /root/foo/ 
total 4 
-rw-r--r-- 1 root root 2 Nov 10 22:35 count.txt 

[Container] 2019/11/10 22:35:11 Phase complete: BUILD State: SUCCEEDED 
[Container] 2019/11/10 22:35:11 Phase context status code:  Message:  
[Container] 2019/11/10 22:35:11 Entering phase POST_BUILD 
[Container] 2019/11/10 22:35:11 Phase complete: POST_BUILD State: SUCCEEDED 
[Container] 2019/11/10 22:35:11 Phase context status code:  Message:  

2 个答案:

答案 0 :(得分:6)

文档尚不清楚,但是AWS CodeBuild Local缓存可以only cache directories(在撰写本文时)。这有点令人困惑,因为AWS CodeBuild buildspec specification allows the path to be individual files or a wildcard,但实际上指定文件会产生错误。

Unable to initialize cache download: only directories can be cached locally: ...

在您的示例中,您使用将缓存指定为

cache:
  paths:
    - '/root/foo/*'

其中*表示foo内部的所有单个文件和文件夹,但仅缓存文件夹。

指定整个目录应该起作用

cache:
  paths:
    - /root/foo/

答案 1 :(得分:2)

我一直在尝试使缓存自身能够成功运行,

不是来自任何公开来源,而是一些观察结果:

  • 仅当构建时间超过5分钟时,缓存才可用。

  • 如果新版本成功放置在同一版本主机上,则可以使用缓存。

  • 如果新构建在上一次构建的5至15分钟内运行,则可以使用
  • 缓存。缓存可能会根据上一次构建时间保持可用状态,最长为15分钟。

  • 尽管构建时间超过5分钟,但由于将构建放置在不同的构建主机上,缓存可能无法始终正常工作。

  • 此外,在缓存将新版本加速到5分钟以下的情况下,该版本将不会被缓存,从而导致后续未命中。

尽管我相信CodeBuild工程师有充分的理由进行这种设计,但我认为上述限制使得该本地缓存功能的使用受到限制。