如何为编译和测试配置保证相同的库版本

时间:2019-04-15 09:56:50

标签: unit-testing dependencies sbt dependency-management

我有一个中型的 sbt 项目(〜20个子项目,〜50个直接依赖项,〜400个传递依赖项)。我正在准备项目以启用单元测试[0]。我遇到了奇怪的问题,注意到CompileTest目标之间用于某些依赖项的确切版本有所不同,即单元测试(sbt test)使用的库版本与直接访问项目(sbt run)。

我在sbt Compile/dependencyList [1]和sbt Test/dependencyList的输出之间产生了差异,并看到了大约50个具有不同版本的依赖项。我知道我可以使用dependencyOverride设置,但是手动处理数百个依赖关系似乎不切实际。每当我们想更新直接依赖项时,我也必须更新该列表。

处理这种情况的预期方法是什么?即如何确保在单元测试期间使用的依赖版本与在生产环境中运行时使用的依赖版本相同?

[0]:迟到总比没有好! :)

[1]:dependencyList是来自sbt-dependency-graph的命令。

2 个答案:

答案 0 :(得分:1)

首先考虑确认依赖地狱是否确实是导致此问题的原因。我们可以这样做的一种方法是通过对包含所有运行时依赖项的胖罐进行测试。

向插件添加sbt-assembly,然后定义以下自定义命令:

commands += Command.command("testWithFatJar") { state =>
    "set assembly / test := {}" ::
    "assembly" ::
    "set Test / fullClasspath := Attributed.blank((assembly / assemblyOutputPath).value) :: (Test / fullClasspath).value.toList" ::
    "test" :: state
}

请注意我们如何首先组装胖子罐,然后使用以下命令将其放在测试类路径中

Test / fullClasspath := Attributed.blank((assembly / assemblyOutputPath).value) :: (Test / fullClasspath).value.toList

,使其处于第一位置。在这里,我们利用JVM的classpath ordering属性

  

您指定多个类路径条目的顺序是   重要。 Java解释器将在   目录按它们在类路径变量中出现的顺序

这意味着如果我们在类路径上有相同的类,那么将使用遇到的第一个类。

胖罐的位置由assemblyOutputPath任务提供:

inspect assembly::assemblyOutputPath
[info] Task: java.io.File
[info] Description:
[info]  output path of the fat jar

现在执行testWithFatJar应该运行以下测试类路径:

sbt:how-to-guarantee-same-library-versions-for-compile-and-test-configurations> show Test / fullClasspath
[info] * Attributed(/Users/mario_galic/code/stackoverflow/how-to-guarantee-same-library-versions-for-compile-and-test-configurations/target/scala-2.12/how-to-guarantee-same-library-versions-for-compile-and-test-configurations-assembly-0.1.0-SNAPSHOT.jar)
[info] * Attributed(/Users/mario_galic/code/stackoverflow/how-to-guarantee-same-library-versions-for-compile-and-test-configurations/target/scala-2.12/test-classes)
[info] * Attributed(/Users/mario_galic/code/stackoverflow/how-to-guarantee-same-library-versions-for-compile-and-test-configurations/target/scala-2.12/classes)
[info] * Attributed(/Users/mario_galic/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar)
[info] * Attributed(/Users/mario_galic/.ivy2/cache/org.scalatest/scalatest_2.12/bundles/scalatest_2.12-3.0.5.jar)
[info] * Attributed(/Users/mario_galic/.ivy2/cache/org.scalactic/scalactic_2.12/bundles/scalactic_2.12-3.0.5.jar)
[info] * Attributed(/Users/mario_galic/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.12.8.jar)
[info] * Attributed(/Users/mario_galic/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.0.6.jar)

请注意*-assembly-0.1.0-SNAPSHOT.jar的位置。一旦我们确认依赖是地狱的原因,那么我们就可以开始思考需要进行哪些永久性的构建修改来解决它。

答案 1 :(得分:1)

这是我的脚本,用于检测仅存在于生产配置中但不存在于测试配置中的依赖项(及其版本)。

它依赖于安装了 dependency graph SBT plugin

我将其集成到我们的 CI 中,并在检测到任何仅生产依赖项时使构建失败。

#!/bin/bash                                                                                                                                                                                             

# http://redsymbol.net/articles/unofficial-bash-strict-mode/                                                                                                                                            
set -euo pipefail
IFS=$'\n\t'

echo Getting dependencyList for production config
sbt "-Dsbt.log.noformat=true" dependencyList > dependency-list-production-raw.txt

echo Getting dependencyList for test config
sbt "-Dsbt.log.noformat=true" test:dependencyList > dependency-list-test-raw.txt

clean_up_dependency_list () {
  # non GNU sed version (less readable IMHO)                                                                                                                                                            
  # sed -n 's/[[:space:]]*\[info\][[:space:]]*\([^:]\{1,\}:[^:]\{1,\}:[^:]\{1,\}\)\s*$/\1/p' | sort | uniq                                                                                              
  sed --regexp-extended -n 's/^\s*\[info\]\s*([^:]+:[^:]+:[^:]+)\s*$/\1/p' | sort | uniq
}

clean_up_dependency_list < dependency-list-production-raw.txt > dependency-list-production.txt

clean_up_dependency_list < dependency-list-test-raw.txt > dependency-list-test.txt


echo Dependencies in dependency-list-production.txt: "$(wc -l < dependency-list-production.txt)"
echo Dependencies in dependency-list-test.txt: "$(wc -l < dependency-list-test.txt)"

dependencies_only_in_prod=$(comm -23 dependency-list-production.txt dependency-list-test.txt)

if [ -n "$dependencies_only_in_prod" ]; then
    cat <<EOF                                                                                                                                                                                           
                                                                                                                                                                                                        
    ERROR: Dependencies that are _not present in test_                                                                                                                                                  
    configuration detected!                                                                                                                                                                             
                                                                                                                                                                                                        
    This may signal a problem where a _different_ (older?) version of                                                                                                                                   
    a dependency will be used in production than what would be used in                                                                                                                                  
    production.                                                                                                                                                                                         
                                                                                                                                                                                                        
    Offending dependencies that are present  _only in prod_:                                                                                                                                            
                                                                                                                                                                                                        
$dependencies_only_in_prod                                                                                                                                                                              
                                                                                                                                                                                                        
    Full list of dependencies in test:                                                                                                                                                                  
                                                                                                                                                                                                        
$(cat dependency-list-test.txt)                                                                                                                                                                         
                                                                                                                                                                                                        
    List of dependencies in production:                                                                                                                                                                 
                                                                                                                                                                                                        
$(cat dependency-list-production.txt)                                                                                                                                                                   
                                                                                                                                                                                                        
EOF                                                                                                                                                                                                     

    exit 1
fi