我有3个NUnit测试程序集正在测试为Kentico CMS平台编写的自定义代码。我可以在Visual Studio(使用NUnit适配器)和NUnit控制台运行器中运行这些测试。我现在正在尝试在我的TeamCity构建服务器上运行那些测试。我已经安装了NUnit运行程序,它返回了一些结果,但是我发现并不是所有的测试都在执行。在一种情况下,我的一个装配体报告没有测试夹具,尽管我知道确实存在。
我有一个名为gto.ecommerce.core.tests.dll
的测试程序集。当它在TeamCity服务器上运行时,我得到以下输出(直接从NUnit控制台运行器获得):
NUnit Console Runner 3.8.0
Copyright (c) 2018 Charlie Poole, Rob Prouse
Runtime Environment
OS Version: Microsoft Windows NT 6.3.9600.0
CLR Version: 4.0.30319.42000
Test Files
D:\TeamCity\buildAgent\work\4a231fb0e41e27f5\Tests\gto.ecommerce.core.tests\bin\Build\gto.ecommerce.core.tests.dll
Run Settings
DisposeRunners: True
WorkDirectory: D:\TeamCity\buildAgent\work\4a231fb0e41e27f5\packages\NUnit.ConsoleRunner.3.8.0\tools
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.6
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 8
Test Run Summary
Overall result: Passed
Test Count: 6, Passed: 6, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2019-01-15 10:16:53Z
End time: 2019-01-15 10:16:54Z
Duration: 1.219 seconds
Results (nunit3) saved as TestResult.xml
这些与在执行TeamCity构建步骤时返回的结果完全相同。
但是,这里应该进行更多测试。如果将TeamCity构建的二进制文件复制到本地计算机并运行相同的NUnit控制台运行程序命令,则会得到以下结果:
NUnit Console Runner 3.8.0
Copyright (c) 2018 Charlie Poole, Rob Prouse
Runtime Environment
OS Version: Microsoft Windows NT 10.0.14393.0
CLR Version: 4.0.30319.42000
Test Files
C:\temp\gto-gtoengineering\TeamCity\gto.ecommerce.core.tests\Build\gto.ecommerce.core.tests.dll
Run Settings
DisposeRunners: True
WorkDirectory: Z:\
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.6
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 8
Test Run Summary
Overall result: Passed
Test Count: 33, Passed: 33, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2019-01-15 10:19:31Z
End time: 2019-01-15 10:19:35Z
Duration: 4.494 seconds
Results (nunit3) saved as TestResult.xml
注意我的机器怎么说,总共有33个测试,这是正确的数字。
如果我在我拥有的另一个程序集rwy.common.core.tests
上运行相同的场景,那么这是TeamCity服务器的结果:
NUnit Console Runner 3.8.0
Copyright (c) 2018 Charlie Poole, Rob Prouse
Runtime Environment
OS Version: Microsoft Windows NT 6.3.9600.0
CLR Version: 4.0.30319.42000
Test Files
D:\TeamCity\buildAgent\work\4a231fb0e41e27f5\Tests\rwy.common.core.tests\bin\Build\rwy.common.core.tests.dll
Errors, Failures and Warnings
1) Invalid : D:\TeamCity\buildAgent\work\4a231fb0e41e27f5\Tests\rwy.common.core.tests\bin\Build\rwy.common.core.tests.dl
l
Has no TestFixtures
Run Settings
DisposeRunners: True
WorkDirectory: D:\TeamCity\buildAgent\work\4a231fb0e41e27f5\packages\NUnit.ConsoleRunner.3.8.0\tools
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.6
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 8
Test Run Summary
Overall result: Failed
Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2019-01-15 10:21:16Z
End time: 2019-01-15 10:21:17Z
Duration: 1.046 seconds
Results (nunit3) saved as TestResult.xml
但是再次,我将相同的二进制文件复制到本地计算机上,并得到以下结果:
NUnit Console Runner 3.8.0
Copyright (c) 2018 Charlie Poole, Rob Prouse
Runtime Environment
OS Version: Microsoft Windows NT 10.0.14393.0
CLR Version: 4.0.30319.42000
Test Files
C:\temp\gto-gtoengineering\TeamCity\rwy.common.core.tests\Build\rwy.common.core.tests.dll
Run Settings
DisposeRunners: True
WorkDirectory: Z:\
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.6
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 8
Test Run Summary
Overall result: Passed
Test Count: 20, Passed: 20, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2019-01-15 10:22:55Z
End time: 2019-01-15 10:23:00Z
Duration: 4.242 seconds
Results (nunit3) saved as TestResult.xml
肯定应该有20个测试,但是在TeamCity服务器上运行时,它说找不到任何测试。
那么,有谁知道为什么运行我假定为相同版本的NUnit和控制台运行程序的两台不同的机器会产生截然不同的结果?除了Windows,我看不到任何版本号上的明显差异,但是我认为应该使用相同的.NET框架。
我确实使用了大量的TestCaseSource
和TestFixtureSource
属性来增加参数化测试的测试数量,但是我不确定这是原因-如果它可以在一台机器上运行,则无法看看为什么另一个会有所不同。
答案 0 :(得分:0)
经过大量跟踪,我发现原因不是NUnit或TeamCity,而是我使用Kentico中的库,特别是我使用Kentico的CMS.Tests
库来帮助单元测试我的自定义Kentico代码为explained here。
因此,我已经更新了我的问题,使其更加针对Kentico,并将在下面提供解决此问题的解决方案。
经过一番互联网搜索,我发现NUnit控制台运行程序具有--trace
command line option。通过传入--trace=Verbose
,我可以将跟踪文件写入到控制台运行程序的工作目录集中,然后比较两台计算机。我的本地开发机器显示正确找到了所有装置,但是TeamCity服务器将生成跟踪文件,其输出类似于以下内容:
InternalTrace: Initializing at level Debug
14:41:48.559 Debug [ 5] DefaultTestAssemblyBuilder: Loading D:\TeamCity\buildAgent\work\4a231fb0e41e27f5\Tests\rwy.common.core.tests\bin\Build\rwy.common.core.tests.dll in AppDomain domain-
14:41:48.570 Debug [ 5] DefaultTestAssemblyBuilder: Examining assembly for test fixtures
14:41:48.579 Debug [ 5] DefaultTestAssemblyBuilder: Found 12 classes to examine
14:41:48.691 Error [ 5] DefaultTestAssemblyBuilder: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at System.ModuleHandle.ResolveType(RuntimeModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(CustomAttributeRecord caRecord, MetadataImport scope, Assembly& lastAptcaOkAssembly, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, Object[] attributes, IList derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg)
at System.Reflection.CustomAttribute.IsCustomAttributeDefined(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Int32 attributeCtorToken, Boolean mustBeInheritable)
at System.Reflection.CustomAttribute.IsDefined(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
at NUnit.Framework.Internal.Reflect.GetMethodsWithAttribute(Type fixtureType, Type attributeType, Boolean inherit)
at NUnit.Framework.Internal.TestFixture..ctor(ITypeInfo fixtureType, Object[] arguments)
at NUnit.Framework.Internal.Builders.DefaultSuiteBuilder.BuildFrom(ITypeInfo typeInfo)
at NUnit.Framework.Api.DefaultTestAssemblyBuilder.GetFixtures(Assembly assembly, IList names)
WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
我不了解的部分是对Microsoft.VisualStudio.QualityTools.UnitTestFramework
的引用,因为它与NUnit无关-这是MSTest库,我的项目中根本没有MSTest单元测试。
那是我意识到这是一个Kentico问题的原因-因为被引用来帮助编写fake Info objects and providers的Kentico CMS.Tests.dll
对于NUnit 和 MSTest都适用。
在开发计算机Microsoft.VisualStudio.QualityTools.UnitTestFramework
上安装了Visual Studio,如this question中所述,这意味着每次运行测试都没有问题-可以在系统上找到相关的DLL。但是,除非明确安装,否则它在构建服务器上永远不可用。
为解决这个问题,我遵循了this advice:
Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
复制到我的解决方案中,并将其提交给我的仓库。Copy Local
为真。通过简单地引用此DLL,构建会将其与所有其他依赖项一起复制到bin
文件夹中,然后可以在无法全局使用的环境中使用。