人工版本: 5.8.4
在Artifactory中,文件通过文件的校验和(SHA1)存储在内部数据库中,并且出于检索目的,SHA-256非常有用(用于验证文件是否完好)。
请先阅读以下内容:https://www.jfrog.com/confluence/display/RTF/Checksum-Based+Storage
假设有2个Jenkins作业,这会创建少量工件/文件(rpm / jar / etc)。就我而言,我将使用一个简单的.txt文件,该文件以MM / DD / YYYY格式存储日期,以及一些其他jobA / B特定的生成结果文件(jar / rpms等)。
Jenkins_jobA>生成jobA.date_mm_dd_yy.txt
Jenkins_jobA>生成jobB.date_mm_dd_yy.txt
Jenkins jobA和jobB每天以给定的运行顺序运行多次。有时作业A首先运行,而有时作业B。
由于两个作业的文件内容大部分相同(每天),因此jobA的.txt文件和jobB.txt文件的SHA-1值将相同,即在Artifactory中,这两个文件将存储在第一个文件中2个基于字符的目录文件夹结构(根据基于校验和的存储机制)。
在Linux中的两个文件上基本运行sha1sum和sha256sum,将返回完全相同的输出。
随着时间的流逝,这些工件(.txt等)从一个存储库升级到另一个存储库(升级过程,即从快照->阶段->版本库),所以我目前用Groovy编写的逻辑是找到位于“虚拟”存储库(按顺序包含一组物理本地存储库)后面的工件的URI :
// Groovy code
import groovy.json.JsonSlurper
import groovy.json.JsonOutput
jsonSlurper = new JsonSlurper()
// The following function will take artifact.SHA_256 as it's parameter to find URI of the artifact
def checkSumBasedSearch(artifactSha) {
virt_repo = "jar-repo" // this virtual may have many physical repos release/stage/snapshot for jar(maven) or it can be a YUM repo for (rpm) or generic repo for (.txt file)
// Note: Virtual repos don't span different repo types (i.e. a virtual repository in Artifactory for "Maven" artifacts (jar/war/etc) can NOT see YUM/PyPi/Generic physical Repos).
// Run aqlCmd on Linux, requires "...", "..", "..." for every distinctive words / characters in the cmd line.
checkSum_URL = artifactoryURL + "/api/search/checksum?sha256="
aqlCmd = ["curl", "-u", username + ":" + password, "${checkSum_URL}" + artifactSha + "&repos=" + virt_repo]
}
def procedure = aqlCmd.execute()
def standardOut = new StringBuilder(), standardErr = new StringBuilder()
procedure.waitForProcessOutput(standardOut, standardErr)
// Fail early
if (! standardErr ) {
println "\n\n-- checkSumBasedSearch() - standardErr exists ---\n" + standardErr +"\n\n-- Exiting with error 12!!!\n\n"
System.exit(12)
}
def obj = jsonSlurper.parseText(standardOut.toString())
def results = obj.results
def uri = results[0].uri // This would work, if a file's sha-1 /256 is always different or sits in different repo at least.
return uri
// to get the URL, I can use:
//aqlCmd = ["curl", "-u", username + ":" + password, "${uri}"]
//def procedure = aqlCmd.execute()
//def obj = jsonSlurper.parseText(standardOut.toString())
//def url = obj.downloadUri
//return url
//aqlCmd = [ "curl", "-u", username + ":" + password, "${url}", "-o", somedirectory + "/" + variableContainingSomeArtifactFilenameThatIWant ]
//
// def procedure = aqlCmd.execute()
//def standardOut = new StringBuilder(), standardErr = new StringBuilder()
//procedure.waitForProcessOutput(standardOut, standardErr)
// Now, I'll get the artifact downloaded in some Directory as some Filename.
}
我担心的是,由于两个文件(即使 不同的名称 -或file-<versioned-timestamp>.txt
)都具有相同的内容,并且每天生成多次,因此可以为jobA或jobB下载特定版本的文件吗?
在Artifactory中,包含相同内容的所有文件的SHA_256属性将相同!! (Artifactory将使用SHA-1高效地存储这些文件以节省空间,新上传的内容只是对用户透明的最小数据库级别的事务。)
问题:
以上逻辑将返回作业A的文件或作业B的.txt文件或任何首先上传文件或最新/访问权限的作业的.txt文件。到LastModified -aka-上次上传时间?
如何在给定的时间戳下下载jobA的.txt文件和jobB的.txt文件?
我在其余的Api通话中是否需要添加更多属性?
如果我只是担心文件内容,那么它是否来自JobA .txt或job的.txt文件都没关系(取决于sha-1 / 256),但是在复杂的情况下,可能文件名,其中包含他们想知道要下载的文件(A / B)的有意义的信息!
答案 0 :(得分:0)
您可以使用AQL(人工查询语言)
https://www.jfrog.com/confluence/display/RTF/Artifactory+Query+Language
curl -u<username>:<password> -XPOST https://repo.jfrog.io/artifactory/api/search/aql -H "Content-Type: text/plain" -T ./search
名为search的文件的内容为:
items.find(
{
"artifact.module.build.name":{"$eq":"<build name>"},
"artifact.sha1":"<sha1>"
}
)
上面的逻辑(在原始问题中)将任意返回其中的一个,因为您要获取返回的第一个结果,并且订单上没有保证。
由于您的文本文件的名称中包含时间戳,因此您可以将名称添加到上述给定的aql中,它也会按名称进行过滤。
AQL搜索API比校验和搜索更灵活,可以使用它并根据所需参数自定义查询。
答案 1 :(得分:0)
所以,我最终还是这样做了,而不仅仅是在每种情况下都从数组中返回了第[0]个元素。
// Do NOT return [0] first element as yet as Artifactory uses SHA-1/256 so return [Nth].uri where artifact's full name matches with the sha256
// def uri = results[0].uri
def nThIndex=0
def foundFlag = 'false'
for (r in results) {
println "> " + r.uri + " < " + r.uri.toString() + " artifact: " + artFullName
if ( r.uri.toString().contains(artFullName) ) {
foundFlag = 'true'
println "- OK - Found artifact: " + artFullName + " at results[" + nThIndex + "] index."
break; // i.e. a match for the artifact name with SHA-256 we want - has been found.
} else {
nThIndex++;
}
}
if ( foundFlag == 'true' ) {
def uri = results[nThIndex].uri
return uri
} else {
// Fail early if results were found based on SHA256 but not for the artifact but for some other filename with same SHA256
if (! standardErr ) {
println "\n\n\n\n-- [Cool] -- checkSum_Search() - SHA-256 unwanted situation occurred !!! -- results Array was set with some values BUT it didn't contain the artifact (" + artFullName + ") that we were looking for \n\n\n-- !!! Artifact NOT FOUND in the results array during checkSum_Search()---\n\n\n-- Exiting with error 17!!!\n\n\n\n"
System.exit(17) // Nooka
}
}