我正在研究的项目以奇怪的方式使用GIT。基本上它一次写入并推送一个提交。该项目可能导致一个分支拥有数十万次提交。在测试时,我们发现只有大约500次提交后,GIT推送的性能开始下降。在使用过程监视器进行进一步调查后,我们认为降级是由于整个树的行走导致分支被推动。既然我们只是在任何给定时间推送一个新的提交,有没有办法优化它?
或者有没有办法将提交历史限制为50次提交以减少这种开销?
我正在使用LibGit2Sharp版本0.20.1.0
更新1
为了测试,我编写了以下代码:
void Main()
{
string remotePath = @"E:\GIT Test\Remote";
string localPath = @"E:\GIT Test\Local";
string localFilePath = Path.Combine(localPath, "TestFile.txt");
Repository.Init(remotePath, true);
Repository.Clone(remotePath, localPath);
Repository repo = new Repository(localPath);
for(int i = 0; i < 2000; i++)
{
File.WriteAllText(localFilePath, RandomString((i % 2 + 1) * 10));
repo.Stage(localFilePath);
Commit commit = repo.Commit(
string.Format("Commit number: {0}", i),
new Signature("TestAuthor", "TestEmail@Test.com", System.DateTimeOffset.Now),
new Signature("TestAuthor", "TestEmail@Test.com", System.DateTimeOffset.Now));
Stopwatch pushWatch = Stopwatch.StartNew();
Remote defaultRemote = repo.Network.Remotes["origin"];
repo.Network.Push(defaultRemote, "refs/heads/master:refs/heads/master");
pushWatch.Stop();
Trace.WriteLine(string.Format("Push {0} took {1}ms", i, pushWatch.ElapsedMilliseconds));
}
}
private const string Characters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
private static readonly Random Random = new Random();
/// <summary>
/// Get a Random string of the specified length
/// </summary>
public static string RandomString(int size)
{
char[] buffer = new char[size];
for (int i = 0; i < size; i++)
{
buffer[i] = Characters[Random.Next(Characters.Length)];
}
return new string(buffer);
}
然后运行此处找到的进程监视器: http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx
每次推送的时间一般都很低,随着时间的推移,频率和延迟都会增加。在查看进程监视器的输出时,我相信这些尖峰排列在一个很长的伸展位置,正在访问.git \ objects文件夹中的对象。由于某种原因偶尔会出现拉动,因此对象的大量读取在看得更近时似乎是在提交和对象中漫步。
以上流程是我们在项目中实际执行的实际流程的精简版本。在我们的实际流程中,我们首先从“Master”创建一个新的分支“Temp”,提交“Temp”,推送“Temp”,将“Temp”与“Master”合并然后推送“Master”。当我们对该流程的每个部分进行计时时,我们发现推送是迄今为止运行时间最长的操作,随着提交工作堆积在“Master”上,它的使用时间也在增加。
更新2 我最近更新使用libgit2sharp版本0.20.1.0,这个问题仍然存在。有谁知道为什么会这样?
更新3 我们更改了一些代码,以便在“Master”分支上创建第一次提交的临时分支,以减少提交树遍历开销,但发现它仍然存在。下面是一个应该易于编译和运行的示例。它显示了无论提交位置如何,在创建新分支时都会发生树遍历。为了查看树遍历,我使用了上面的进程监视工具和命令行GIT Bash来检查它打开的每个对象是什么。有谁知道为什么会这样?这是预期的行为还是我做错了什么?这似乎是导致问题的推动力。
void Main()
{
string remotePath = @"E:\GIT Test\Remote";
string localPath = @"E:\GIT Test\Local";
string localFilePath = Path.Combine(localPath, "TestFile.txt");
Repository.Init(remotePath, true);
Repository.Clone(remotePath, localPath);
// Setup Initial Commit
string newBranch;
using (Repository repo = new Repository(localPath))
{
CommitRandomFile(repo, 0, localFilePath, "master");
newBranch = CreateNewBranch(repo, "master");
repo.Checkout(newBranch);
}
// Commit 1000 times to the new branch
for(int i = 1; i < 1001; i++)
{
using(Repository repo = new Repository(localPath))
{
CommitRandomFile(repo, i, localFilePath, newBranch);
}
}
// Create a single new branch from the first commit ever
// For some reason seems to walk the entire commit tree
using(Repository repo = new Repository(localPath))
{
CreateNewBranch(repo, "master");
}
}
private const string Characters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
private static readonly Random Random = new Random();
/// <summary>
/// Generate and commit a random file to the specified branch
/// </summary>
public static void CommitRandomFile(Repository repo, int seed, string rootPath, string branch)
{
File.WriteAllText(rootPath, RandomString((seed % 2 + 1) * 10));
repo.Stage(rootPath);
Commit commit = repo.Commit(
string.Format("Commit: {0}", seed),
new Signature("TestAuthor", "TestEmail@Test.com", System.DateTimeOffset.Now),
new Signature("TestAuthor", "TestEmail@Test.com", System.DateTimeOffset.Now));
Stopwatch pushWatch = Stopwatch.StartNew();
repo.Network.Push(repo.Network.Remotes["origin"], "refs/heads/" + branch + ":refs/heads/" + branch);
pushWatch.Stop();
Trace.WriteLine(string.Format("Push {0} took {1}ms", seed, pushWatch.ElapsedMilliseconds));
}
/// <summary>
/// Create a new branch from the specified source
/// </summary>
public static string CreateNewBranch(Repository repo, string sourceBranch)
{
Branch source = repo.Branches[sourceBranch];
string newBranch = Guid.NewGuid().ToString();
repo.Branches.Add(newBranch, source.Tip);
Stopwatch pushNewBranchWatch = Stopwatch.StartNew();
repo.Network.Push(repo.Network.Remotes["origin"], "refs/heads/" + newBranch + ":refs/heads/" + newBranch);
pushNewBranchWatch.Stop();
Trace.WriteLine(string.Format("Push of new branch {0} took {1}ms", newBranch, pushNewBranchWatch.ElapsedMilliseconds));
return newBranch;
}
/// <summary>
/// Get a Random string of the specified length
/// </summary>
public static string RandomString(int size)
{
char[] buffer = new char[size];
for (int i = 0; i < size; i++)
{
buffer[i] = Characters[Random.Next(Characters.Length)];
}
return new string(buffer);
}