Question

所以我有一个带有3000万行的.txt文件，我用MD5和SHA1对它们进行散列并将其插入我的MySQL数据库。它完全正常，但每个哈希需要1秒。那是3000万秒......

问题：它不够快，每1秒1太慢了。

我想要的东西：某种类型的线程可以加速它，所以它确实像20秒1秒！（也许那太不现实了）

这是我的代码，随意操作:(我使用MySql.Data作为参考）

        bool success;
        int done = 0;
        string command;
        WebClient putHash = new WebClient();

        string server = "my_server";
        string database = "my_db";
        string uid = "my_dbuser";
        string password = "my_pass";
        string connectionstring = "SERVER=" + server + ";DATABASE=" + database + ";UID=" + uid + ";PASSWORD=" + password + ";";

        MySqlConnection cnn = new MySqlConnection(connectionstring);
        MySqlCommand cmd = new MySqlCommand();

        foreach (string line in File.ReadLines(@"directory with .txt file which has 30million lines"))
        {
                string linefixed = line.Replace(" ", "").Replace("'", "").Replace(";", "").Replace(")", "").Replace("\\", "").Replace("=", "");
                success = false;
                byte[] hashedv = new UTF8Encoding().GetBytes(linefixed);
                byte[] hash = ((HashAlgorithm)CryptoConfig.CreateFromName("MD5")).ComputeHash(hashedv);
                string encodedinput = BitConverter.ToString(hash).Replace("-", string.Empty).ToLower();

                byte[] bytes = Encoding.UTF8.GetBytes(linefixed);

                var sha1 = SHA1.Create();
                byte[] hashBytes = sha1.ComputeHash(bytes);

                byte[] hexed = hashBytes;

                var sb = new StringBuilder();
                foreach (byte b in hexed)
                {
                    var hex = b.ToString("x2");
                    sb.Append(hex);
                }
                string sha1done = sb.ToString();

                while (!success)
                {
                    try
                    {
                        command = "INSERT INTO passwdhashes (password,MD5,SHA1) VALUES('" + linefixed + "','" + encodedinput + "','" + sha1done + "')" ;
                        cmd.CommandText = command;
                        cmd.Connection = cnn;
                        cnn.Open();
                        cmd.ExecuteNonQuery();
                        cnn.Close();
                        success = true;
                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine(ex.Message);
                        cnn.Close();
                    }
                }
            done = done + 1;
            Console.WriteLine("\n" + done);
        }
        Console.ReadKey();
    }
}

}

Answer 1

您可以一次插入多行：https://dev.mysql.com/doc/refman/5.5/en/insert.html

像这样：INSERT INTO passwordHashed（密码，MD5，SHA1）VALUES（1,2,3），（4,5,6），（7,8,9）;

这是示例/伪代码，可能无法编译：

List<ElementType> elementsToInsert;

foreach (string line in File.ReadLines(@"file")) {

    elementsToInsert.Add(element);
    if (elementsToInsert.size == 100) {
        // execute SQL insertion
        elementsToInsert.clear();
    }
}

ElementType应该是一个包含您用于插入的字段的类。

使用C＃提高在MySQL数据库中插入数据的速度

1 个答案: