在文件夹中搜索csv文件并插入mysql数据库

时间:2016-05-09 08:22:31

标签: c# mysql parsing

我试图在基于此post的mysql数据库中的目录/插入中解析文件,并且几乎它正在工作,除了每列的csv值被舍入并在数据库中成为INT值。我假设我需要将DECIMAL更改为其他内容,但我无法弄清楚。

(例如:1.11变为1,45.5变为46)

csv文件:

test1,test2,test3,test4 
1.11,1.23,67.4,4.5
1.12,5.42,45.5,6.45

我的代码:

public void sqlTest()
    {
        string connectionString = "server=localhost;database=test;UserId=root;password=test;";
        using (MySqlConnection connection = new MySqlConnection(connectionString))
        {
            connection.Open();
            using (MySqlCommand cmd = connection.CreateCommand())
            {
                cmd.CommandText = @"DROP TABLE test IF EXISTS";
                cmd.CommandText = @"CREATE TABLE test ( 
                                        Id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
                                        test1 DECIMAL,
                                        test2 DECIMAL,
                                        test3 DECIMAL,
                                        test4 DECIMAL
                                    )";
                cmd.ExecuteNonQuery();
            }
            connection.Close();
            connection.Open();

            using (MySqlCommand insertCommand = connection.CreateCommand())
            {
                insertCommand.CommandText =
                    @"INSERT INTO test (test1, test2, test3, test4)
                      VALUES (@test1, @test2, @test3, @test4)";

                insertCommand.Parameters.Add(new MySqlParameter("@test1", DbType.Decimal));
                insertCommand.Parameters.Add(new MySqlParameter("@test2", DbType.Decimal));
                insertCommand.Parameters.Add(new MySqlParameter("@test3", DbType.Decimal));
                insertCommand.Parameters.Add(new MySqlParameter("@test4", DbType.Decimal));

                string[] files = Directory.GetFiles("D:/computer/csv files", "*.csv");

                foreach (string file in files)
                {
                    string[] lines = System.IO.File.ReadAllLines(file);
                    bool parse = false;

                    foreach (string tmpLine in lines)
                    {
                        string line = tmpLine.Trim();
                        if (!parse && line.StartsWith("test1"))
                        {
                            parse = true;
                            continue;
                        }
                        if (!parse || string.IsNullOrEmpty(line))
                        {
                            continue;
                        }

                        foreach (MySqlParameter parameter in insertCommand.Parameters)
                        {
                            parameter.Value = null;
                        }

                        string[] values = line.Split(new[] { ',' });

                        for (int i = 0; i < values.Length; i++)
                        {
                            MySqlParameter param = insertCommand.Parameters[i];

                            decimal value;
                            param.Value = decimal.TryParse(values[i], out value) ? value : 0;

                        }
                        insertCommand.ExecuteNonQuery();
                    }

                }
            }
            connection.Close();
        }
    }        
}

感谢任何指导或提示!

1 个答案:

答案 0 :(得分:0)

如果我在你的位置,我会先将表结构更改为

 CREATE TABLE `test`(
   `test1` DOUBLE(13,2),
   `test2` DOUBLE(13,2),
   `test3` DOUBLE(13,2),
   `test4` DOUBLE(13,2) );

然后,您可以使用 LOAD DATA INFILE 命令将csv文件导入到mysql中的表中,而不是读取文件的内容并为每条记录执行插入操作。

查看此链接MySQL - LOAD DATA INFILE.以获取完整文档。 示例用法是这样的

LOAD DATA INFILE 'test.csv' INTO TABLE test
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES

根据您的服务器环境,您需要正确编辑csv文件的路径并确保您的mysql本地infile标志设置为1,否则您可以使用此命令设置它

mysql -u username -p --local-infile=1

回到你的代码,它可以简化为

...
...
...

public void sqlTest()
{
    string connectionString = "server=localhost;database=test;UserId=root;password=test;";
    using (MySqlConnection connection = new MySqlConnection(connectionString))
    {
        connection.Open();
        string query = @"DROP TABLE test IF EXISTS;
                        CREATE TABLE `test`(
                       `test1` DOUBLE(13,2),
                       `test2` DOUBLE(13,2),
                       `test3` DOUBLE(13,2),
                       `test4` DOUBLE(13,2) );";
        using (MySqlCommand cmd = new MySqlCommand(query, connection))
        {               
            cmd.ExecuteNonQuery();
            string[] files = Directory.GetFiles("D:/computer/csv files", "*.csv");

            foreach (string file in files)
            {
               query =@"LOAD DATA INFILE '"+file+"' INTO TABLE test
                        FIELDS TERMINATED BY ','
                        LINES TERMINATED BY '\r\n'
                        IGNORE 1 LINES"; 
                using (MySqlCommand cmd1 = new MySqlCommand(query, connection))
                {   
                    cmd1.ExecuteNonQuery();
                }                   

            }
        }

    }
}
...
...
...