用Python将CSV数据加载到MySQL中

时间:2012-04-14 15:09:17

标签: python mysql

不确定我在这里缺少什么,但这段代码运行时没有任何错误消息,但表中没有任何内容。我正在将三列中的CSV值加载到mysql表

import csv
import MySQLdb

mydb = MySQLdb.connect(host='localhost',
    user='root',
    passwd='',
    db='mydb')
cursor = mydb.cursor()

csv_data = csv.reader(file('students.csv'))
for row in csv_data:

    cursor.execute('INSERT INTO testcsv(names, \
          classes, mark )' \
          'VALUES("%s", "%s", "%s")', 
          row)
#close the connection to the database.
cursor.close()
print "Done"

如果有其他人可以看看,我们将不胜感激。

谢谢。

7 个答案:

答案 0 :(得分:66)

我认为你必须mydb.commit()进行所有插入。

像这样的东西

import csv
import MySQLdb

mydb = MySQLdb.connect(host='localhost',
    user='root',
    passwd='',
    db='mydb')
cursor = mydb.cursor()

csv_data = csv.reader(file('students.csv'))
for row in csv_data:

    cursor.execute('INSERT INTO testcsv(names, \
          classes, mark )' \
          'VALUES("%s", "%s", "%s")', 
          row)
#close the connection to the database.
mydb.commit()
cursor.close()
print "Done"

答案 1 :(得分:3)

以上答案似乎不错。但是另一种实现方法是在数据库连接中添加自动提交选项。这会自动提交数据库中执行的所有其他操作,从而避免每次都使用提及sql.commit()的情况。

 mydb = MySQLdb.connect(host='localhost',
        user='root',
        passwd='',
        db='mydb',autocommit=true)

答案 2 :(得分:3)

使用pymsql(如果有帮助的话)

import pymysql
import csv
db = pymysql.connect("localhost","root","12345678","data" )

cursor = db.cursor()
csv_data = csv.reader(open('test.csv'))
next(csv_data)
for row in csv_data:
    cursor.execute('INSERT INTO PM(col1,col2) VALUES(%s, %s)',row)

db.commit()
cursor.close()

答案 3 :(得分:1)

如果是熊猫数据框,你可以这样做:

发送数据

csv_data.to_sql=(con=mydb, name='<the name of your table>',
  if_exists='replace', flavor='mysql')

避免使用for

答案 4 :(得分:0)

  from __future__ import print_function
import csv
import MySQLdb

print("Enter  File  To Be Export")
conn = MySQLdb.connect(host="localhost", port=3306, user="root", passwd="", db="database")
cursor = conn.cursor()
#sql = 'CREATE DATABASE test1'
sql ='''DROP TABLE IF EXISTS `test1`; CREATE TABLE test1 (policyID int, statecode varchar(255), county varchar(255))'''
cursor.execute(sql)

with open('C:/Users/Desktop/Code/python/sample.csv') as csvfile:
    reader = csv.DictReader(csvfile, delimiter = ',')
    for row in reader:
        print(row['policyID'], row['statecode'], row['county'])
        # insert
        conn = MySQLdb.connect(host="localhost", port=3306, user="root", passwd="", db="database")
        sql_statement = "INSERT INTO test1(policyID ,statecode,county) VALUES (%s,%s,%s)"
        cur = conn.cursor()
        cur.executemany(sql_statement,[(row['policyID'], row['statecode'], row['county'])])
        conn.escape_string(sql_statement)
        conn.commit()

答案 5 :(得分:0)

如果您没有 pandas sqlalchemy 库,请使用pip导入

> pip install pandas
> pip install sqlalchemy

我们可以使用 pandas sqlalchemy 直接插入数据库

import csv
import pandas as pd
from sqlalchemy import create_engine,types

engine = create_engine('mysql://root:*Enter password here*@localhost/*Enter Databse name here*') # enter your password and database names here

df = pd.read_csv("Excel_file_name.csv",sep=',',quotechar='\'',encoding='utf8') # Replace Excel_file_name with your excel sheet name
df.to_sql('Table_name',con=engine,index=False,if_exists='append') # Replace Table_name with your sql table name

答案 6 :(得分:-1)

最快的方法是通过“ load data infile”语句使用MySQL批量加载器。到目前为止,这是最快的方法,远胜过Python。如果必须使用Python,则可以从Python本身调用语句“ load data infile”。