我正在使用scrapy来抓取一些图像,我想将信息存储到mysql中。我的项目中有一些字段,当我将项目写入mysql时,一切正常,但是一个字段无法写入mysql, mysql表中的字段为空。 该项目:
class ProductItem(scrapy.Item):
brand = scrapy.Field()
series = scrapy.Field()
commodity = scrapy.Field()
designer = scrapy.Field()
year = scrapy.Field()
description = scrapy.Field()
url = scrapy.Field()
name = scrapy.Field()
管道:
class MySQLPipeline(object):
def __init__(self):
host = 'localhost'
user = 'root'
passwd = '123456'
db = 'haijuhui'
self.db = MySQLdb.connect(host=host, user=user, passwd=passwd, db=db)
self.cursor = self.db.cursor()
def process_item(self, item, spider):
pp('============================')
pp(item['description'])
pp('============================')
fields = ''
values = ''
for k, v in item.items():
fields += k + ','
values += "'" + v + "'" + ','
fields = fields[0:-1]
values = values[0:-1]
insert_sql = "INSERT INTO product (%s) values (%s)" % (fields, values)
self.cursor.execute(insert_sql)
self.db.commit()
return item
管道的印刷品:
'============================'
u'\r\n Butterfly gets its name from the \u201cbutterfly\u201d shape created by the outer corner that joins the armrest and the backrest. It is a modular seating system that responds effectively to the needs of modularity and comfort, suitable for free and informal usage. Its modular structure is shaped around an ottoman, which is the base piece, and which, when integrated with the backrest and/or the armrest, becomes a central or terminal element, either left or right. The combination of these elements, using a simple joint system, makes it possible to freely set up versatile configurations, from in-line sofas to corner solutions, from front-facing to lounge layouts. This first textile sofa in B&B Italia Outdoor collection is characterised by soft and sinuous shapes. Completely upholstered, it leverages the company\u2019s production technology, with materials and solutions designed for outdoor use, in order to take outdoors the quality and comfort that has always characterized indoor seating. The system includes a family of complements, small tables and consoles, seamlessly integrated with the seats, making it possible to design many compositions suitable for different space customization requirements. The strong material component of Butterfly\u2019s cover and the softness of its shapes are an invitation to comfort, which, together with the modularity of the system, make it ideal for transversal projects, from hotels to marine applications.\r\n '
'============================'
........
insert sql:
u"INSERT INTO product (designer,description,commodity,url,series,brand,year,name) values ('PATRICIA URQUIOLA','\r\n Butterfly gets its name from the \u201cbutterfly\u201d shape created by the outer corner that joins the armrest and the backrest. It is a modular seating system that responds effectively to the needs of modularity and comfort, suitable for free and informal usage. Its modular structure is shaped around an ottoman, which is the base piece, and which, when integrated with the backrest and/or the armrest, becomes a central or terminal element, either left or right. The combination of these elements, using a simple joint system, makes it possible to freely set up versatile configurations, from in-line sofas to corner solutions, from front-facing to lounge layouts. This first textile sofa in B&B Italia Outdoor collection is characterised by soft and sinuous shapes. Completely upholstered, it leverages the company\u2019s production technology, with materials and solutions designed for outdoor use, in order to take outdoors the quality and comfort that has always characterized indoor seating. The system includes a family of complements, small tables and consoles, seamlessly integrated with the seats, making it possible to design many compositions suitable for different space customization requirements. The strong material component of Butterfly\u2019s cover and the softness of its shapes are an invitation to comfort, which, together with the modularity of the system, make it ideal for transversal projects, from hotels to marine applications.\r\n ','SOFAS','http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-01-OUTDOOR-BUTTERFLY-BUTTERFLY_01.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-02-OUTDOOR-BUTTERFLY-BUTTERFLY_02.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-03-OUTDOOR-BUTTERFLY-BUTTERFLY_03.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-04-OUTDOOR-BUTTERFLY-BUTTERFLY_04.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-05-OUTDOOR-BUTTERFLY-BUTTERFLY_05.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-06-OUTDOOR-BUTTERFLY-BUTTERFLY_06.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-07-OUTDOOR-BUTTERFLY-BUTTERFLY_07.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-08-OUTDOOR-BUTTERFLY-BUTTERFLY_08.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-09-OUTDOOR-BUTTERFLY-BUTTERFLY_09.jpg,http://www.bebitalia.com/App_Uploads/Prodotti/0385/Gallery/big-10-OUTDOOR-BUTTERFLY-BUTTERFLY_10.jpg','BUTTERFLY','B&B Italia Outdoor','2015','BUTTERFLY - PHOTO GALLERY - 1,BUTTERFLY - PHOTO GALLERY - 2,BUTTERFLY - PHOTO GALLERY - 3,BUTTERFLY - PHOTO GALLERY - 4,BUTTERFLY - PHOTO GALLERY - 5,BUTTERFLY - PHOTO GALLERY - 6,BUTTERFLY - PHOTO GALLERY - 7,BUTTERFLY - PHOTO GALLERY - 8,BUTTERFLY - PHOTO GALLERY - 9,BUTTERFLY - PHOTO GALLERY - 10')"
作为jkavalik的评论,当执行语句从产品中选择描述,十六进制(描述)时,描述不为空,其值如下:
0D0A202020202020202020202020202020202020202020202020427574746572666C79206765747320697473206E616D652066726F6D2074686520E2809C627574746572666C79E2809D207368617065206372656174656420627920746865206F7574657220636F726E65722074686174206A6F696E73207468652061726D7265737420616E6420746865206261636B726573742E2049742069732061206D6F64756C61722073656174696E672073797374656D207468617420726573706F6E6473206566666563746976656C7920746F20746865206E65656473206F66206D6F64756C617269747920616E6420636F6D666F72742C207375697461626C6520666F72206672656520616E6420696E666F726D616C2075736167652E20497473206D6F64756C617220737472756374757265206973207368617065642061726F756E6420616E206F74746F6D616E2C2077686963682069732074686520626173652070696563652C20616E642077686963682C207768656E20696E7465677261746564207769746820746865206261636B7265737420616E642F6F72207468652061726D726573742C206265636F6D657320612063656E7472616C206F72207465726D696E616C20656C656D656E742C20656974686572206C656674206F722072696768742E2054686520636F6D62696E6174696F6E206F6620746865736520656C656D656E74732C207573696E6720612073696D706C65206A6F696E742073797374656D2C206D616B657320697420706F737369626C6520746F20667265656C792073657420757020766572736174696C6520636F6E66696775726174696F6E732C2066726F6D20696E2D6C696E6520736F66617320746F20636F726E657220736F6C7574696F6E732C2066726F6D2066726F6E742D666163696E6720746F206C6F756E6765206C61796F7574732E20546869732066697273742074657874696C6520736F666120696E20422642204974616C6961204F7574646F6F7220636F6C6C656374696F6E206973206368617261637465726973656420627920736F667420616E642073696E756F7573207368617065732E20436F6D706C6574656C79207570686F6C7374657265642C206974206C65766572616765732074686520636F6D70616E79E28099732070726F64756374696F6E20746563686E6F6C6F67792C2077697468206D6174657269616C7320616E6420736F6C7574696F6E732064657369676E656420666F72206F7574646F6F72207573652C20696E206F7264657220746F2074616B65206F7574646F6F727320746865207175616C69747920616E6420636F6D666F727420746861742068617320616C7761797320636861726163746572697A656420696E646F6F722073656174696E672E205468652073797374656D20696E636C7564657320612066616D696C79206F6620636F6D706C656D656E74732C20736D616C6C207461626C657320616E6420636F6E736F6C65732C207365616D6C6573736C7920696E74656772617465642077697468207468652073656174732C206D616B696E6720697420706F737369626C6520746F2064657369676E206D616E7920636F6D706F736974696F6E73207375697461626C6520666F7220646966666572656E7420737061636520637573746F6D697A6174696F6E20726571756972656D656E74732E20546865207374726F6E67206D6174657269616C20636F6D706F6E656E74206F6620427574746572666C79E280997320636F76657220616E642074686520736F66746E657373206F6620697473207368617065732061726520616E20696E7669746174696F6E20746F20636F6D666F72742C2077686963682C20746F676574686572207769746820746865206D6F64756C6172697479206F66207468652073797374656D2C206D616B6520697420696465616C20666F72207472616E7376657273616C2070726F6A656374732C2066726F6D20686F74656C7320746F206D6172696E65206170706C69636174696F6E732E0D0A2020202020202020202020202020202020202020
mysql表
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| brand | varchar(255) | NO | | NULL | |
| series | varchar(255) | NO | | NULL | |
| commodity | varchar(255) | NO | | NULL | |
| designer | varchar(255) | NO | | NULL | |
| year | varchar(255) | NO | | NULL | |
| description | longtext | NO | | NULL | |
| url | text | NO | | NULL | |
| name | text | NO | | NULL | |
+-------------+--------------+------+-----+---------+----------------+
除描述字段外,所有字段值都可以插入表中。
我找不到导致这种情况的原因,请帮帮我..