Question

我创建了一个抓取网站的脚本：1688.com问题是，该网站是中文的，所以每当我尝试检索文本时，它都会给我一堆unicode，当我导出到CSV文件时，文件中没有任何内容。我的代码：

# -*- coding: utf-8 -*-
import csv
from urllib import urlopen
from bs4 import BeautifulSoup as BS

csv_content = open('content.csv', 'w+')
writer_content = csv.writer(csv_content)

url = urlopen('https://fuzhuang.1688.com/nvzhuang?
spm=a260k.635.1998214976.1.7eqUGT')
html = BS(url, 'lxml')
container = html.find('ul', {'class' : 'ch-box fd-clr'})
offers = container.find_all('div', {'class' : 'ch-offer-body'})
lst = []

for offer in offers:
    offer_box = offer.find('div', {'component-name' : '@alife/ocms-
component-1688-pc-ch-offer-pic'})
    images = offer_box.find('img')['src']
    title = offer.find('div', {'class' : 'ocms-component-1688-pc-ch-offer-
title-0-1-11'}).text
    price = offer.find('div', {'class' : 'ocms-component-1688-pc-ch-offer-
price-0-1-14'}).text
    lst.append(price)

对于lst中的项目： writer_content.writerow（[项目]）

print lst

输出

[u'\n\n\n\xa5\n109.00\n\n\n\u6210\u4ea4\n329\n\u4ef6\n\n\n', u'\n\n\n\xa5\n56.00\n\n\n\u6210\u4ea4\n195\n\u4ef6\n\n\n', u'\n\n\n\xa5\n83.00\n\n\n\u6210\u4ea4\n109\n\u4ef6\n\n\n', u'\n\n\n\xa5\n69.00\n\n\n\u6210\u4ea4\n208\n\u4ef6\n\n\n', u'\n\n\n\xa5\n46.00\n\n\n\u6210\u4ea4\n204\n\u4ef6\n\n\n', u'\n\n\n\xa5\n45.00\n\n\n\u6210\u4ea4\n54\n\u4ef6\n\n\n', u'\n\n\n\xa5\n82.00\n\n\n\u6210\u4ea4\n38\n\u4ef6\n\n\n', u'\n\n\n\xa5\n48.90\n\n\n\u6210\u4ea4\n318\n\u4ef6\n\n\n']

我已经尝试过编码和解码utf-8，如果你告诉我如何解决这个问题，我将非常感激。

Answer 1

此代码将中文符号保存为txt：

for Python3：

         ...
(all your code above)
for i in range(len(lst)):    
    lst[i]=lst[i].replace('\n','') #getting rig of `'\n'` newlines

写给txt：

with open(r'C:\Users\Username\list.txt','w',newline='',encoding='utf-8-sig') as f:
    for i in lst:
        f.write(i+'\t')

for Python2：

import unicodecsv as ucsv
with open(r'C:\Users\Username\list1.txt','wb') as f:
    w = ucsv.writer(f,encoding='utf-8-sig')
    for i in lst:
        w.writerow([i+'\t'])

在抓取时无法检索中文文本

1 个答案: