Question

我打算用英语抓取dintex.net网站，但找不到任何方法来转换英语的抓取数据。我也使用了googletans，但它也显示错误，因此还有其他方法可以将该页面或数据转换为英语吗？

import scrapy
from googletrans import Translator
class DtSpider(scrapy.Spider):
name = 'dt'
start_urls = ['http://www.dintex.net']

def parse(self, response):
    urls = response.xpath('//*[@class="listing-btn btn btn-primary btn-block w-100"]/@href').extract()
    for url in urls:
        url = response.urljoin(url)
        yield scrapy.Request(url=url, callback=self.parse_details)

    np = response.xpath('//*[@class="page-item"]/a[@rel="next"]/@href').extract_first()
    ap = response.urljoin(np)
    yield scrapy.Request(url=ap,callback=self.parse)

def parse_details(self,response):
    Title = response.xpath('//*[@class="show-info__title"]/text()').extract_first()
    Location = response.xpath('//*[@class="show-info__location"]/p/text()').extract_first()
    Contact = response.xpath('//*[@class="show-info__contact-details__phone-link"]/text()').extract_first()
    Contact = Contact.replace('Whatsapp ','')
    Description = response.xpath('//*[@class="show-info__section-text"]/p/text()').extract_first()
    Manufacture = response.xpath('//td[contains(text(),"Fabricante")]/following-sibling::td/text()').extract_first()
    Model = response.xpath('//td[contains(text(),"Modelo")]/following-sibling::td/text()').extract_first()
    Year = response.xpath('//td[contains(text(),"Año")]/following-sibling::td/text()').extract_first()
    Condition = response.xpath('//td[contains(text(),"Condición")]/following-sibling::td/text()').extract_first()
    img = response.xpath('//*[@class="gallery__item"]/img/@src').extract_first()
    thumbs =  response.xpath('//img/@lazy-src').extract()

    #t = Translator()
    #Title = t.translate(Title).text
    #Location = t.translate(Location).text
    #Contact = t.translate(Contact).text
    #Description = t.translate(Description).text
    #Manufacture = t.translate(Manufacture).text
    #Model = t.translate(Model).text
    #Year = t.translate(Year).text
    #Condition = t.translate(Condition).text

    yield{'Title': Title,
    'Location' : Location,
    'Contact' : Contact,
    'Description' : Description,
    'Manufacture' : Manufacture,
    'Model' : Model,
    'Year' : Year,
    'Condition' : Condition,
    'Img' : img,
    'Thums' : thumbs
    }

Answer 1

我认为您应该随请求发送此Cookie

googtrans=/es/en

该页面允许根据可用语言/地区的选择进行本地化。

您需要执行以下操作查看scrapy request from scrapy docs

的Cookie部分

您产生的请求可能需要更改这样的内容（未经测试）

scrapy.Request(url=url, cookies= {'googletrans': '/es/en'}, callback=self.parse_details)

有什么方法可以翻译网页语言，或在使用scrapy进行抓取的同时翻译抓取的数据？

1 个答案: