如何覆盖CsvItemExporter的join_multivalued选项

Question

CsvItemExporter有一个选项join_multivalued，默认为逗号（=','）。

如何在scrapy项目中将其更改为另一个char？

Answer 1

以这种方式将其设置为自定义CSVItemExporter：

from scrapy.conf import settings
from scrapy.contrib.exporter import CsvItemExporter

class CSVkwItemExporter(CsvItemExporter):

    def __init__(self, *args, **kwargs):
        kwargs['fields_to_export'] = settings.getlist('EXPORT_FIELDS') or None
        kwargs['encoding'] = settings.get('EXPORT_ENCODING', 'utf-8')

        super(CSVkwItemExporter, self).__init__(*args, **kwargs)

        self._join_multivalued = settings.get('MY_CSV_DELIMITER', ',')

其中MY_CSV_DELIMITER是分隔符的示例设置。

Answer 2

Scrapy版本1.5.0

your_scrapy_project / exporters.py

from scrapy.conf import settings
from scrapy.exporters import CsvItemExporter

class MyCsvItemExporter(CsvItemExporter):

    def __init__(self, *args, **kwargs):

        delimiter = settings.get('CSV_DELIMITER', ';')
        kwargs['delimiter'] = delimiter

        super(MyCsvItemExporter, self).__init__(*args, **kwargs) 

your_scrapy_project / settings.py

FEED_EXPORT_ENCODING = 'utf-8'

FEED_EXPORTERS = {
    'csv': 'your_scrapy_project.exporters.MyCsvItemExporter',
}

CSV_DELIMITER = ';'