Unicode issue in scrapy python

问题

For two hours, I am searching for this topic and I have tried a lot of solutions but noen worked in my case Here's the code first

import scrapy

class HamburgSpider(scrapy.Spider):
    name = 'hamburg'
    #allowed_domains = ['https://www.hamburg.de']
    start_urls = ['https://www.hamburg.de/branchenbuch/hamburg/10239785/n0/']
    custom_settings = {
        'FEED_EXPORT_FORMAT': 'utf-8'
    }

    def parse(self, response):
        #response=response.body.encode('utf-8')
        items = response.xpath("//div[starts-with(@class, 'item')]")
        for item in items:
            business_name = item.xpath(".//h3[@class='h3rb']/text()").get()
            address1 = item.xpath(".//div[@class='address']/p[@class='extra post']/text()[1]").get()
            address2 = item.xpath(".//div[@class='address']/p[@class='extra post']/text()[2]").get()
            phone = item.xpath(".//div[@class='address']/span[@class='extra phone']/text()").get()

            yield {
                'Business Name': business_name,
                'Address1': address1,
                'Address2': address2,
                'Phone Number': phone
            }

In the code I put this line

custom_settings = { 'FEED_EXPORT_FORMAT': 'utf-8' }

The line supposed to deal with the issue of encoding but when exporting the results to csv, I found that the issue is still there. I simply need to show this example of text Poppenbütteler Bogen 29a sa shown on the website. What I found is that the output is different

回答1:

You have the wrong setting name.

FEED_EXPORT_FORMAT is not one of the settings scrapy uses by default, you want FEED_EXPORT_ENCODING instead.

来源：https://stackoverflow.com/questions/64781553/unicode-issue-in-scrapy-python

标签

python

unicode

scrapy