Question

这个问题之前已经被问过几次了，但每次人们说“只需添加 UTF-8”就可以了。据我了解，我现在正在处理的情况似乎无法通过 UTF-8 hack 解决？基本上，我的程序从网站上抓取数据，但此数据包含特殊的欧洲字符，如“č、š、ř”等...添加 encoding="UTF-8" 后，错误消失了，但结果 CSV 文件包含完全损坏的字符特殊字符应该位于的位置。这会破坏整个文件并使其无法使用。

我自己在互联网上找不到任何解决方案，我不知道如何处理。我需要将这些特殊字符写入文件。另一个警告是我还需要脚本是跨平台的。我不希望它只是为了“摆脱错误”而以某种方式特定于 Windows。

这是我的代码：

with open('links.csv') as read:
    reader = csv.reader(read)
    link_list = list(reader)
    with open('ScrapedContent.csv', 'w+', newline='') as write:
        writer = csv.writer(write)
        for link in link_list:
            driver.get(', '.join(link))
            title = wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "h1.page-title span.text.ng-binding")))
            offers = wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "a.switcher.ng-binding.ng-scope span.ng-binding.ng-scope")))
            address = wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "tr.c-aginfo__table__row td.ng-binding")))
            try:
                wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "button.value.link.ng-binding.ng-scope"))).click()
                phone_number = wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "span.phone.ng-binding")))
            except TimeoutException:
                pass
            try:
                wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "button.value.link.ng-binding"))).click()
                email = wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, "a.value.link.ng-binding")))
            except TimeoutException:
                pass
            try:
                phone_number = phone_number.text
            except AttributeError:
                phone_number = ""
                pass
            try:
                email = email.text
            except AttributeError:
                email = ""
                pass
            print(title.text, " ", offers.text, " ", address.text, " ", phone_number, " ", email)
            writer.writerow([title.text, offers.text, address.text, phone_number, email])
        driver.quit()

我在代码中找不到任何可能导致这种情况发生的错误。感谢您提供有关如何解决此问题的任何建议！

Answer 1

当您不添加 utf-8 时，文件看起来是否正确？它们采用什么编码？

我曾经在抓取以与响应标头中声明的编码不同的编码返回数据的网页时遇到类似问题，这有点搞砸了 requests。

我最终得到了以下为我解决的函数：

def _load_xml_content(url):
    """Loade XML content from URL, ensuring the encoding is correct."""
    response = requests.get(url)
    try:
        xml = response.text.encode(response.encoding).decode('utf-8')
    except Exception:
        xml = response.text
    return xml

直到今天，我还不能 100% 确定发生了什么……但也可能值得一试 - 也许它也能神奇地为您解决问题。

UnicodeEncodeError: 'charmap' codec can't encode character '\u011b' in position 57: character maps to <undefined> (但不能使用 UTF-8)

1 个答案: