Question

你可能知道，在Perl＆＃34; utf8＆＃34;意味着Perl对UTF-8的理解更加宽松，它允许在技术上不是UTF-8中有效代码点的字符。相比之下＆＃34; UTF-8＆＃34; （或＆＃34; utf-8＆＃34;）是Perl对UTF-8更严格的理解，它不允许无效的代码点。

我有一些与这种区别相关的使用问题：

默认情况下，Encode :: encode会将替换字符替换为无效字符。即使你正在通过更宽松的＆＃34; utf8＆＃34;也是如此。作为编码？
当您使用＆＃34; UTF-8＆＃34;来读取和写入open＆＃39;的文件时会发生什么？角色替换会发生在坏人身上还是会发生其他事情？
将open与＆＃39;＆gt;：utf8＆＃39;等图层一起使用有什么区别？以及像＆＃39;＆gt;：编码（utf8）＆＃39; ？这两种方法都可以同时使用＆ut; utf8＆＃39;和＆＃39; UTF-8＆＃39;？

Answer 1

                   ╔════════════════════════════════════════════╤══════════════════════╗
                   ║                                            │                      ║
                   ║                  On Read                   │       On Write       ║
                   ║                                            │                      ║
        Perl       ╟─────────────────────┬──────────────────────┼──────────────────────╢
        5.26       ║                     │                      │                      ║
                   ║ Invalid encoding    │ Outside of Unicode,  │ Outside of Unicode,  ║
                   ║ other than sequence │ Unicode nonchar, or  │ Unicode nonchar, or  ║
                   ║ length              │ Unicode surrogate    │ Unicode surrogate    ║
                   ║                     │                      │                      ║
╔══════════════════╬═════════════════════╪══════════════════════╪══════════════════════╣
║                  ║                     │                      │                      ║
║ :encoding(UTF-8) ║ Warns and Replaces  │ Warns and Replaces   │ Warns and Replaces   ║
║                  ║                     │                      │                      ║
╟──────────────────╫─────────────────────┼──────────────────────┼──────────────────────╢
║                  ║                     │                      │                      ║
║ :encoding(utf8)  ║ Warns and Replaces  │ Accepts              │ Warns and Accepts    ║
║                  ║                     │                      │                      ║
╟──────────────────╫─────────────────────┼──────────────────────┼──────────────────────╢
║                  ║                     │                      │                      ║
║ :utf8            ║ Corrupt scalar      │ Accepts              │ Warns and Accepts    ║
║                  ║                     │                      │                      ║
╚══════════════════╩═════════════════════╧══════════════════════╧══════════════════════╝

Click here if you have trouble viewing the above table

请注意:encoding(UTF-8)实际上使用utf8解码，然后检查范围是否允许字符（因为它识别"\x{20_000}"甚至"\x{1000_0000_0000_0000}"）。这减少了错误消息的数量，因此它很好。

（编码名称不区分大小写。）

用于生成上表的测试：

阅读

<强> :encoding(UTF-8)

printf "\xC3\xA9\n\xEF\xBF\xBF\n\xED\xA0\x80\n\xF8\x88\x80\x80\x80\n\x80\n" |
   perl -MB -nle'
      use open ":std", ":encoding(UTF-8)";
      my $sv = B::svref_2object(\$_);
      printf "%vX%s (internal: %vX, UTF8=%d)\n", $_, length($_)==1 ? "" : " = $_", $sv->PVX, utf8::is_utf8($_);
   '
utf8 "\xFFFF" does not map to Unicode.
utf8 "\xD800" does not map to Unicode.
utf8 "\x200000" does not map to Unicode.
utf8 "\x80" does not map to Unicode.
E9 (internal: C3.A9, UTF8=1)
5C.78.7B.46.46.46.46.7D = \x{FFFF} (internal: 5C.78.7B.46.46.46.46.7D, UTF8=1)
5C.78.7B.44.38.30.30.7D = \x{D800} (internal: 5C.78.7B.44.38.30.30.7D, UTF8=1)
5C.78.7B.32.30.30.30.30.30.7D = \x{200000} (internal: 5C.78.7B.32.30.30.30.30.30.7D, UTF8=1)
5C.78.38.30 = \x80 (internal: 5C.78.38.30, UTF8=1)

<强> :encoding(utf8)

$ printf "\xC3\xA9\n\xEF\xBF\xBF\n\xED\xA0\x80\n\xF8\x88\x80\x80\x80\n\x80\n" |
   perl -MB -nle'
      use open ":std", ":encoding(utf8)";
      my $sv = B::svref_2object(\$_);
      printf "%vX%s (internal: %vX, UTF8=%d)\n", $_, length($_)==1 ? "" : " = $_", $sv->PVX, utf8::is_utf8($_);
   '
utf8 "\x80" does not map to Unicode.
E9 (internal: C3.A9, UTF8=1)
FFFF (internal: EF.BF.BF, UTF8=1)
D800 (internal: ED.A0.80, UTF8=1)
200000 (internal: F8.88.80.80.80, UTF8=1)
5C.78.38.30 = \x80 (internal: 5C.78.38.30, UTF8=1)

<强> :utf8

$ printf "\xC3\xA9\n\xEF\xBF\xBF\n\xED\xA0\x80\n\xF8\x88\x80\x80\x80\n\x80\n" |
   perl -MB -nle'
      use open ":std", ":encoding(utf8)";
      my $sv = B::svref_2object(\$_);
      printf "%vX%s (internal: %vX, UTF8=%d)\n", $_, length($_)==1 ? "" : " = $_", $sv->PVX, utf8::is_utf8($_);
   '
E9 (internal: C3.A9, UTF8=1)
FFFF (internal: EF.BF.BF, UTF8=1)
D800 (internal: ED.A0.80, UTF8=1)
200000 (internal: F8.88.80.80.80, UTF8=1)
Malformed UTF-8 character: \x80 (unexpected continuation byte 0x80, with no preceding start byte) in printf at -e line 4, <> line 5.
0 (internal: 80, UTF8=1)

写入

<强> :encoding(UTF-8)

$ perl -e'
   use open ":std", ":encoding(UTF-8)";
   print "\x{E9}\n";
   print "\x{FFFF}\n";
   print "\x{D800}\n";
   print "\x{20_0000}\n";
' >a
Unicode non-character U+FFFF is not recommended for open interchange in print at -e line 4.
Unicode surrogate U+D800 is illegal in UTF-8 at -e line 5.
Code point 0x200000 is not Unicode, may not be portable in print at -e line 6.
"\x{ffff}" does not map to utf8.
"\x{d800}" does not map to utf8.
"\x{200000}" does not map to utf8.

$ od -t c a
0000000 303 251  \n   \   x   {   F   F   F   F   }  \n   \   x   {   D
0000020   8   0   0   }  \n   \   x   {   2   0   0   0   0   0   }  \n
0000040

$ cat a
é
\x{FFFF}
\x{D800}
\x{200000}

<强> :encoding(utf8)

$ perl -e'
   use open ":std", ":encoding(utf8)";
   print "\x{E9}\n";
   print "\x{FFFF}\n";
   print "\x{D800}\n";
   print "\x{20_0000}\n";
' >a
Unicode surrogate U+D800 is illegal in UTF-8 at -e line 4.
Code point 0x200000 is not Unicode, may not be portable in print at -e line 5.

$ od -t c a
0000000 303 251  \n 355 240 200  \n 370 210 200 200 200  \n
0000015

$ cat a
é
▒
▒

<强> :utf8

与:encoding(utf8)相同的结果。

使用Perl 5.26进行测试。

默认情况下，Encode :: encode会将替换字符替换为无效字符。即使你正在通过更宽松的＆＃34; utf8＆＃34;也是如此。作为编码？

Perl字符串是32位或64位字符的字符串，具体取决于构建。 utf8可以编码任何72位整数。因此，它能够编码可以被要求编码的所有字符。

使用Encode :: encode with＆＃34; utf8＆＃34;

1 个答案:

阅读

写入