外部字符出现在Git管理的文件中

时间:2010-10-20 18:22:55

标签: windows visual-studio visual-studio-2008 unicode encoding

我在Windows 7上通过cygwin使用git 1.7.2.3,并在切换分支时看到我的一些源文件中出现奇怪的文件。 git status报告所有内容都保持不变,但他们会出现疯狂的角色。我已经在GitHub上确认文件与它们应该在repo中一样。

我的副本:

਍        ⼀⼀⼀ 㰀猀甀洀洀愀爀礀㸀ഀഀ
        /// Set up method.
਍        ⼀⼀⼀ 㰀⼀猀甀洀洀愀爀礀㸀ഀഀ
        [SetUp]
਍        瀀甀戀氀椀挀 漀瘀攀爀爀椀搀攀 瘀漀椀搀 匀攀琀甀瀀⠀⤀ഀഀ
        {
਍            琀栀椀猀⸀匀挀漀瀀攀 㴀 渀攀眀 吀爀愀渀猀愀挀琀椀漀渀匀挀漀瀀攀⠀⤀㬀ഀഀ
਍            琀栀椀猀⸀琀攀猀琀䤀琀攀洀 㴀 渀攀眀 嘀椀攀眀䐀漀挀甀洀攀渀琀䠀椀猀琀漀爀礀⠀ ഀഀ
                625016, 
਍                㔀㜀㤀㤀㘀Ⰰ ഀഀ
                'T', 
਍                ㌀㐀㠀㌀㔀㈀㤀Ⰰ ഀഀ
                DateTime.Parse("2003-01-08 09:57:04.957"),
਍                ㌀Ⰰ ഀഀ
                "Invoice (PG-PS) - SUPP(11/16/2008)", 
਍                ∀䘀䤀一䄀一䌀䔀∀Ⰰ ഀഀ
                DateTime.Parse("2008-04-11 11:15:07.770"), 
਍                䀀∀尀尀䐀伀匀䬀尀䌀䜀䐀伀䌀匀尀㌀㜀㐀㤀㄀㐀尀㐀㘀 㐀㘀尀戀椀氀猀氀椀瀀开㄀ 㠀㄀㘀㐀㠀⸀搀漀挀∀⤀㬀ഀഀ
        }

回购复制:

        /// <summary>
        /// Set up method.
        /// </summary>
        [SetUp]
        public override void Setup()
        {
            this.Scope = new TransactionScope();

            this.testItem = new ViewDocumentHistory( 
                625016, 
                57996, 
                'T', 
                3483529, 
                DateTime.Parse("2003-01-08 09:57:04.957"),
                3, 
                "Invoice (PG-PS) - SUPP(11/16/2008)", 
                "FINANCE", 
                DateTime.Parse("2008-04-11 11:15:07.770"), 
                @"\\DOSK\CGDOCS\374914\46046\bilslip_1081648.doc");
        }

我正在使用.gitattributes文件来确保行结尾是正确的,因为我们正在开发Windows。

*.cs       eol=crlf text
*.csproj   eol=crlf text
*.sln      eol=crlf text
*.xml      eol=crlf text

text是我尝试解决问题的补充,因为git diff在修改文件时将文件解释为二进制文件。没有任何影响。

据我所知,这也发生在1.7.2.3中的新结账,但不是1.6.5.1(mysysgit)。需要注意的是1.6不支持我在Windows上工作所需的.gitattributes。这似乎是一个相当新的错误,我没有改变任何配置。

有没有人知道造成这种情况的原因是什么?


编辑:

hexdump -C ViewDocumentHistoryTests.cs | sed -n "130,212p"

000008d0  00 20 00 20 00 2f 00 2f  00 2f 00 20 00 3c 00 73  |. . ./././. .<.s|
000008e0  00 75 00 6d 00 6d 00 61  00 72 00 79 00 3e 00 0d  |.u.m.m.a.r.y.>..|
000008f0  00 0d 0a 00 20 00 20 00  20 00 20 00 20 00 20 00  |.... . . . . . .|
00000900  20 00 20 00 2f 00 2f 00  2f 00 20 00 53 00 65 00  | . ./././. .S.e.|
00000910  74 00 20 00 75 00 70 00  20 00 6d 00 65 00 74 00  |t. .u.p. .m.e.t.|
00000920  68 00 6f 00 64 00 2e 00  0d 00 0d 0a 00 20 00 20  |h.o.d........ . |
00000930  00 20 00 20 00 20 00 20  00 20 00 20 00 2f 00 2f  |. . . . . . ././|
00000940  00 2f 00 20 00 3c 00 2f  00 73 00 75 00 6d 00 6d  |./. .<./.s.u.m.m|
00000950  00 61 00 72 00 79 00 3e  00 0d 00 0d 0a 00 20 00  |.a.r.y.>...... .|
00000960  20 00 20 00 20 00 20 00  20 00 20 00 20 00 5b 00  | . . . . . . .[.|
00000970  53 00 65 00 74 00 55 00  70 00 5d 00 0d 00 0d 0a  |S.e.t.U.p.].....|
00000980  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
00000990  00 70 00 75 00 62 00 6c  00 69 00 63 00 20 00 6f  |.p.u.b.l.i.c. .o|
000009a0  00 76 00 65 00 72 00 72  00 69 00 64 00 65 00 20  |.v.e.r.r.i.d.e. |
000009b0  00 76 00 6f 00 69 00 64  00 20 00 53 00 65 00 74  |.v.o.i.d. .S.e.t|
000009c0  00 75 00 70 00 28 00 29  00 0d 00 0d 0a 00 20 00  |.u.p.(.)...... .|
000009d0  20 00 20 00 20 00 20 00  20 00 20 00 20 00 7b 00  | . . . . . . .{.|
000009e0  0d 00 0d 0a 00 20 00 20  00 20 00 20 00 20 00 20  |..... . . . . . |
000009f0  00 20 00 20 00 20 00 20  00 20 00 20 00 74 00 68  |. . . . . . .t.h|
00000a00  00 69 00 73 00 2e 00 53  00 63 00 6f 00 70 00 65  |.i.s...S.c.o.p.e|
00000a10  00 20 00 3d 00 20 00 6e  00 65 00 77 00 20 00 54  |. .=. .n.e.w. .T|
00000a20  00 72 00 61 00 6e 00 73  00 61 00 63 00 74 00 69  |.r.a.n.s.a.c.t.i|
00000a30  00 6f 00 6e 00 53 00 63  00 6f 00 70 00 65 00 28  |.o.n.S.c.o.p.e.(|
00000a40  00 29 00 3b 00 0d 00 0d  0a 00 0d 00 0d 0a 00 20  |.).;........... |
00000a50  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
00000a60  00 20 00 20 00 20 00 74  00 68 00 69 00 73 00 2e  |. . . .t.h.i.s..|
00000a70  00 74 00 65 00 73 00 74  00 49 00 74 00 65 00 6d  |.t.e.s.t.I.t.e.m|
00000a80  00 20 00 3d 00 20 00 6e  00 65 00 77 00 20 00 56  |. .=. .n.e.w. .V|
00000a90  00 69 00 65 00 77 00 44  00 6f 00 63 00 75 00 6d  |.i.e.w.D.o.c.u.m|
00000aa0  00 65 00 6e 00 74 00 48  00 69 00 73 00 74 00 6f  |.e.n.t.H.i.s.t.o|
00000ab0  00 72 00 79 00 28 00 20  00 0d 00 0d 0a 00 20 00  |.r.y.(. ...... .|
00000ac0  20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00  | . . . . . . . .|
00000ad0  20 00 20 00 20 00 20 00  20 00 20 00 20 00 36 00  | . . . . . . .6.|
00000ae0  32 00 35 00 30 00 31 00  36 00 2c 00 20 00 0d 00  |2.5.0.1.6.,. ...|
00000af0  0d 0a 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |... . . . . . . |
00000b00  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
00000b10  00 20 00 35 00 37 00 39  00 39 00 36 00 2c 00 20  |. .5.7.9.9.6.,. |
00000b20  00 0d 00 0d 0a 00 20 00  20 00 20 00 20 00 20 00  |...... . . . . .|
00000b30  20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00  | . . . . . . . .|
00000b40  20 00 20 00 20 00 27 00  54 00 27 00 2c 00 20 00  | . . .'.T.'.,. .|
00000b50  0d 00 0d 0a 00 20 00 20  00 20 00 20 00 20 00 20  |..... . . . . . |
00000b60  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
00000b70  00 20 00 20 00 33 00 34  00 38 00 33 00 35 00 32  |. . .3.4.8.3.5.2|
00000b80  00 39 00 2c 00 20 00 0d  00 0d 0a 00 20 00 20 00  |.9.,. ...... . .|
00000b90  20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00  | . . . . . . . .|
00000ba0  20 00 20 00 20 00 20 00  20 00 20 00 44 00 61 00  | . . . . . .D.a.|
00000bb0  74 00 65 00 54 00 69 00  6d 00 65 00 2e 00 50 00  |t.e.T.i.m.e...P.|
00000bc0  61 00 72 00 73 00 65 00  28 00 22 00 32 00 30 00  |a.r.s.e.(.".2.0.|
00000bd0  30 00 33 00 2d 00 30 00  31 00 2d 00 30 00 38 00  |0.3.-.0.1.-.0.8.|
00000be0  20 00 30 00 39 00 3a 00  35 00 37 00 3a 00 30 00  | .0.9.:.5.7.:.0.|
00000bf0  34 00 2e 00 39 00 35 00  37 00 22 00 29 00 2c 00  |4...9.5.7.".).,.|
00000c00  0d 00 0d 0a 00 20 00 20  00 20 00 20 00 20 00 20  |..... . . . . . |
00000c10  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
00000c20  00 20 00 20 00 33 00 2c  00 20 00 0d 00 0d 0a 00  |. . .3.,. ......|
00000c30  20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00  | . . . . . . . .|
*
00000c50  22 00 49 00 6e 00 76 00  6f 00 69 00 63 00 65 00  |".I.n.v.o.i.c.e.|
00000c60  20 00 28 00 50 00 47 00  2d 00 50 00 53 00 29 00  | .(.P.G.-.P.S.).|
00000c70  20 00 2d 00 20 00 53 00  55 00 50 00 50 00 28 00  | .-. .S.U.P.P.(.|
00000c80  31 00 31 00 2f 00 31 00  36 00 2f 00 32 00 30 00  |1.1./.1.6./.2.0.|
00000c90  30 00 38 00 29 00 22 00  2c 00 20 00 0d 00 0d 0a  |0.8.).".,. .....|
00000ca0  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
*
00000cc0  00 22 00 46 00 49 00 4e  00 41 00 4e 00 43 00 45  |.".F.I.N.A.N.C.E|
00000cd0  00 22 00 2c 00 20 00 0d  00 0d 0a 00 20 00 20 00  |.".,. ...... . .|
00000ce0  20 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00  | . . . . . . . .|
00000cf0  20 00 20 00 20 00 20 00  20 00 20 00 44 00 61 00  | . . . . . .D.a.|
00000d00  74 00 65 00 54 00 69 00  6d 00 65 00 2e 00 50 00  |t.e.T.i.m.e...P.|
00000d10  61 00 72 00 73 00 65 00  28 00 22 00 32 00 30 00  |a.r.s.e.(.".2.0.|
00000d20  30 00 38 00 2d 00 30 00  34 00 2d 00 31 00 31 00  |0.8.-.0.4.-.1.1.|
00000d30  20 00 31 00 31 00 3a 00  31 00 35 00 3a 00 30 00  | .1.1.:.1.5.:.0.|
00000d40  37 00 2e 00 37 00 37 00  30 00 22 00 29 00 2c 00  |7...7.7.0.".).,.|
00000d50  20 00 0d 00 0d 0a 00 20  00 20 00 20 00 20 00 20  | ...... . . . . |
00000d60  00 20 00 20 00 20 00 20  00 20 00 20 00 20 00 20  |. . . . . . . . |
00000d70  00 20 00 20 00 20 00 40  00 22 00 5c 00 5c 00 44  |. . . .@.".\.\.D|
00000d80  00 4f 00 53 00 4b 00 5c  00 43 00 47 00 44 00 4f  |.O.S.K.\.C.G.D.O|
00000d90  00 43 00 53 00 5c 00 33  00 37 00 34 00 39 00 31  |.C.S.\.3.7.4.9.1|
00000da0  00 34 00 5c 00 34 00 36  00 30 00 34 00 36 00 5c  |.4.\.4.6.0.4.6.\|
00000db0  00 62 00 69 00 6c 00 73  00 6c 00 69 00 70 00 5f  |.b.i.l.s.l.i.p._|
00000dc0  00 31 00 30 00 38 00 31  00 36 00 34 00 38 00 2e  |.1.0.8.1.6.4.8..|
00000dd0  00 64 00 6f 00 63 00 22  00 29 00 3b 00 0d 00 0d  |.d.o.c.".).;....|
00000de0  0a 00 20 00 20 00 20 00  20 00 20 00 20 00 20 00  |.. . . . . . . .|
00000df0  20 00 7d 00 0d 00 0d 0a  00 0d 00 0d 0a 00 20 00  | .}........... .|

看来这是某种编码问题。

1 个答案:

答案 0 :(得分:3)

您将文件保存为UTF-16,Windows文本编辑器错误地称之为“Unicode”的编码。

UTF-16与ASCII不兼容,因此无法与git使用的diff工具一起正常工作。你得到的是每个换行符输入的单字节更改(可能是由于LF和Windows CRLF行结束之间的转换)导致UTF-16代码单元的双字节对齐超出一个,导致低要交换的字节和高字节:

original text:               <     s     u     m     m     a     r     y     >
representation in UTF-16LE:  3C 00 73 00 75 00 6D 00 6D 00 61 00 72 00 79 00 3E 00
accidentally misaligned:     00 3C 00 73 00 75 00 6D 00 6D 00 61 00 72 00 79 00 3E
decoded from misaligned:     㰀    猀    甀    洀    洀    愀    爀    礀    㸀

以ASCII兼容的编码保存文件,您将不会遇到此问题。最好是:UTF-8-without-BOM。