我在尝试从托管代码PInvoke本机调用时遇到System.AccessViolation Exception和Heap块修改过去的请求大小。本机代码是COBOL win32 dll。问题仅出现在.NET 4.0中，并且在3.5中不是问题。
这是PInvoke的VB代码
Declare Ansi Function RunNAS2008 Lib "NAS.DLL" Alias _
"NAS2008" (ByVal f As String, ByVal fExt As String, _
ByVal wa As String, ByVal go As String, _
ByVal c As String, ByVal cExt As String) As Integer`
fExt，wa和cEXT由本机代码更新。
字符串是固定长度的，当符合3.5时，相同的代码不会抛出异常。
这是生成的IL代码
[DllImport("NAS.DLL", CharSet = CharSet.Ansi, EntryPoint = "NAS2008", ExactSpelling = true, SetLastError = true)]
public static extern int RunNAS2008([MarshalAs(UnmanagedType.VBByRefStr)] ref string f, [MarshalAs(UnmanagedType.VBByRefStr)] ref string fExt, [MarshalAs(UnmanagedType.VBByRefStr)] ref string wa, [MarshalAs(UnmanagedType.VBByRefStr)] ref string go, [MarshalAs(UnmanagedType.VBByRefStr)] ref string c, [MarshalAs(UnmanagedType.VBByRefStr)] ref string cExt);
这是例外
HEAP[NAtest.exe]: Heap block at 006FF6E0 modified at 00700241 past requested size of b59
(1be4.d20): Break instruction exception - code 80000003 (first chance)
eax=006ff6e0 ebx=00700241 ecx=77a9a798 edx=001be20d esi=006ff6e0 edi=00000b59
eip=77af0474 esp=001be454 ebp=001be454 iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
ntdll!RtlpBreakPointHeap+0x23:
托管调用堆栈
001be64c 77af0474 [InlinedCallFrame: 001be64c] Microsoft.Win32.Win32Native.CoTaskMemFree(IntPtr)
001be648 5b4e230b System.StubHelpers.VBByValStrMarshaler.ClearNative(IntPtr)*** WARNING: Unable to verify checksum for C:\Windows\assembly\NativeImages_v4.0.30319_32\mscorlib\0b6fafc69f01aa1a982b7f0bc40d48f0\mscorlib.ni.dll
001be688 004a0d47 DomainBoundILStubClass.IL_STUB_PInvoke(System.String ByRef, System.String ByRef, System.String ByRef, System.String ByRef, System.String ByRef, System.String ByRef)
001be690 004a0ab7 [InlinedCallFrame: 001be690] NAtest.Form1.RunNAS2008(System.String ByRef, System.String ByRef, System.String ByRef, System.String ByRef, System.String ByRef, System.String ByRef)
001be738 004a0ab7 NAtest.Form1.Button1_Click(System.Object, System.EventArgs)*** WARNING: Unable to verify checksum for NAtest.exe
001be760 5db34ae8 System.Windows.Forms.Control.OnClick(System.EventArgs)*** WARNING: Unable to verify checksum for C:\Windows\assembly\NativeImages_v4.0.30319_32\System.Windows.Forms\d5f67a0c3cc91bdf34152fe7f752042c\System.Windows.Forms.ni.dll
001be778 5db370a2 System.Windows.Forms.Button.OnClick(System.EventArgs)
001be790 5e0c6174 System.Windows.Forms.Button.OnMouseUp(System.Windows.Forms.MouseEventArgs)
001be7ac 5e0995b5 System.Windows.Forms.Control.WmMouseUp(System.Windows.Forms.Message ByRef, System.Windows.Forms.MouseButtons, Int32)
001be840 5e45a1bf System.Windows.Forms.Control.WndProc(System.Windows.Forms.Message ByRef)
001be844 5e4618dd [InlinedCallFrame: 001be844]
001be898 5e4618dd System.Windows.Forms.ButtonBase.WndProc(System.Windows.Forms.Message ByRef)
001be8dc 5db9de00 System.Windows.Forms.Button.WndProc(System.Windows.Forms.Message ByRef)
001be8e8 5db870f3 System.Windows.Forms.Control+ControlNativeWindow.OnMessage(System.Windows.Forms.Message ByRef)
001be8f0 5db87071 System.Windows.Forms.Control+ControlNativeWindow.WndProc(System.Windows.Forms.Message ByRef)
001be904 5db86fb6 System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)
001bead8 002e0ab8 [InlinedCallFrame: 001bead8]
001bead4 5dba2eec DomainBoundILStubClass.IL_STUB_PInvoke(MSG ByRef)
001bead8 5db971ff [InlinedCallFrame: 001bead8] System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG ByRef)
001beb1c 5db971ff System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr, Int32, Int32)
001beb20 5db96e2c [InlinedCallFrame: 001beb20]
001bebb8 5db96e2c System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)
001bec10 5db96c81 System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)
001bec40 5e08f3d0 System.Windows.Forms.Application.Run(System.Windows.Forms.ApplicationContext)
001bec4c 6e84c53c Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.OnRun()*** WARNING: Unable to verify checksum for C:\Windows\assembly\NativeImages_v4.0.30319_32\Microsoft.VisualBas#\c8cba834d3d1a531d2f00252f4b267cc\Microsoft.VisualBasic.ni.dll
001bec78 6e84c3e8 Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.DoApplicationModel()
001beca4 6e84cc46 Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(System.String[])
001becf0 004a00c5 NAtest.My.MyApplication.Main(System.String[]) [17d14f5c-a337-4978-8281-53493378c1071.vb @ 82]
001bef44 601621bb [GCFrame: 001bef44]
原生呼叫堆栈
0:000> k
ChildEBP RetAddr
001be470 77ab29c0 ntdll!RtlpBreakPointHeap+0x23
001be48c 77af14cf ntdll!RtlpValidateHeapEntry+0x16d
001be4d4 77aaab3a ntdll!RtlDebugFreeHeap+0x9a
001be5c8 77a53472 ntdll!RtlpFreeHeap+0x5d
001be5e8 76ec6e6a ntdll!RtlFreeHeap+0x142
001be5fc 76ec6f54 ole32!CRetailMalloc_Free+0x1c
001be60c 60162543 ole32!CoTaskMemFree+0x13
001be63c 5b4e230b clr!PInvokeStackImbalanceHelper+0x22
001be754 5db34ae8 mscorlib_ni+0x85230b
001be770 5db370a2 System_Windows_Forms_ni+0x1c4ae8
001be788 5e0c6174 System_Windows_Forms_ni+0x1c70a2
001be7a4 5e0995b5 System_Windows_Forms_ni+0x756174
001be830 5e45a1bf System_Windows_Forms_ni+0x7295b5
001be890 5e4618dd System_Windows_Forms_ni+0xaea1bf
001be8d4 5db9de00 System_Windows_Forms_ni+0xaf18dd
001be8e0 5db870f3 System_Windows_Forms_ni+0x22de00
001be8e8 5db87071 System_Windows_Forms_ni+0x2170f3
001be8fc 5db86fb6 System_Windows_Forms_ni+0x217071
001be984 75bd62fa System_Windows_Forms_ni+0x216fb6
001be9b0 75bd6d3a USER32!InternalCallWinProc+0x23
我已启用 PInvokeStackInbalanceMDA 来调试此问题。
IMO这是由IL Stubs生成的System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)指令引起的
这里生成了ILStubs，尤其是对clear native
的调用 Cleanup {
/*( 0)*/ ldloc.0
/*( 1)*/ ldc.i4.0
/*( 2)*/ ble IL_012f
/*( 0)*/ ldloc.2
/*( 1)*/ call void [mscorlib] System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)
IL_012f: /*( 0)*/ ldloc.0
/*( 1)*/ ldc.i4.1
/*( 2)*/ ble IL_013d
/*( 0)*/ ldloc.s 0x5
/*( 1)*/ call void [mscorlib] System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)
IL_013d: /*( 0)*/ ldloc.0
/*( 1)*/ ldc.i4.2
/*( 2)*/ ble IL_014b
/*( 0)*/ ldloc.s 0x8
/*( 1)*/ call void [mscorlib] System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)
IL_014b: /*( 0)*/ ldloc.0
/*( 1)*/ ldc.i4.3
/*( 2)*/ ble IL_0159
/*( 0)*/ ldloc.s 0xb
/*( 1)*/ call void [mscorlib] System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)
IL_0159: /*( 0)*/ ldloc.0
/*( 1)*/ ldc.i4.4
/*( 2)*/ ble IL_0167
/*( 0)*/ ldloc.s 0xe
/*( 1)*/ call void [mscorlib] System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)
IL_0167: /*( 0)*/ ldloc.0
/*( 1)*/ ldc.i4.5
/*( 2)*/ ble IL_0175
/*( 0)*/ ldloc.s 0x11
/*( 1)*/ call void [mscorlib] System.StubHelpers.VBByValStrMarshaler::ClearNative(native int)
IL_0175: /*( 0)*/ endfinally
// } Cleanup
任何帮助都将不胜感激。