os.walk造成内核恐慌

时间：2013-04-23 17:15:50

标签： python macos python-2.7 osx-mountain-lion

根据

，我正在尝试走的目录中有大约400k个文件 
$ find . -type f | wc -l

每次我运行使用os.walk的脚本时，它都会将我的笔记本电脑发送到内核恐慌中。这非常一致。

OSX Mountain Lion，Python 2.7.4。

有什么想法吗？

编辑：示例代码

import os


def collect_files_for_path(path):
    all_files = []

    for root, dirs, files in os.walk(path):

        for f in files:
            full = os.path.join(root, f)
            all_files.append(full)

    return all_files


files = collect_files_for_path('some/path')

编辑2 ：恐慌日志

Tue Apr 23 11:17:22 2013
panic(cpu 0 caller 0xffffff800d4ee9a5): "hfs_lock: locking against myself!"@/SourceCache/xnu/xnu-2050.22.13/bsd/hfs/hfs_cnode.c:1751
Backtrace (CPU 0), Frame : Return Address
0xffffff813aca3390 : 0xffffff800d21d626
0xffffff813aca3400 : 0xffffff800d4ee9a5
0xffffff813aca3430 : 0xffffff800d518563
0xffffff813aca34b0 : 0xffffff800d3127ff
0xffffff813aca3500 : 0xffffff800d309001
0xffffff813aca3540 : 0xffffff800d32584c
0xffffff813aca35f0 : 0xffffff800d325ee5
0xffffff813aca3780 : 0xffffff800d511279
0xffffff813aca37b0 : 0xffffff800d4fc953
0xffffff813aca3830 : 0xffffff800d312a21
0xffffff813aca38c0 : 0xffffff800d2daa5e
0xffffff813aca39b0 : 0xffffff800d312a7e
0xffffff813aca39e0 : 0xffffff800d2dc68c
0xffffff813aca3a10 : 0xffffff800d2dc7e7
0xffffff813aca3a30 : 0xffffff800d5177d0
0xffffff813aca3a80 : 0xffffff800d312410
0xffffff813aca3ab0 : 0xffffff800d2eac05
0xffffff813aca3b50 : 0xffffff800d2ea1a4
0xffffff813aca3c10 : 0xffffff800d304a32
0xffffff813aca3d90 : 0xffffff800d2fd40c
0xffffff813aca3f50 : 0xffffff800d5e063a
0xffffff813aca3fb0 : 0xffffff800d2cdd23

BSD process name corresponding to current thread: Python

Mac OS version:
12D78

Kernel version:
Darwin Kernel Version 12.3.0: Sun Jan  6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64
Kernel UUID: 3EB7D8A7-C2D3-32EC-80F4-AB37D61492C6
Kernel slide:     0x000000000d000000
Kernel text base: 0xffffff800d200000
System model name: MacBookPro8,1 (Mac-94245B3640C91C81)

System uptime in nanoseconds: 299018314244
last loaded kext at 77719537176: org.virtualbox.kext.VBoxNetAdp 4.2.1 (addr 0xffffff7f8ef2b000, size 28672)
last unloaded kext at 182522584862: com.apple.driver.AppleUSBUHCI   5.2.5 (addr 0xffffff7f8dbf3000, size 65536)
loaded kexts:
org.virtualbox.kext.VBoxNetAdp  4.2.1
org.virtualbox.kext.VBoxNetFlt  4.2.1
org.virtualbox.kext.VBoxUSB 4.2.1
org.virtualbox.kext.VBoxDrv 4.2.1
org.pqrs.driver.PCKeyboardHack  8.0.0
org.pqrs.driver.KeyRemap4MacBook    8.0.0
com.radiosilenceapp.nke.PrivateEye  1
com.Logitech.Unifying.HID Driver    1.2.0
com.Logitech.Control Center.HID Driver  3.5.1
com.apple.filesystems.msdosfs   1.8
com.apple.driver.AppleHWSensor  1.9.5d0
com.apple.iokit.IOBluetoothSerialManager    4.1.3f3
com.apple.driver.AudioAUUC  1.60
com.apple.filesystems.autofs    3.0
com.apple.driver.AGPM   100.12.87
com.apple.driver.AppleMikeyHIDDriver    122
com.apple.driver.AppleHDA   2.3.7fc4
com.apple.iokit.IOUserEthernet  1.0.0d1
com.apple.driver.AppleMikeyDriver   2.3.7fc4
com.apple.driver.AppleLPC   1.6.0
com.apple.driver.AppleUpstreamUserClient    3.5.10
com.apple.Dont_Steal_Mac_OS_X   7.0.0
com.apple.driver.ApplePolicyControl 3.3.0
com.apple.driver.ACPI_SMC_PlatformPlugin    1.0.0
com.apple.driver.AppleIntelHD3000Graphics   8.1.0
com.apple.iokit.BroadcomBluetoothHCIControllerUSBTransport  4.1.3f3
com.apple.driver.AppleSMCLMU    2.0.3d0
com.apple.driver.AppleSMCPDRC   1.0.0
com.apple.driver.AppleIntelSNBGraphicsFB    8.1.0
com.apple.driver.AppleBacklight 170.2.5
com.apple.driver.AppleMCCSControl   1.1.11
com.apple.driver.SMCMotionSensor    3.0.3d1
com.apple.driver.AppleUSBTCButtons  237.1
com.apple.driver.AppleUSBTCKeyboard 237.1
com.apple.driver.AppleIRController  320.15
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 34
com.apple.iokit.SCSITaskUserClient  3.5.5
com.apple.driver.XsanFilter 404
com.apple.iokit.IOAHCIBlockStorage  2.3.1
com.apple.driver.AppleUSBHub    5.5.5
com.apple.driver.AirPort.Brcm4331   614.20.16
com.apple.iokit.AppleBCM5701Ethernet    3.6.0b1
com.apple.driver.AppleFWOHCI    4.9.6
com.apple.driver.AppleSDXC  1.4.0
com.apple.driver.AppleUSBEHCI   5.5.0
com.apple.driver.AppleAHCIPort  2.5.1
com.apple.driver.AppleSmartBatteryManager   161.0.0
com.apple.driver.AppleEFINVRAM  1.7
com.apple.driver.AppleIntelCPUPowerManagementClient 196.0.0
com.apple.driver.AppleACPIButtons   1.7
com.apple.driver.AppleRTC   1.5
com.apple.driver.AppleHPET  1.8
com.apple.driver.AppleSMBIOS    1.9
com.apple.driver.AppleACPIEC    1.7
com.apple.driver.AppleAPIC  1.6
com.apple.nke.applicationfirewall   4.0.39
com.apple.security.quarantine   2
com.apple.driver.AppleIntelCPUPowerManagement   196.0.0
com.apple.iokit.IOSerialFamily  10.0.6
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 2.3.7fc4
com.apple.iokit.IOAudioFamily   1.8.9fc11
com.apple.kext.OSvKernDSPLib    1.6
com.apple.iokit.IOSurface   86.0.4
com.apple.iokit.IOBluetoothFamily   4.1.3f3
com.apple.driver.AppleHDAController 2.3.7fc4
com.apple.iokit.IOHDAFamily 2.3.7fc4
com.apple.iokit.IOFireWireIP    2.2.5
com.apple.driver.IOPlatformPluginLegacy 1.0.0
com.apple.driver.AppleSMBusPCI  1.0.11d0
com.apple.iokit.AppleBluetoothHCIControllerUSBTransport 4.1.3f3
com.apple.driver.IOPlatformPluginFamily 5.3.0d51
com.apple.driver.AppleGraphicsControl   3.3.0
com.apple.driver.AppleBacklightExpert   1.0.4
com.apple.iokit.IONDRVSupport   2.3.7
com.apple.driver.AppleSMBusController   1.0.11d0
com.apple.iokit.IOGraphicsFamily    2.3.7
com.apple.driver.AppleSMC   3.1.4d2
com.apple.iokit.IOSCSIBlockCommandsDevice   3.5.5
com.apple.iokit.IOUSBMassStorageClass   3.5.1
com.apple.driver.AppleUSBMultitouch 237.3
com.apple.driver.AppleThunderboltDPInAdapter    1.8.9
com.apple.driver.AppleThunderboltDPAdapterFamily    1.8.9
com.apple.driver.AppleThunderboltPCIDownAdapter 1.2.6
com.apple.iokit.IOUSBHIDDriver  5.2.5
com.apple.driver.CoreStorage    296.16
com.apple.driver.AppleUSBMergeNub   5.5.5
com.apple.driver.AppleUSBComposite  5.2.5
com.apple.iokit.IOSCSIMultimediaCommandsDevice  3.5.5
com.apple.iokit.IOBDStorageFamily   1.7
com.apple.iokit.IODVDStorageFamily  1.7.1
com.apple.iokit.IOCDStorageFamily   1.7.1
com.apple.iokit.IOAHCISerialATAPI   2.5.1
com.apple.iokit.IOSCSIArchitectureModelFamily   3.5.5
com.apple.driver.AppleThunderboltNHI    1.6.3
com.apple.iokit.IOThunderboltFamily 2.2.6
com.apple.iokit.IOUSBUserClient 5.5.5
com.apple.iokit.IO80211Family   522.4
com.apple.iokit.IOEthernetAVBController 1.0.2b1
com.apple.iokit.IONetworkingFamily  3.0
com.apple.iokit.IOFireWireFamily    4.5.5
com.apple.iokit.IOAHCIFamily    2.3.1
com.apple.iokit.IOUSBFamily 5.5.5
com.apple.driver.AppleEFIRuntime    1.7
com.apple.iokit.IOHIDFamily 1.8.1
com.apple.iokit.IOSMBusFamily   1.1
com.apple.security.sandbox  220.2
com.apple.kext.AppleMatch   1.0.0d1
com.apple.security.TMSafetyNet  7
com.apple.driver.DiskImages 345
com.apple.iokit.IOStorageFamily 1.8
com.apple.driver.AppleKeyStore  28.21
com.apple.driver.AppleACPIPlatform  1.7
com.apple.iokit.IOPCIFamily 2.7.3
com.apple.iokit.IOACPIFamily    1.4
com.apple.kec.corecrypto    1.0

3 个答案:

答案 0 :(得分：3)

你不应该只是通过os.walk来恐慌系统。

你可能应该fsck你的文件系统（卸载时）并检查磁盘是否有坏块。

答案 1 :(得分：2)

如果find很乐意“走”目录，那么问题似乎更有可能是在内存中构建那个巨大的列表（all_files），这可能很容易占用4-8GB RAM。

如果删除行...

，问题是否仍然存在？ 
all_files.append(full)

...是否有任何理由需要构建一个完整的列表，而不是使用os.walk()返回的迭代器对该列表执行任何操作？

<强>更新

  

单独打印路径会使系统陷入内核恐慌。

很奇怪。如果在打印相同的文件名后总是失败，那么可能会为您提供导致问题的文件的线索。

否则，os.walk()的{​​{3}}非常简单，并且是用纯Python编写的，所以如果你在新文件中复制了该代码，并添加了一些调试消息，它可能会有所帮助追查问题。

但最终，没有任何用户进程应该能够引起内核恐慌，使其成为OSX错误，因此您可能有更多的运气联系Apple客户服务 - 他们实际上得到报酬来解决这些问题。 ;-)

答案 2 :(得分：0)

你的python代码没有引起内核恐慌。 Mac上发生内核恐慌的原因很少：

  1. 硬件问题
  2. 操作系统本身的错误
  3. 已安装的内核驱动程序中的错误

    4. Python没有安装内核驱动程序，因此即使它是一致的，根本原因也是你的python代码之下的东西。

    我在你的恐慌痕迹中看到了虚拟盒。那可能是你的问题。这是一个描述如何阅读恐慌日志的页面：http://www.embracingchaos.com/2011/05/macbook-crashes-kernel-panics-and-coping-with-an-apple-genius.html

相关问题
最新问题