Question

我的模块包含一个应该是可腌制的类，实例和定义我有以下结构：

MyModule
|-Submodule
  |-MyClass

在关于SO的其他问题中，我已经发现dill能够腌制类定义，并且确实可以通过将MyClass的定义复制到单独的脚本中并在其中进行腌制来使其工作，例如：

import dill as pickle

class MyClass(object):
    ...

instance = MyClass(...)
with open(..., 'wb') as file:
   pickle.dump(instance, file)

但是，在导入类时它不起作用：

酸洗：

from MyModule.Submodule import MyClass
import dill as pickle

instance = MyClass(...)
with open(.., 'wb') as file:
    pickle.dump(instance, file)

正在加载：

import dill as pickle

with open(..., 'rb') as file:
    instance = pickle.load(file)

>>> ModuleNotFoundError: No module named 'MyModule'

我认为类定义是通过引用保存的，尽管它不应该像莳萝中的默认设置那样具有引用。当MyClass被称为__main__.MyClass时，可以正确完成此操作，这是在主脚本中定义了类时发生的。

我想知道，有什么方法可以将MyClass与MyModule分离吗？有什么方法可以使其像顶级导入（__main__.MyClass）一样使莳萝知道如何将其加载到我的另一台计算机上？

相关问题： Why dill dumps external classes by reference no matter what

Answer 1

我是dill的作者。这是您上面提到的问题的重复。相关的GitHub功能请求为：https://github.com/uqfoundation/dill/issues/128。

我认为更大的问题是，您希望对未安装的另一个文件中定义的对象进行腌制。我相信，目前尚不可能。

作为一种解决方法，我相信您可以通过提取类（或模块）的源代码并动态地对其进行酸洗，或者提取源代码并在{ {1}}。

Answer 2

我使用以下肮脏的技巧成功保存了类的实例和定义：

    class DataCaptureService : Service() {

        private var isServiceStarted = false

        override fun onBind(intent: Intent?): IBinder? {
            return null
        }

override fun onCreate() {
        super.onCreate()
        wakeLock = (getSystemService(Context.POWER_SERVICE) as PowerManager).run {
                    newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "WakelockTag123").apply {
                        acquire()
                    }
                }
    }


        override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
            val serviceAction = intent?.action
            LogUtils.logD("OnStartCommand(). Action=$serviceAction")
            if (Constants.INTENT.ACTION_STOP_SERVICE == serviceAction) {
                LogUtils.logD("Stopping data capture service")
                stopForeground(true)
                stopSelf()
            } else if (Constants.INTENT.ACTION_START_SERVICE == serviceAction && !isServiceStarted) {
                LogUtils.logD("Starting data capture service")
                isServiceStarted = true
                // Here showing notification using a utility method (startForeground(id, notification))
                createNotification(this)
                // Doing some stuff here
                ----------------------
                //
            }
            return START_STICKY
        }

        override fun onDestroy() {
            super.onDestroy()
            if (isServiceStarted) {
                LogUtils.logD("onDestroy of DataCaptureService method is invoked")
                // Doing some stuff here
                ----------------------
                //
                isServiceStarted = false
                if (wakeLock.isHeld) {
                    wakeLock.release()
                }
            }
        }

        override fun onTaskRemoved(rootIntent: Intent?) {
            LogUtils.logD("onTaskRemoved of DataCaptureService method is invoked")
            ensureServiceStaysRunning()
            super.onTaskRemoved(rootIntent)
        }

        private fun ensureServiceStaysRunning() {
            val restartAlarmInterval = 60 * 1000
            val resetAlarmTimer = 30 * 1000L
            // From this broadcast I am restarting the service
            val restartIntent = Intent(this, ServiceRestartBroadcast::class.java)
            restartIntent.action = "RestartedViaAlarm"
            restartIntent.flags = Intent.FLAG_RECEIVER_FOREGROUND
            val alarmMgr = getSystemService(Context.ALARM_SERVICE) as AlarmManager
            val restartServiceHandler = @SuppressLint("HandlerLeak")
            object : Handler() {
                override fun handleMessage(msg: Message) {
                    val pendingIntent = PendingIntent.getBroadcast(applicationContext, 87, restartIntent, PendingIntent.FLAG_CANCEL_CURRENT)
                    val timer = System.currentTimeMillis() + restartAlarmInterval
                    val sdkInt = Build.VERSION.SDK_INT
                    if (sdkInt < Build.VERSION_CODES.KITKAT)
                        alarmMgr.set(AlarmManager.RTC_WAKEUP, timer, pendingIntent)
                    else if (Build.VERSION_CODES.KITKAT <= sdkInt && sdkInt < Build.VERSION_CODES.M)
                        alarmMgr.setExact(AlarmManager.RTC_WAKEUP, timer, pendingIntent)
                    else if (sdkInt >= Build.VERSION_CODES.M) {
                        alarmMgr.setExactAndAllowWhileIdle(AlarmManager.RTC_WAKEUP, timer, pendingIntent)
                    }
                    sendEmptyMessageDelayed(0, resetAlarmTimer)
                    stopSelf()
                }
            }
            restartServiceHandler.sendEmptyMessageDelayed(0, 0)
        }

    }

使用class MyClass(object): def save(path): import __main__ with open(__file__) as f: code = compile(f.read(), "somefile.py", 'exec') globals = __main__.__dict__ locals = {'instance': self, 'savepath': path} exec(code, globals, locals) if __name__ == '__main__': # Script is loaded in top level, MyClass is now available under the qualname '__main__.MyClass' import dill as pickle # copy the attributes of the 'MyModule.Submodule.MyClass' instance to a bew 'MyClass' instance. new_instance = MyClass.__new__(MyClass) new_instance.__dict__ = locals()['instance'].__dict__ with open(locals()['savepath'], 'wb') as f: pickle.dump(new_instance, f)语句可以在exec内部执行文件，因此也将保存类定义。不使用保存功能，不应将此脚本作为主脚本执行。

Answer 3

Dill实际上仅将对象的定义存储在__main__中，而不是将其存储在模块中，因此解决此问题的一种方法是在main中重新定义这些对象：

def mainify(obj):
    import __main__
    import inspect
    import ast

    s = inspect.getsource(obj)
    m = ast.parse(s)
    co = compile(m, '<string>', 'exec')
    exec(co, __main__.__dict__)

然后：

from MyModule.Submodule import MyClass
import dill as pickle

mainify(MyClass)
instance = MyClass(...)
with open(.., 'wb') as file:
    pickle.dump(instance, file)

现在，即使没有MyModule.Submodule的地方，您也应该可以从任何地方装载泡菜。

带莳萝的模块中的Pickle类定义

3 个答案: