Question

I have a rather unique case where my Python program generates a large amount of dynamic code (such as dynamically generated classes and lambdas), and it needs to save its state with Pickle. I'd like to preserve as much as possible, but it's acceptable to simply fail to preserve objects that are unpickleable.

In other words, I'd like to be able to pickle the dict mydict, which contains some unpickleable objects, by simply executing del mydict[unpickleablekey] for every unpickleable object.

What's the simplest and most Pythonic way to implement this?

Answer 1

I ran into a similar issue when trying to serialize JSON with instances that could not be pickled. I wrote a custom JSON encoder which may help in your case.

from json import JSONEncoder
class JsonEncoder(JSONEncoder):
    def default(self, obj):
        return repr(obj)

Then you would use it like so:

encoded_dict = JsonEncoder().encode(mydict)
with open('dump.pkl', 'w') as f:
    pickle.dump(encoded_dict, f)

To read it back:

import json
with open('dump.pkl', 'r') as f:
    decoded_dict = json.loads(pickle.load(f))

One thing to note is that by using the repr to encode values will cause things like integer or float values will be converted to a string representation and not the true type. The default encoder can be enhanced to only return the repr if the type is not int, float, etc...

Ignore unpickleable objects when pickling

1 个答案: