使用自定义验证程序存储重复的KeyProperty时出现BadValueError

时间:2016-06-08 20:27:10

标签: python google-app-engine google-cloud-datastore app-engine-ndb

我试图通过使用自定义验证器来简化ndb KeyProperties的JSON序列化和反序列化,该验证器将字符串转换为相应类型的Key

想法是拥有这样的属性:

def key_validator(kind):
  def validator(prop, value):
    if not isinstance(value, ndb.Key):
      return ndb.Key(kind, value)
    return value
  return validator

class Bar(ndb.Model):

  foo = ndb.KeyProperty('Foo', validator=key_validator('Foo'))

如您所见,验证器将任何字符串转换为给定类型的Key。 目标是能够将包含密钥id的JSON对象传递给populate方法,如下所示:

bar = Bar()
bar.populate(json.loads('{"foo": "1234"}'))

哪个应该有效地做到这一点:

bar = Bar()
bar.foo = ndb.Key("Foo", "1234")

问题是这需要覆盖KeyProperty,因为在执行某些基本验证后调用验证器失败,因为"1234"显然不是Key,请参阅{{ 3}}

为了完成这项工作,我已经创建了一个" ValidationMixin"以及在进行任何其他验证之前调用验证器的新KeyProperty(并且还将Key序列化为id)。

class ValidationMixin(object):
  # make sure to call _validator before we do as the very first validation step
  def _do_validate(self, value):
    if self._validator is not None:
      newvalue = self._validator(self, value)
      if newvalue is not None:
        value = newvalue
    return super(ValidationMixin, self)._do_validate(value)

# A KeyProperty that allows a validator to generate a Key.
# In addition it serializes to just the id of the key
class KeyProperty(ValidationMixin, ndb.KeyProperty):
  # return just the id of the key
  def _get_for_dict(self, entity):
    value = self._get_value(entity)
    if self._repeated:
      return [v.id() for v in value]
    elif value is not None:
      return value.id()
    return value

使用此KeyProperty就像非重复属性的魅力一样。不幸的是,对于具有repeated=True的属性,它会失败。

当我致电bar.populate(json.loads('[{"foo": "1234"}]'))后跟put()

时,会引发以下异常
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3451, in _put
    return self._put_async(**ctx_options).get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 824, in put
    key = yield self._put_batcher.add(entity, options)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 430, in _help_tasklet_along
    value = gen.send(val)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 358, in _put_tasklet
    keys = yield self._conn.async_put(options, datastore_entities)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1852, in async_put
    pbs = [entity_to_pb(entity) for entity in entities]
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 697, in entity_to_pb
    pb = ent._to_pb()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3167, in _to_pb
    prop._serialize(self, pb, projection=self._projection)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1422, in _serialize
    values = self._get_base_value_unwrapped_as_list(entity)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1192, in _get_base_value_unwrapped_as_list
    wrapped = self._get_base_value(entity)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1180, in _get_base_value
    return self._apply_to_values(entity, self._opt_call_to_base_type)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1355, in _apply_to_values
    newvalue = function(value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1234, in _opt_call_to_base_type
    value = _BaseValue(self._call_to_base_type(value))
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1255, in _call_to_base_type
    return call(value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1331, in call
    newvalue = method(self, value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 2013, in _validate
  raise datastore_errors.BadValueError('Expected Key, got %r' % (value,))
BadValueError: Expected Key, got [Key('Foo', '486944fe896a44c689275e6f19e3084a')]

正如您所看到的,它抱怨值是列表而不是单个Key。请注意,put()中的异常不在populate中,因此_set_value执行的初始验证成功。

所以我的问题是,我的方法是破碎还是应该有效?如果它应该工作,为什么它不起作用,如何解决?

更新

根据堆栈跟踪,代码执行通过issue 268,这很奇怪,因为属性被重复,并且应该是model.py, line 1355中的另一个分支

更新2

我刚刚发现,当我从模型中删除未重复的KeyProperty时,它会起作用。看起来序列化被破坏,错误的KeyProperty实例被传递给_seralize方法

1 个答案:

答案 0 :(得分:1)

好的,找到了。 KeyProperty有一个非常奇怪的构造函数“签名魔法”(model.py, line 1963)。

关键是如果第一个参数是一个字符串,它就成了属性的字段名,而不是那种!如果要按字符串指定种类,则必须使用关键字参数,否则kind参数必须是实际类型而不仅仅是名称。如果我错了,请纠正我,但这不是公开文件的一部分。这真的很令人困惑,因为使用ndb.Key,您实际上可以将类型指定为字符串作为第一个位置参数。

实际上,我有3个KeyProperties具有相同的类型,但属性名称不同。但是,由于我将类型指定为字符串,它实际上成了名称。所以这三个属性都使用相同的名称。因此,重复的属性值使用非重复的KeyProperty实例序列化,从而导致此崩溃。

解决方案是使用关键字参数指定种类:

foo = ndb.KeyProperty(kind='Foo', validator=key_validator('Foo'))

从/向JSON序列化KeyProperties现在运行良好。