我刚刚开始使用hadoop,而且我正在使用Avro(fastavro)。
1-我想验证架构并转换为.avro文件。
{
"type": "record",
"name": "Node",
"fields": [
{
"name": "nom",
"type": "string"
},
{
"name": "zone",
"type": {
"type": "map",
"values": "string"
}
},
{
"name": "price",
"type": "float"
},
{
"name": "type",
"type": "string"
}
]
}
我的测试文件(验证架构):
#!/usr/local/bin/python
# -*- coding: utf-8 -*-
import json
import fastavro
schema = json.load(open("myschema.avsc"))
records = [
{
"nom": "blabla",
"zone": ["north", "south", "east"],
"prix": 4.0,
"type": "geoloc"
}
]
fastavro.writer(open("myschema.avro", "wb"), schema, records)
我发现了这个错误:
Traceback (most recent call last):
File "test-schema.py", line 17, in <module>
fastavro.writer(open("myschema.avro", "wb"), schema, records)
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 614, in writer
output.write(record)
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 537, in write
write_data(self.io, record, self.schema)
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 432, in write_data
return fn(fo, datum, schema)
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 363, in write_record
name, field.get('default')), field['type'])
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 432, in write_data
return fn(fo, datum, schema)
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 232, in write_map
for key, val in iteritems(datum):
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/six.py", line 27, in py3_iteritems
return obj.items()
AttributeError: 'list' object has no attribute 'items'
2-如果我添加一个数组:
{
"name": "ingredients",
"type": ["string"]
},
错误:
File "/var/www/data-machine/HDFS/env/lib/python3.5/site-packages/fastavro/writer.py", line 345, in write_union
raise ValueError(msg)
ValueError: ["north", "south", "east"] (type <class 'list'>) do not match ['string']
并且,完成后,我想制作&#34;区域&#34; field optionnal ...
谢谢:) 和Fabrice
答案 0 :(得分:1)
您的地图记录信息有误。它期待像
这样的东西"zone":{"key1":"val1","key2":"val2","key3":"val3"},
它是一张地图,而不是一套。如果你想要类似你的例子,你需要使用数组而不是地图