上下文
游戏的远程公共API大约每五秒生成300000行JSON数据。此数据在下次更新后立即消失,永远不会再次访问。我希望能够保存这些数据的压缩历史记录,这些记录可以可靠,高效地编制索引和搜索。
问题:
考虑到可能的压缩方法以及最重要的查询效率,在这个规模上存储JSON数据的最佳数据库系统是什么?
编辑添加示例解压缩的JSON(大约30000个中的一个对象):
{
"accountName": "Kabancheg11",
"lastCharacterName": "KabanNeedBuff",
"id": "427d66d977b5a63af08b3c39173a1a567eb783fde6d6f97df505dbe6c1fa5988",
"stash": "~b/o 30 chaos",
"stashType": "PremiumStash",
"items": [
{
"verified": false,
"w": 1,
"h": 1,
"ilvl": 71,
"icon": "http://web.poecdn.com/image/Art/2DItems/Amulets/Amulet37.png?scale=1&w=1&h=1&v=25bebeac11abfd9f7931d574765521093",
"league": "Hardcore",
"id": "a7bb36a28e55865b9d766dcebd5bc0c6893d5ecb6cd9870dc0bcd3c9f1164f29",
"sockets": [],
"name": "",
"typeLine": "Onyx Amulet",
"identified": false,
"corrupted": false,
"lockedToCharacter": false,
"implicitMods": [
"+15 to all Attributes"
],
"frameType": 3,
"x": 11,
"y": 0,
"inventoryId": "Stash1",
"socketedItems": []
},
{
"verified": false,
"w": 1,
"h": 3,
"ilvl": 75,
"icon": "http://web.poecdn.com/image/Art/2DItems/Weapons/OneHandWeapons/Daggers/Heartbreaker.png?scale=1&w=1&h=3&v=b4f18d5c0602a17381df6a45b4c57be33",
"league": "Hardcore",
"id": "c955f054ffae9439e5d257c4a444b02423b2dbe304d1fa581227d21a0f203aa9",
"sockets": [
{
"group": 0,
"attr": "D"
},
{
"group": 0,
"attr": "D"
}
],
"name": "<<set:MS>><<set:M>><<set:S>>Heartbreaker",
"typeLine": "Royal Skean",
"identified": true,
"corrupted": false,
"lockedToCharacter": false,
"properties": [
{
"name": "Dagger",
"values": [],
"displayMode": 0
},
{
"name": "Physical Damage",
"values": [
[
"15-59",
0
]
],
"displayMode": 0
},
{
"name": "Critical Strike Chance",
"values": [
[
"6.60%",
0
]
],
"displayMode": 0
},
{
"name": "Attacks per Second",
"values": [
[
"1.45",
0
]
],
"displayMode": 0
}
],
"requirements": [
{
"name": "Level",
"values": [
[
"50",
0
]
],
"displayMode": 0
},
{
"name": "Dex",
"values": [
[
"71",
0
]
],
"displayMode": 1
},
{
"name": "Int",
"values": [
[
"102",
0
]
],
"displayMode": 1
}
],
"implicitMods": [
"40% increased Global Critical Strike Chance"
],
"explicitMods": [
"45% increased Spell Damage",
"+50 to maximum Mana",
"+50 to maximum Energy Shield",
"10% faster start of Energy Shield Recharge",
"Your Spells have Culling Strike"
],
"flavourText": [
"A heart can be stabbed by thoughts, \r",
"If your mind is sharp enough."
],
"frameType": 3,
"x": 0,
"y": 9,
"inventoryId": "Stash2",
"socketedItems": []
},
{
"verified": false,
"w": 2,
"h": 3,
"ilvl": 75,
"icon": "http://web.poecdn.com/image/Art/2DItems/Weapons/OneHandWeapons/Scepters/MontregulsGrasp.png?scale=1&w=2&h=3&v=fed985831c32e8e25b7663608a2b7a5c3",
"league": "Hardcore",
"id": "a78e6bfff62145c78b92714cc7de10e4484c42745d48b585ef905b3972c2a319",
"sockets": [
{
"group": 0,
"attr": "S"
},
{
"group": 0,
"attr": "S"
}
],
"name": "",
"typeLine": "Void Sceptre",
"identified": false,
"corrupted": false,
"lockedToCharacter": false,
"properties": [
{
"name": "One Handed Mace",
"values": [],
"displayMode": 0
},
{
"name": "Physical Damage",
"values": [
[
"42-63",
0
]
],
"displayMode": 0
},
{
"name": "Critical Strike Chance",
"values": [
[
"6.50%",
0
]
],
"displayMode": 0
},
{
"name": "Attacks per Second",
"values": [
[
"1.25",
0
]
],
"displayMode": 0
}
],
"requirements": [
{
"name": "Str",
"values": [
[
"104",
0
]
],
"displayMode": 1
},
{
"name": "Int",
"values": [
[
"122",
0
]
],
"displayMode": 1
}
],
"implicitMods": [
"15% increased Elemental Damage"
],
"frameType": 3,
"x": 0,
"y": 6,
"inventoryId": "Stash3",
"socketedItems": []
},
{
"verified": false,
"w": 2,
"h": 4,
"ilvl": 74,
"icon": "http://web.poecdn.com/image/Art/2DItems/Weapons/TwoHandWeapons/TwoHandAxes/TwoHandAxe11Unique.png?scale=1&w=2&h=4&v=49f4f7aff220d356fc6300d7454df63a3",
"league": "Hardcore",
"id": "c8d722c909939d1fda4431722c732e7af4c17781d9d9c58da613bf776ea1e993",
"sockets": [
{
"group": 0,
"attr": "D"
}
],
"name": "",
"typeLine": "Superior Karui Chopper",
"identified": false,
"corrupted": false,
"lockedToCharacter": false,
"properties": [
{
"name": "Two Handed Axe",
"values": [],
"displayMode": 0
},
{
"name": "Quality",
"values": [
[
"+10%",
1
]
],
"displayMode": 0
},
{
"name": "Physical Damage",
"values": [
[
"88-138",
1
]
],
"displayMode": 0
},
{
"name": "Critical Strike Chance",
"values": [
[
"5.00%",
0
]
],
"displayMode": 0
},
{
"name": "Attacks per Second",
"values": [
[
"1.15",
0
]
],
"displayMode": 0
}
],
"requirements": [
{
"name": "Str",
"values": [
[
"151",
0
]
],
"displayMode": 1
},
{
"name": "Dex",
"values": [
[
"43",
0
]
],
"displayMode": 1
}
],
"frameType": 3,
"x": 2,
"y": 0,
"inventoryId": "Stash4",
"socketedItems": []
},
{
"verified": false,
"w": 2,
"h": 3,
"ilvl": 51,
"icon": "http://web.poecdn.com/image/Art/2DItems/Armours/BodyArmours/StrInt3AUnique2.png?scale=1&w=2&h=3&v=3ef1aa6bc50b635ca334653ca95485a63",
"league": "Hardcore",
"id": "26c55af00fb31f63bec7229b622c3c79fb6efcfa03d3d2595672a92778d04d99",
"sockets": [
{
"group": 0,
"attr": "I"
},
{
"group": 1,
"attr": "I"
}
],
"name": "<<set:MS>><<set:M>><<set:S>>Ambu's Charge",
"typeLine": "Crusader Chainmail",
"identified": true,
"corrupted": false,
"lockedToCharacter": false,
"properties": [
{
"name": "Armour",
"values": [
[
"477",
1
]
],
"displayMode": 0
},
{
"name": "Energy Shield",
"values": [
[
"141",
1
]
],
"displayMode": 0
}
],
"requirements": [
{
"name": "Level",
"values": [
[
"43",
0
]
],
"displayMode": 0
},
{
"name": "Str",
"values": [
[
"64",
0
]
],
"displayMode": 1
},
{
"name": "Int",
"values": [
[
"64",
0
]
],
"displayMode": 1
}
],
"explicitMods": [
"182% increased Armour and Energy Shield",
"+15% to all Elemental Resistances",
"Gain an Endurance Charge when you take a Critical Strike",
"2% of Life Regenerated per second while on Low Life",
"Share Endurance Charges with nearby party members"
],
"flavourText": [
"Nothing stops the pain like a courageous rush into battle."
],
"frameType": 3,
"x": 0,
"y": 0,
"inventoryId": "Stash5",
"socketedItems": []
},
{
"verified": false,
"w": 1,
"h": 1,
"ilvl": 64,
"icon": "http://web.poecdn.com/image/Art/2DItems/Jewels/basicint.png?scale=1&w=1&h=1&v=cd579ea22c05f1c6ad2fd015d7a710bd3",
"league": "Hardcore",
"id": "34103c553f0ae3e36c695664a7540dd6e547338596c33554ca3a8443dd5349c1",
"sockets": [],
"name": "<<set:MS>><<set:M>><<set:S>>Cataclysm Heart",
"typeLine": "Cobalt Jewel",
"identified": true,
"corrupted": false,
"lockedToCharacter": false,
"explicitMods": [
"2% increased Cast Speed",
"7% increased maximum Energy Shield",
"+13% to Lightning Resistance"
],
"descrText": "Place into an allocated Jewel Socket on the Passive Skill Tree. Right click to remove from the Socket.",
"frameType": 2,
"x": 0,
"y": 3,
"inventoryId": "Stash6",
"socketedItems": []
},
{
"verified": false,
"w": 1,
"h": 1,
"ilvl": 0,
"icon": "http://web.poecdn.com/image/Art/2DItems/Gems/VaalGems/VaalArc.png?scale=1&w=1&h=1&v=b4f32328e279496ebb227521e8dce6793",
"support": false,
"league": "Hardcore",
"id": "be9a23cfe0a53942b19c22da32b7d461b464838749110d04689e15498dcad86b",
"sockets": [],
"name": "",
"typeLine": "Vaal Arc",
"identified": true,
"corrupted": true,
"lockedToCharacter": false,
"properties": [
{
"name": "Vaal, Spell, Chaining, Lightning",
"values": [],
"displayMode": 0
},
{
"name": "Level",
"values": [
[
"20 (Max)",
0
]
],
"displayMode": 0
},
{
"name": "Mana Cost",
"values": [
[
"0",
0
]
],
"displayMode": 0
},
{
"name": "Souls Per Use",
"values": [
[
"32",
0
]
],
"displayMode": 0
},
{
"name": "Can Store %0 Use",
"values": [
[
"1",
0
]
],
"displayMode": 3
},
{
"name": "Cast Time",
"values": [
[
"0.80 sec",
0
]
],
"displayMode": 0
},
{
"name": "Critical Strike Chance",
"values": [
[
"5.00%",
0
]
],
"displayMode": 0
},
{
"name": "Damage Effectiveness",
"values": [
[
"80%",
0
]
],
"displayMode": 0
}
],
"requirements": [
{
"name": "Level",
"values": [
[
"70",
0
]
],
"displayMode": 0
},
{
"name": "Int",
"values": [
[
"155",
0
]
],
"displayMode": 1
}
],
"secDescrText": "An arc of lightning stretches from the caster to a targeted nearby enemy and chains on to many additional targets.",
"explicitMods": [
"Deals 49 to 938 Lightning Damage",
"Chain +40 Times",
"100% chance to Shock enemies"
],
"descrText": "Place into an item socket of the right colour to gain this skill. Right click to remove from a socket.",
"frameType": 4,
"x": 0,
"y": 4,
"inventoryId": "Stash7",
"socketedItems": []
}
],
"public": true
}
示例查询可能会询问嵌套在每个对象中的项目数量(如上所示)有ilvl
字段大于或等于整数。