使用LZ4解压缩时如何知道输出缓冲区何时太小?

时间:2017-02-08 13:42:30

标签: c compression lz4

LZ4_decompress_safe的文档说:

{"variant": [{"product_id": 10065509898,"title": "XS\/ BROWN","price": 49.05,"sku": "BA1015_BROWN_XS","position": 1,"grams": 0,"inventory_policy": "deny","compare_at_price": null,"fulfillment_service": "manual","inventory_management": "shopify","option1": "BROWN","option2": "XS","option3": null,"taxable": true,"barcode": "800123054849","inventory_quantity": 0,"old_inventory_quantity": 0,"inventory_quantity_adjustment": 1,"weight": 1,"weight_unit": "lb","requires_shipping": true},{"product_id": 10065509898,"title": "S\/ BROWN","price": 49.05,"sku": "BA1015_BROWN_S","position": 2,"grams": 0,"inventory_policy": "deny","compare_at_price": null,"fulfillment_service": "manual","inventory_management": "shopify","option1": "BROWN","option2": "S","option3": null,"taxable": true,"barcode": "800123054856","inventory_quantity": 0,"old_inventory_quantity": 0,"inventory_quantity_adjustment": 1,"weight": 1,"weight_unit": "lb","requires_shipping": true},{"product_id": 10065509898,"title": "M\/ BROWN","price": 49.05,"sku": "BA1015_BROWN_M","position": 3,"grams": 0,"inventory_policy": "deny","compare_at_price": null,"fulfillment_service": "manual","inventory_management": "shopify","option1": "BROWN","option2": "M","option3": null,"taxable": true,"barcode": "800123054863","inventory_quantity": 0,"old_inventory_quantity": 0,"inventory_quantity_adjustment": 1,"weight": 1,"weight_unit": "lb","requires_shipping": true},{"product_id": 10065509898,"title": "L\/ BROWN","price": 49.05,"sku": "BA1015_BROWN_L","position": 4,"grams": 0,"inventory_policy": "deny","compare_at_price": null,"fulfillment_service": "manual","inventory_management": "shopify","option1": "BROWN","option2": "L","option3": null,"taxable": true,"barcode": "800123054870","inventory_quantity": 0,"old_inventory_quantity": 0,"inventory_quantity_adjustment": 1,"weight": 1,"weight_unit": "lb","requires_shipping": true},{"product_id": 10065509898,"title": "XL\/ BROWN","price": 49.05,"sku": "BA1015_BROWN_XL","position": 5,"grams": 0,"inventory_policy": "deny","compare_at_price": null,"fulfillment_service": "manual","inventory_management": "shopify","option1": "BROWN","option2": "XL","option3": null,"taxable": true,"barcode": "800123054887","inventory_quantity": 0,"old_inventory_quantity": 0,"inventory_quantity_adjustment": 1,"weight": 1,"weight_unit": "lb","requires_shipping": true},{"product_id": 10065509898,"title": "XXL\/ BROWN","price": 49.05,"sku": "BA1015_BROWN_XXL","position": 6,"grams": 0,"inventory_policy": "deny","compare_at_price": null,"fulfillment_service": "manual","inventory_management": "shopify","option1": "BROWN","option2": "XXL","option3": null,"taxable": true,"barcode": "800123054894","inventory_quantity": 0,"old_inventory_quantity": 0,"inventory_quantity_adjustment": 1,"weight": 1,"weight_unit": "lb","requires_shipping": true}]}

但是没有说明如何区分问题是否是目标缓冲区太小或者输入格式错误/参数组合错误/ ..

如果我不知道目标解压缩大小是什么,我怎么知道我是否应该使用更大的缓冲区重试?

1 个答案:

答案 0 :(得分:2)

有一个issue opened,目前还没有公共API可以区分错误。

作为启发式,查看the code会显示可能的返回值:

    /* end of decoding */
    if (endOnInput)
       return (int) (((char*)op)-dest);     /* Nb of output bytes decoded */
    else
       return (int) (((const char*)ip)-source);   /* Nb of input bytes read */

    /* Overflow error detected */
_output_error:
    return (int) (-(((const char*)ip)-source))-1;

所以只有两种情况:

  • 解码成功,您获得肯定结果(其结果取决于您是处于完全模式还是部分模式)
  • 或解码失败并且您得到否定结果

如果是否定结果,则值为-(position_in_input + 1)

这表明,通过重试(更大)更大的缓冲区并检查故障是否发生在同一位置,可以很好地预测目标缓冲区是否太小:

  • 如果第二次减压尝试成功,那你很好!
  • 如果第二次解压缩尝试在同一位置失败,则问题可能出在输入
  • 否则,你必须再次使用更大的缓冲区。

或以其他方式说,只要结果不同,再试一次,否则,就是你的结果。

<强>限制

输入指针不一定每次前进一个字节,它可以在两个地方提前length个字节,其中length从输入读取并且无界限。

如果由于输出缓冲区太小而导致解码失败,并且新输出缓冲区对于length来说仍然太小,那么即使输入不是(必然)格式错误,解码也会在相同位置失败。 / p>

如果误报是个问题,那么可以尝试:

  • 通过检查返回位置的输入流
  • 来解码length
  • 只需根据Mark Adler's answer分配255 * <input size> - 2526,这对小投入来说是合理的。