Python 2.7 pyLZMA工作原理,Python 3.4 LZMA模块没有

时间:2015-09-25 17:39:37

标签: python compression lzma

import sys
import os
import zlib

try:
    import pylzma as lzma
except ImportError:
    import lzma

from io import StringIO
import struct

#-----------------------------------------------------------------------------------------------------------------------

def read_ui8(c):
    return struct.unpack('<B', c)[0]
def read_ui16(c):
    return struct.unpack('<H', c)[0]
def read_ui32(c):
    return struct.unpack('<I', c)[0]

def parse(input):
    """Parses the header information from an SWF file."""
    if hasattr(input, 'read'):
        input.seek(0)
    else:
        input = open(input, 'rb')

    header = { }

    # Read the 3-byte signature field
    header['signature'] = signature = b''.join(struct.unpack('<3c', input.read(3))).decode()

    # Version
    header['version'] = read_ui8(input.read(1))

    # File size (stored as a 32-bit integer)
    header['size'] = read_ui32(input.read(4))

    # Payload

    if header['signature'] == 'FWS':
        print("The opened file doesn't appear to be compressed")
        buffer = input.read(header['size'])
    elif header['signature'] == 'CWS':
        print("The opened file appears to be compressed with Zlib")
        buffer = zlib.decompress(input.read(header['size']))
    elif header['signature'] == 'ZWS':
        print("The opened file appears to be compressed with Lzma")
        # ZWS(LZMA)
        # | 4 bytes       | 4 bytes    | 4 bytes       | 5 bytes    | n bytes    | 6 bytes         |
        # | 'ZWS'+version | scriptLen  | compressedLen | LZMA props | LZMA data  | LZMA end marker |
        size = read_ui32(input.read(4))
        buffer = lzma.decompress(input.read())

    # Containing rectangle (struct RECT)

    # The number of bits used to store the each of the RECT values are
    # stored in first five bits of the first byte.

    nbits = read_ui8(buffer[0]) >> 3

    current_byte, buffer = read_ui8(buffer[0]), buffer[1:]
    bit_cursor = 5

    for item in 'xmin', 'xmax', 'ymin', 'ymax':
        value = 0
        for value_bit in range(nbits-1, -1, -1): # == reversed(range(nbits))
            if (current_byte << bit_cursor) & 0x80:
                value |= 1 << value_bit
            # Advance the bit cursor to the next bit
            bit_cursor += 1

            if bit_cursor > 7:
                # We've exhausted the current byte, consume the next one
                # from the buffer.
                current_byte, buffer = read_ui8(buffer[0]), buffer[1:]
                bit_cursor = 0

        # Convert value from TWIPS to a pixel value
        header[item] = value / 20

    header['width'] = header['xmax'] - header['xmin']
    header['height'] = header['ymax'] - header['ymin']

    header['frames'] = read_ui16(buffer[0:2])
    header['fps'] = read_ui16(buffer[2:4])

    input.close()
    return header

header = parse(sys.argv[1]);

print('SWF header')
print('----------')
print('Version:      %s' % header['version'])
print('Signature:    %s' % header['signature'])
print('Dimensions:   %s x %s' % (header['width'], header['height']))
print('Bounding box: (%s, %s, %s, %s)' % (header['xmin'], header['xmax'], header['ymin'], header['ymax']))
print('Frames:       %s' % header['frames'])
print('FPS:          %s' % header['fps'])

我的印象是内置的python 3.4 LZMA模块与Python 2.7 pyLZMA模块的工作原理相同。 我提供的代码在2.7和3.4上运行,但是当它在3.4上运行时(它没有pylzma所以它转向内置的lzma)我得到以下错误:

_lzma.LZMAError: Input format not supported by decoder

为什么pylzma可以工作但是Python 3.4的lzma却没有?

1 个答案:

答案 0 :(得分:3)

虽然我没有回答为什么这两个模块的工作方式不同,但我确实有一个解决方案。

我无法让非流LZMA lzma.decompress工作,因为我对LZMA / XZ / SWF规范知之甚少,但我让lzma.LZMADecompressor工作。为了完整起见,我认为SWF LZMA使用此标题格式(未100%确认):

Bytes  Length  Type  Endianness  Description
 0- 2  3       UI8   -           SWF Signature: ZWS
 3     1       UI8   -           SWF Version
 4- 7  4       UI32  LE          SWF FileLength aka File Size

 8-11  4       UI32  LE          SWF? Compressed Size (File Size - 17)

12     1       -     -           LZMA Decoder Properties
13-16  4       UI32  LE          LZMA Dictionary Size
17-    -       -     -           LZMA Compressed Data (including rest of SWF header)

然而,LZMA文件格式规范说它应该是:

Bytes  Length  Type  Endianness  Description
 0     1       -     -           LZMA Decoder Properties
 1- 4  4       UI32  LE          LZMA Dictionary Size
 5-12  8       UI64  LE          LZMA Uncompressed Size
13-    -       -     -           LZMA Compressed Data

我永远无法真正了解Uncompressed Size应该是什么(如果甚至可以为此格式定义)。 pylzma似乎并不关心这一点,而Python 3.3 lzma确实如此。但是,似乎显式未知大小有效,可以指定为UI64,其值为2^64,例如8*b'\xff'8*'\xff',所以通过稍微改变标题而不是使用:

buffer = lzma.decompress(input.read())

尝试:

d = lzma.LZMADecompressor(format=lzma.FORMAT_ALONE)
buffer = d.decompress(input.read(5) + 8*b'\xff' + input.read())

注意:我没有可用的本地python3解释器,所以只通过略微修改的读取程序在线测试,因此它可能无法开箱即用。

编辑:确认在python3中工作但是有些事情需要改变,比如Marcus提到的解压缩(使用buffer[0:1]而不是buffer[0]轻松解决)。读取整个文件并不是必需的,一个小块,比如256字节应该可以读取整个SWF头。 frames字段也有点古怪,但我相信你所要做的就是有点转移,即:

header['frames'] = read_ui16(buffer[0:2]) >> 8

SWF file format spec

LZMA file format spec