使用Python将3D 32位floatarray保存为48位整数PNG以匹配Kitti Ground Truth格式

Question

Kitti有“光流”基准。他们要求流量估算为48位PNG文件，以与他们拥有的地面真实文件的格式匹配。

Ground Truth PNG图片可用于download here

Kitti有一个Matlab DevKit，用于估算和地面真值比较。

我想将网络中的流量输出为48位整数PNG文件，以便可以将我的流量估算值与其他Kitti基准流量估算值进行比较。

来自网络的numpy缩放流文件为downloadable from here

但是，在Python中将float32 3D数组流转换为3通道48位文件（每通道16位）时遇到了问题，因为图像库提供程序似乎对此不提供支持，或者因为我在做我的代码有问题。有人可以帮忙吗？

我尝试了很多不同的库，并阅读了很多文章。

不幸的是，Scipy输出的png仅为24位。 使用scipy available here

生成的输出流量估算值png

# Numpy Flow to 48bit PNG with 16bits per channel

import scipy as sp
from scipy import misc
import numpy as np
import png
import imageio
import cv2
from PIL import Image
from matplotlib import image

"""From Kitti DevKit:-

Optical flow maps are saved as 3-channel uint16 PNG images: The first 
channel
contains the u-component, the second channel the v-component and the 
third
channel denotes if the pixel is valid or not (1 if true, 0 otherwise). To 
convert
the u-/v-flow into floating point values, convert the value to float, 
subtract 2^15 and divide the result by 64.0:"""

Scaled_Flow = np.load('Scaled_Flow.npy') # This is a 32bit float
# This is the very first Kitti Test Flow Output from image_2 testing folder  
# passed through DVF
# The network that produced this flow is only trained to 51 steps, so it 
# won't provide an accurate correspondence
# But the Estimated Flow PNG should look green

ones = np.float32(np.ones((2,375,1242,1))) # Kitti devkit readme says 
that third channel is 1 if flow is valid for that pixel
# 2 for batch size, 3 for height, 3 for width, 1 for this extra layer of 
ones.
with_ones = np.concatenate((Scaled_Flow, ones), axis=3)

im = sp.misc.toimage(with_ones[-1,:,:,:], cmin=-1.0, cmax=1.0) # saves image object
im.save("Scipy_24bit.png", dtype="uint48") # Outputs 24bit only.

Flow = np.int16(with_ones) # An attempt at converting the format from 
float 32 to 16 bit integers
f512 = Flow * 512 # Kitti instructs that the flows are scaled by 512.

x = np.array(Scaled_Flow)
x.astype(np.uint16) # another attempt at converting it to unsigned 16 bit 
integers

try: # try PyPNG
    with open('PyPNGuint48bit.png', 'wb') as f:
        writer = png.Writer(width=375, height=1242, bitdepth=16)
        # Convert z to the Python list of lists expected by
        # the png writer.
        #z2list = x.reshape(-1, x.shape[1]*x.shape[2]).tolist()
        writer.write(f, x)
except:
    print("png lib approach didn't work, it might be to do with the 
sizing")

try: # try imageio
    imageio.imwrite('imageio_Flow_48bit.png', x, format='PNG-FI')
except:
    print("imageio approach didn't work, it probably couldn't handle the 
datatype")

try: # try OpenCV
    cv2.imwrite('OpenCVFlow_48bit_.png',x )
except:
    print("OpenCV approach didn't work, it probably couldn't handle the 
datatype")

try: #try: # try PIL
    im = Image.fromarray(x)
    im.save("PILLOW_Flow_48bit.png", format="PNG")
except:
    print("PILLOW approach didn't work, it probably couldn't handle the 
datatype")

try: # try Matplotlib
    image.imsave('MatplotLib_Flow_48bit.png', x)
except:
    print("Matplotlib approach didn't work, ValueError: object too deep 
for desired array")'''

我想获得一个与Kitti Ground真相相同的48bit png文件， 看起来绿色。目前，Scipy输出的是蓝色的24bit png文件， 看上去很白。

Answer 1

这是我对您想做什么的理解：

1. 从Scaled_Flow.npy加载数据。这是一个形状为（2，375，1242，2）的32位浮点numpy数组。

2. 通过以下方式将Scaled_Flow[1]（形状为（375，1242，2）的数组）转换为16位无符号整数：

   • 乘以64，
   • 添加2**15，并且
   • 将值投射到np.uint16。

   这是您引用的描述的反义词：“要将u- / v-flow转换为浮点值，请将其转换为float，减去2 ^ 15并将结果除以64.0”。

3. 通过将所有1组成一个数组，将三维尺寸的长度从2增加到3。
4. 将结果保存到PNG文件中。

这是您可以执行此操作的一种方法。为了创建PNG文件，我将使用numpngw，这是我编写的用于从numpy数组创建PNG和动画PNG文件的库。如果您给numpngw.write_png一个数据类型为np.uint16的numpy数组，它将创建一个每通道16位的PNG文件（在这种情况下，即为48位图片）。

import numpy as np
from numpngw import write_png


Scaled_Flow = np.load('Scaled_Flow.npy')
sf16 = (64*Scaled_Flow[-1] + 2**15).astype(np.uint16)
imgdata = np.concatenate((sf16, np.ones(sf16.shape[:2] + (1,), dtype=sf16.dtype)), axis=2)

write_png('sf48.png', imgdata)

这是该脚本创建的图像。