将双端队列保存到csv数据框中

时间:2019-01-14 00:02:23

标签: python regex deque saving-data

我正在使用opencv跟踪元素视频(基本上是在hsv阈值化之后计算元素的数量)。我有一个deque缓冲区来存储质心位置。我选择了64的有限缓冲区(以30 fps的速度约2秒,可能会更长)。我的目标是以一种以后可以随时使用的格式将数据保存到.csv文件中(请参阅下文)。此外,我正在计算检测到的区域的数量。格式就像

cX  cY  number
444 265   19
444 265   19
444 264   19
444 264   19
...

其中最大元素的cX是X的质心,cY是最大元素的Y的质心,以及检测到的区域数。列命名不是主要目标,尽管这样做会很好。

出于显示目的,我需要将质心设置为tuple。我使用appendleft使它们逐帧增长:

center_points = deque(maxlen=64)
object_number = deque(maxlen=64)
iteration_counter = 1

    while True


        # read video frames..
        # do stuff...
        # get contours
            my_cnts = cv2.findContours(...)
        # get largest object
            c = max(my_cnts, key=cv2.contourArea)
            ((x, y), radius) = cv2.minEnclosingCircle(c)
            M = cv2.moments(c)
            big_center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"]))
# count object number as int name it 'num'

center_points.appendleft(big_center)
object_number.appendleft(num)

现在,当缓冲区已满时,我想将数据保存到文件中:

# Convert to array to save
    # Wait until the iteration number is divisible by the buffer length
    if(iteration_number % 64 == 0):
        print("Saving on iteration..." + str(iteration_number))
        array_to_save = np.array([center_points, object_number]).T


        with open(filename,'a') as outfile:
            np.savetxt(outfile, array_to_save,
                       delimiter=',', fmt='%s')
# Add 1 to the counter
    iteration_number = iteration_number + 1

问题

上面的代码可以工作并编写如下内容:

(444 265) 19
(444 265) 19
(444 264) 19
(444 263) 19

我想做类似np.array(center_points)的事情并将其绑定到object_number。我在尺寸方面遇到了麻烦(例如(64,2)和(64)不兼容)。我尝试过np.appendnp.stack,但是找不到格式化数据的正确方法。

否则,我可以按原样保留代码,但是我想以某种方式摆脱第1列和第2列的括号并保存该对象(尝试在array_to_save上使用正则表达式没有成功)。所有三列均应为数字或保存为字符串,但在稍后阅读时很容易将其检索为数字。

更新

基于我尝试过的评论

array_to_save = np.concatenate([np.array(center_points), object_number[:, None]])
    TypeError: sequence index must be integer, not 'tuple'

我也尝试过

array_to_save = np.concatenate([np.array(center_points), np.array(object_number)[:, None]])
    ValueError: all the input array dimensions except for the concatenation axis must match exactly

2 个答案:

答案 0 :(得分:1)

您可以沿列维度concatenate排列数组,以便从(X, 3)(X, 2)数组中创建(X,)数组。为了准备进行连接,所有数组都需要具有相同数量的维,因此需要向平面数组object_number(X,) -> (X, 1)添加额外的维。这可以通过object_number[:, np.newaxis]object_number[:, None]完成。完整的解决方案是:

np.concatenate([np.array(center_points),
                np.array(object_number)[:, None]], axis=-1)

答案 1 :(得分:0)

我认为,部分困难在于np.savetxt()在以numpy数组保存的元组中不能很好地工作。我已经开发了一些测试代码,我认为它们可以复制问题的关键方面并为他们提供解决方案:

import numpy as np
from collections import deque

# Create test data
center_points = deque(maxlen=64)
number = deque(maxlen=64)
for i in range(10):
    big_center = (i*3,i*100)
    center_points.appendleft(big_center)
    number.appendleft(19)

# Write the test data

array_to_save = np.array([center_points,number]).T
print (array_to_save)
with open("test.txt","w") as outfile:
    outfile.write("\n".join([" ".join([str(a[0]),str(a[1]),str(b)]) for a,b in
        array_to_save]))

# Re-read the test data
center_points2 = deque(maxlen=64)
number2 = deque(maxlen=64)

with open("test.txt","r") as infile:
    for line in infile:
        x = [int(xx) for xx in line.split()]
        center_points2.append((x[0],x[1]))
        number2.append(x[2])
    new_array = np.array([center_points2,number2]).T

print (new_array)

运行时,此代码输出以下内容,表明原始的array_to_save与已读回的new_array相同:

[[(27, 900) 19]
 [(24, 800) 19]
 [(21, 700) 19]
 [(18, 600) 19]
 [(15, 500) 19]
 [(12, 400) 19]
 [(9, 300) 19]
 [(6, 200) 19]
 [(3, 100) 19]
 [(0, 0) 19]]
[[(27, 900) 19]
 [(24, 800) 19]
 [(21, 700) 19]
 [(18, 600) 19]
 [(15, 500) 19]
 [(12, 400) 19]
 [(9, 300) 19]
 [(6, 200) 19]
 [(3, 100) 19]
 [(0, 0) 19]]

文件test.txt如下:

27 900 19
24 800 19
21 700 19
18 600 19
15 500 19
12 400 19
9 300 19
6 200 19
3 100 19
0 0 19

此版本中的文件读写代码比仅调用np.savetxt()稍微复杂一点,但是它显式地处理元组。

更新

或者,如果您希望在numpy数组中进行所有操作,则可以使用:

import numpy as np
from collections import deque

# Create test data
center_points = deque(maxlen=64)
number = deque(maxlen=64)
for i in range(10):
    big_center = (i*3,i*100)
    center_points.appendleft(big_center)
    number.appendleft(19)

print (center_points)
print (number)

# Write the test data

x, y = zip(*center_points)
array_to_save = np.array([x,y,number]).T
print (array_to_save)
np.savetxt("test.txt", array_to_save, fmt="%d")

# Re-read the test data
new_array = np.loadtxt("test.txt", dtype=int)
print (new_array)

center_points2 = deque(zip(new_array.T[0],new_array.T[1]),maxlen=64)
number2 = deque(new_array.T[2],maxlen=64)
print (center_points2)
print (number2)

这使用Transpose/Unzip Function (inverse of zip)?中描述的方法将每个元组的两个元素分成两个列表,然后将它们与number列表一起包含到一个单独的numpy数组中,可以保存savetxt()并重新加载loadtxt()

print()调用只是为了说明程序完成时所用的数据与开始时所用的数据完全相同。它们产生以下输出:

deque([(27, 900), (24, 800), (21, 700), (18, 600), (15, 500), (12, 400), (9, 300), (6, 200), (3, 100), (0, 0)], maxlen=64)
deque([19, 19, 19, 19, 19, 19, 19, 19, 19, 19], maxlen=64)
[[ 27 900  19]
 [ 24 800  19]
 [ 21 700  19]
 [ 18 600  19]
 [ 15 500  19]
 [ 12 400  19]
 [  9 300  19]
 [  6 200  19]
 [  3 100  19]
 [  0   0  19]]
[[ 27 900  19]
 [ 24 800  19]
 [ 21 700  19]
 [ 18 600  19]
 [ 15 500  19]
 [ 12 400  19]
 [  9 300  19]
 [  6 200  19]
 [  3 100  19]
 [  0   0  19]]
deque([(27, 900), (24, 800), (21, 700), (18, 600), (15, 500), (12, 400), (9, 300), (6, 200), (3, 100), (0, 0)], maxlen=64)
deque([19, 19, 19, 19, 19, 19, 19, 19, 19, 19], maxlen=64)