Python内存使用 - 不能正确释放变量

时间:2017-07-20 23:32:31

标签: python python-3.x opencv memory memory-leaks

我正在用Python编写一个脚本来检测带有文本的图像的偏斜。

以下两个函数在我的本地计算机上运行正常,但由于内存使用量超过512MB(有时达到> 1000MB),导致我的Heroku dyno崩溃。

我使用memory-profiler对脚本进行了分析,结果如下。

对于功能#1 ,尽管分配了大量内存,但在分配内存之后,内存会快速释放,正如我所料,因此很好。

然而功能#2 似乎无缘无故地分配内存。变量coords用于2行,但其内存永远不会被释放。此外,即使函数#2返回,分配给它的210 MB仍保持“正在使用”。

功能#1:

此功能的内存配置文件(关键区域第52-54行,按预期释放内存):

Line #    Mem usage    Increment   Line Contents
49    395.9 MiB      0.0 MiB       @profile
50                                 def determine_skew(self):
51    413.1 MiB     17.2 MiB           img = io.imread(self.image, as_grey=True)
--> 52    552.1 MiB    139.0 MiB       edges = canny(img, sigma=self.sigma)
--> 53    554.3 MiB      2.2 MiB       h, a, d = hough_line(edges)
--> 54    429.3 MiB   -125.1 MiB       _, ap, _ = hough_line_peaks(h, a, d, num_peaks=self.num_peaks)
55    429.3 MiB      0.0 MiB           if len(ap) == 0:
56                                         return { "angle" : 0 }
57    429.3 MiB      0.0 MiB           absolute_deviations = [self.calculate_deviation(k) for k in ap]
58    429.3 MiB      0.0 MiB           average_deviation = np.mean(np.rad2deg(absolute_deviations))
59    429.3 MiB      0.0 MiB           ap_deg = [np.rad2deg(x) for x in ap]
60                             
61    429.3 MiB      0.0 MiB           bin_0_45 = []
62    429.3 MiB      0.0 MiB           bin_45_90 = []
63    429.3 MiB      0.0 MiB           bin_0_45n = []
64    429.3 MiB      0.0 MiB           bin_45_90n = []
65    429.3 MiB      0.0 MiB           for ang in ap_deg:
66    429.3 MiB      0.0 MiB               deviation_sum = int(90 - ang + average_deviation)
67    429.3 MiB      0.0 MiB               if self.compare_sum(deviation_sum):
68    429.3 MiB      0.0 MiB                   bin_45_90.append(ang)
69    429.3 MiB      0.0 MiB                   continue
70                                         deviation_sum = int(ang + average_deviation)
71                                         if self.compare_sum(deviation_sum):
72                                             bin_0_45.append(ang)
73                                             continue
74                                         deviation_sum = int(-ang + average_deviation)
75                                         if self.compare_sum(deviation_sum):
76                                             bin_0_45n.append(ang)
77                                             continue
78                                         deviation_sum = int(90 + ang + average_deviation)
79                                         if self.compare_sum(deviation_sum):
80                                             bin_45_90n.append(ang)
81    429.3 MiB      0.0 MiB           angles = [bin_0_45, bin_45_90, bin_0_45n, bin_45_90n]
82                                     
83    429.3 MiB      0.0 MiB           lmax = 0
84    429.3 MiB      0.0 MiB           for j in range(len(angles)):
85    429.3 MiB      0.0 MiB               l = len(angles[j])
86    429.3 MiB      0.0 MiB               if l > lmax:
87    429.3 MiB      0.0 MiB                   lmax = l
88    429.3 MiB      0.0 MiB                   maxi = j
89    429.3 MiB      0.0 MiB           if lmax:
90    429.3 MiB      0.0 MiB               ans_arr = self.get_max_freq_elem(angles[maxi])
91    429.3 MiB      0.0 MiB               ans_res = np.mean(ans_arr)
92                                     else:
93                                         ans_arr = self.get_max_freq_elem(ap_deg)
94                                         ans_res = np.mean(ans_arr)
95                             
96                                     data = {
97    429.3 MiB      0.0 MiB               "averageDeviation": average_deviation,
98    429.3 MiB      0.0 MiB               "angle": ans_res,
99    429.3 MiB      0.0 MiB               "angleBins": angles
100                                         }
101    429.3 MiB      0.0 MiB           return data

功能#2(有问题):

内存配置文件(关键区域为 102-104 行,正确释放内存):

Line #    Mem usage    Increment   Line Contents
94    181.9 MiB      0.0 MiB       @profile
95                                 def openCVSkewDetect(self):
96                                     # Grayscale image and flip foreground (foreground is now "white", background is "black")
97    185.7 MiB      3.8 MiB           gray = cv2.bitwise_not(self.originalImageOpenCV)
98                                     # Threshold the image, setting all foreground pixels to 255 and background pixels to 0
99    185.7 MiB      0.1 MiB           thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
100    185.7 MiB      0.0 MiB          del gray
101                                    # Grab the (x, y) coordinates of all pixel values > 0, then use these coordinates to compute a rotated bounding box that contains all coordinates
--> 102    325.8 MiB    140.0 MiB      coords = np.column_stack(np.where(thresh > 0))
--> 103    325.8 MiB      0.0 MiB      del thresh
--> 104    395.9 MiB     70.1 MiB      angle = copy.copy(cv2.minAreaRect(coords)[-1])
105                                    # The `cv2.minAreaRect` function returns values in the range [-90, 0); as the rectangle rotates clockwise the returned angle trends to 0 -- in this special case we need to add 90 degrees to the angle
106    395.9 MiB      0.0 MiB          if angle < -45:
107    395.9 MiB      0.0 MiB              angle = -(90 + angle)
108                                    # Otherwise, just take the inverse of the angle to make it positive
109                                    else:
110                                        angle = -angle
111    395.9 MiB      0.0 MiB          print(gc.get_referents(coords))
112    395.9 MiB      0.0 MiB          return angle

我已尝试del释放内存,copying以避免重复引用,并从函数#2返回常量,但这些方法都没有效果。

关于为什么第2行中的102-104行拒绝释放内存的任何帮助或解释都将非常感谢!

1 个答案:

答案 0 :(得分:0)

我在我的一些代码中尝试了以下方法,因为Python只是为垃圾收集排队内存而不是实际释放它。

# at the start, but it should not matter
import gc 

# after freeing memory
gc.collect()