我正在用Python编写一个脚本来检测带有文本的图像的偏斜。
以下两个函数在我的本地计算机上运行正常,但由于内存使用量超过512MB(有时达到> 1000MB),导致我的Heroku dyno崩溃。
我使用memory-profiler对脚本进行了分析,结果如下。
对于功能#1 ,尽管分配了大量内存,但在分配内存之后,内存会快速释放,正如我所料,因此很好。
然而功能#2 似乎无缘无故地分配内存。变量coords
用于2行,但其内存永远不会被释放。此外,即使函数#2返回,分配给它的210 MB仍保持“正在使用”。
功能#1:
此功能的内存配置文件(关键区域第52-54行,按预期释放内存):
Line # Mem usage Increment Line Contents
49 395.9 MiB 0.0 MiB @profile
50 def determine_skew(self):
51 413.1 MiB 17.2 MiB img = io.imread(self.image, as_grey=True)
--> 52 552.1 MiB 139.0 MiB edges = canny(img, sigma=self.sigma)
--> 53 554.3 MiB 2.2 MiB h, a, d = hough_line(edges)
--> 54 429.3 MiB -125.1 MiB _, ap, _ = hough_line_peaks(h, a, d, num_peaks=self.num_peaks)
55 429.3 MiB 0.0 MiB if len(ap) == 0:
56 return { "angle" : 0 }
57 429.3 MiB 0.0 MiB absolute_deviations = [self.calculate_deviation(k) for k in ap]
58 429.3 MiB 0.0 MiB average_deviation = np.mean(np.rad2deg(absolute_deviations))
59 429.3 MiB 0.0 MiB ap_deg = [np.rad2deg(x) for x in ap]
60
61 429.3 MiB 0.0 MiB bin_0_45 = []
62 429.3 MiB 0.0 MiB bin_45_90 = []
63 429.3 MiB 0.0 MiB bin_0_45n = []
64 429.3 MiB 0.0 MiB bin_45_90n = []
65 429.3 MiB 0.0 MiB for ang in ap_deg:
66 429.3 MiB 0.0 MiB deviation_sum = int(90 - ang + average_deviation)
67 429.3 MiB 0.0 MiB if self.compare_sum(deviation_sum):
68 429.3 MiB 0.0 MiB bin_45_90.append(ang)
69 429.3 MiB 0.0 MiB continue
70 deviation_sum = int(ang + average_deviation)
71 if self.compare_sum(deviation_sum):
72 bin_0_45.append(ang)
73 continue
74 deviation_sum = int(-ang + average_deviation)
75 if self.compare_sum(deviation_sum):
76 bin_0_45n.append(ang)
77 continue
78 deviation_sum = int(90 + ang + average_deviation)
79 if self.compare_sum(deviation_sum):
80 bin_45_90n.append(ang)
81 429.3 MiB 0.0 MiB angles = [bin_0_45, bin_45_90, bin_0_45n, bin_45_90n]
82
83 429.3 MiB 0.0 MiB lmax = 0
84 429.3 MiB 0.0 MiB for j in range(len(angles)):
85 429.3 MiB 0.0 MiB l = len(angles[j])
86 429.3 MiB 0.0 MiB if l > lmax:
87 429.3 MiB 0.0 MiB lmax = l
88 429.3 MiB 0.0 MiB maxi = j
89 429.3 MiB 0.0 MiB if lmax:
90 429.3 MiB 0.0 MiB ans_arr = self.get_max_freq_elem(angles[maxi])
91 429.3 MiB 0.0 MiB ans_res = np.mean(ans_arr)
92 else:
93 ans_arr = self.get_max_freq_elem(ap_deg)
94 ans_res = np.mean(ans_arr)
95
96 data = {
97 429.3 MiB 0.0 MiB "averageDeviation": average_deviation,
98 429.3 MiB 0.0 MiB "angle": ans_res,
99 429.3 MiB 0.0 MiB "angleBins": angles
100 }
101 429.3 MiB 0.0 MiB return data
功能#2(有问题):
内存配置文件(关键区域为 102-104 行,不正确释放内存):
Line # Mem usage Increment Line Contents
94 181.9 MiB 0.0 MiB @profile
95 def openCVSkewDetect(self):
96 # Grayscale image and flip foreground (foreground is now "white", background is "black")
97 185.7 MiB 3.8 MiB gray = cv2.bitwise_not(self.originalImageOpenCV)
98 # Threshold the image, setting all foreground pixels to 255 and background pixels to 0
99 185.7 MiB 0.1 MiB thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
100 185.7 MiB 0.0 MiB del gray
101 # Grab the (x, y) coordinates of all pixel values > 0, then use these coordinates to compute a rotated bounding box that contains all coordinates
--> 102 325.8 MiB 140.0 MiB coords = np.column_stack(np.where(thresh > 0))
--> 103 325.8 MiB 0.0 MiB del thresh
--> 104 395.9 MiB 70.1 MiB angle = copy.copy(cv2.minAreaRect(coords)[-1])
105 # The `cv2.minAreaRect` function returns values in the range [-90, 0); as the rectangle rotates clockwise the returned angle trends to 0 -- in this special case we need to add 90 degrees to the angle
106 395.9 MiB 0.0 MiB if angle < -45:
107 395.9 MiB 0.0 MiB angle = -(90 + angle)
108 # Otherwise, just take the inverse of the angle to make it positive
109 else:
110 angle = -angle
111 395.9 MiB 0.0 MiB print(gc.get_referents(coords))
112 395.9 MiB 0.0 MiB return angle
我已尝试del
释放内存,copying
以避免重复引用,并从函数#2返回常量,但这些方法都没有效果。
关于为什么第2行中的102-104行拒绝释放内存的任何帮助或解释都将非常感谢!
答案 0 :(得分:0)
我在我的一些代码中尝试了以下方法,因为Python只是为垃圾收集排队内存而不是实际释放它。
# at the start, but it should not matter
import gc
# after freeing memory
gc.collect()