我正在尝试检查哪个部分花费了我的python程序的最大内存。使用这篇文章http://practicalpython.blogspot.com/2017/03/monitoring-memory-usage-in-jupyter.html。
代码是:
OPTIONS (SKIP=0, direct=true)
LOAD DATA
INFILE "..\test.csv" BADFILE ".\test.bad" DISCARDFILE ".\test.dis"
INSERT
INTO TABLE table_test
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
cust_ID,
cust_TYPE,
X_DATE BOUNDFILLER,
X_TIME BOUNDFILLER,
cust_date EXPRESSION "TO_DATE(:X_DATE || ' ' || :X_TIME, 'YYYY-MM-DD HH24.MI.SS')"
)
然而,我收到了错误
import numpy as np
import pandas as pd
def show_mem_usage():
'''Displays memory usage from inspection
of global variables in this notebook'''
gl = sys._getframe(1).f_globals
vars= {}
for k,v in list(gl.items()):
# for pandas dataframes
if hasattr(v, 'memory_usage'):
mem = v.memory_usage(deep=True)
if not np.isscalar(mem):
mem = mem.sum()
vars.setdefault(id(v),[mem]).append(k)
# work around for a bug
elif isinstance(v,pd.Panel):
v = v.values
vars.setdefault(id(v),[sys.getsizeof(v)]).append(k)
total = 0
for k,(value,*names) in vars.items():
if value>1e6:
print(names,"%.3fMB"%(value/1e6))
total += value
print("%.3fMB"%(total/1e6))
任何人都知道发生了什么?感谢。