长期数据采集

时间:2012-07-16 20:25:13

标签: python matplotlib typeerror nonetype

我正在尝试运行一个Python脚本,用于长期实验室数据采集(约1个月)。下面的脚本应该从SQL数据库中获取数据并从DAQ设备获取数据并将其保存到文本文件中。然后它实时绘制1小时的数据。实时绘图已被注释掉,因为我已经得到了内存错误,但这不是我关心的问题。

运行代码3-4天会给我以下错误:

Trackback (most recent call last):
 file *** line 105 in <module>
    deltay=float(float(tupline[3]/10-float(tupline2[3])/10)
TypeError: 'NoneType' object has to attribute '__getitem__'

关于如何让这个脚本运行更长时间的任何想法?

import ctypes
from UniversalLibrary import *
import time
import datetime
from numpy import *
from scipy import *
import MySQLdb as mdb
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import random



boardNum=0
portNum=10
inputVoltage = 3



BoardNum = 0
Gain = BIP5VOLTS
Chan = 0



spTimes =14*24*60*60     ##here you set the time
timeDuration=spTimes
padsToVisualize =[0,1,2,3,4]  ##pads that will be measured and visualized

#plt.ion()    ##this command is necassary in order to have new points on the plot without erasing the old ones

curTime=time.clock()
movingWinDur=60    ##seconds
print "curtime=",curTime

str_one_line=[]
straverage=[]
while (curTime < timeDuration):
    del str_one_line[:]
    padCounter=0;

    d=datetime.datetime.now()
    str_one_line=[(d.strftime("%Y-%m-%d %H:%M:%S")),',']   ##formatting the date -- this format is chosen based on the requirements for connecting to SQL database

    while (padCounter <len(padsToVisualize)):

        #fig1=plt.figure(padsToVisualize[padCounter])    ## making figures for plotting different pads


        print "pad No.: ", str(padsToVisualize[padCounter])
        #l=cbAOut(BoardNum,Chan,Gain,3019)   ## sets the voltage to 3.0 volts (the number 3019 is based on 12-bit resolution --> DAQ resolution)
        #n=cbDConfigPort(BoardNum,portNum,1)   ##the variable n serves aas a command line where the command is sent to the DAQ
        #b=cbDOut(BoardNum,portNum,(63-padsToVisualize[padCounter])) #####based on Prof. Mayer's solution, the line sends the pad number that we are interested in to the DAQ to get the signals 
        curTime=time.clock()
        d=datetime.datetime.now()
        #########################################################################

        ########## this part will use the datetime to fetch the temperature from the SQL data base
        ########## since the time stamps may not exactly match, the code uses a method of interpolation
        freq = 5
        time.sleep(1.8*freq) ## 1.8 is an arbitrary number ; to make sure there is data on sql database
        searchfor=d   ## the following lines find the temperature at time=searchfor
        print searchfor
        minussearchfor = searchfor-datetime.timedelta(0,freq/1.2)
        STRminussearchfor = minussearchfor.strftime("%Y-%m-%d %H:%M:%S")
        print "STRminussearchfor=", STRminussearchfor
        print "minussearchfor= ", minussearchfor
        plussearchfor =searchfor+datetime.timedelta(0,freq/1.2)
        print "plussearchfor= ", plussearchfor
        STRplussearchfor = plussearchfor.strftime("%Y-%m-%d %H:%M:%S")
        print "STRplussearchfor=", STRplussearchfor
        ##Database connection
        db = mdb.connect("localhost", "root", "qwerty@12345", "bondtest")
        cur = db.cursor()
        cur.execute("SELECT * FROM bondtest_data WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime",{'date1':STRminussearchfor,'date2':STRplussearchfor})
##        con=pymssql.connect(host='LAB10-PC\SQLEXPRESS2008R2',user='sa',password='qwerty@12345')
##        cur = con.cursor()
##        cur.execute('SELECT * FROM OVEN11SQL.dbo.OvenLog1 WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime',{'date1':STRminussearchfor,'date2':STRplussearchfor})
        tupline1 = cur.fetchone()
##        print 'between1= ',tupline1[1]
        delta = tupline1[1]-searchfor
##        print "delta = " ,delta
        if (int(searchfor.strftime("%S"))-int(tupline1[1].strftime("%S"))>0):
            delta = int(searchfor.strftime("%S"))-int(tupline1[1].strftime("%S"))
        else:
            delta = int(tupline1[1].strftime("%S"))-int(searchfor.strftime("%S"))
##        print 'delta= ',delta
        time1=tupline1[1]-datetime.timedelta(0,freq/1.2)
        STRtime1=time1.strftime("%Y-%m-%d %H:%M:%S")
        time2=tupline1[1]-datetime.timedelta(0,3*freq/1.2)
        STRtime2=time2.strftime("%Y-%m-%d %H:%M:%S")
##        time.sleep(2*freq) ##the program needs to wait for 3*frequency/2 so that the next point is available in SQL data base for interpolation
        cur.execute('SELECT * FROM bondtest_data WHERE DateTime BETWEEN %(date1)s AND %(date2)s ORDER BY DateTime',{'date1':STRtime2,'date2':STRtime1})
        tupline2 = cur.fetchone()   ##next point is fetched in order to find the slope of the line for temperature change
##        print 'between2= ', tupline2[1]
        deltay=float(float(tupline1[3])/10-float(tupline2[3])/10)
        deltax = int(tupline1[1].strftime("%S"))-int(tupline2[1].strftime("%S"))
        deltax = freq
##        print "deltay= ", deltay
##        print "deltax= ", deltax
        slope = deltay/deltax
##        print 'slope= ', slope

        ##in the following if statements depending on whether the temperature is increasing or decreasing the temperature for the desired point will be calculated
        if (tupline2[3]<tupline1[3]):
            tempsearchfor = float(tupline1[3])/10+delta*slope
##            print '+delta*slope= ',delta*slope
        elif (tupline2[3]>tupline1[3]):
            tempsearchfor = float(tupline1[3])/10-delta*slope
##            print '-delta*slope= ',delta*slope
        else:
            tempsearchfor = float(tupline1[3])/10
##        print 'tempserachfor= ',tempsearchfor

        #########################################################################

        strng = [str(int(padsToVisualize[padCounter])),',',(d.strftime("%Y-%m-%d %H:%M:%S")),',',str(round(curTime,4)),',',str(inputVoltage),',']
        str_one_line.extend(strng)         ##str_one_line is the line that contains the values that will be written to the text file , the order is specified in the variables lists excel file
        xyzCounter=Chan
        EngUnits= array(range(50*1),dtype=float).reshape(50,1)    ## constructing the array that will hold the 50 values for each set of signals and is later used for finding the average
        average = array(range(3*1),dtype=float).reshape(3,1)      ## holds the average of the t50 points for x,y,z
##        straverage=array(range(3*1),dtype=str).reshape(3,1)   ##this array will hold the strings for x,y,z signal to be written into txtfile
        del straverage[:]
        while(xyzCounter<3):   ##the way the oven is set up it has x,y,z outputs from channel 0,1,2 accordingly
            #n=cbDConfigPort(BoardNum,portNum,1)
            #b=cbDOut(BoardNum,portNum,(63-padsToVisualize[padCounter]))     #####based on Prof. Mayer's solution, the line sends the pad number that we are interested in to the DAQ to get the signals
            a=0
            time1=time.clock()
            while (a<50):    ## this while loop takes 50 measurements and writes the average of those points in to average array

                #DataValue=cbAIn(BoardNum, xyzCounter, Gain)
                #EngUnits[a,0]=float((cbToEngUnits(BoardNum,Gain,DataValue))/3/100*1000)
                EngUnits[a,0] = random.uniform(0,0.5)
                average[xyzCounter,0]=float(sum(EngUnits))/len(EngUnits)
                a+=1
            time2=time.clock()
            timePerSample=(time2-time1)/50
            print "time per sample= ",timePerSample
            print "samples per second =", (1/timePerSample)   ##measuring the frequency of the emasurements
            tempstr=str(round((average[xyzCounter,0]),3))    ##in order to remove the two brackets temp refers to temporary
            #tempstr=tempstr[1:-1]
            straverage.append(tempstr)
            xyzCounter+=1


        #print average
        temperaturearray=array(range(1*1),dtype=str).reshape(1,1)
        temperaturearray=[str(tempsearchfor)]
        three_sens_signals=array(range(1*5),dtype=str).reshape(1,5)
        three_sens_signals=[str((straverage[0])),',',str((straverage[1])),',',str((straverage[2])),',']
        str_one_line.extend(three_sens_signals)
        str_one_line.extend(temperaturearray)
        str_one_line.extend(',')

        padCounter+=1
        filename='log_simulation.txt'
        f = open(filename,"a")
      ## writing to file section
    print "padcounter=",padCounter,"   str_one_line=", str_one_line
    for item in str_one_line:  
        f.write("%s" % item )
    f.write("\n")
    f.close()

    curTime=time.clock()

2 个答案:

答案 0 :(得分:1)

简而言之,在这段代码中有很多东西需要改进/重新编写(首先编辑所有import *语句并使用像你应该的命名空间)。话虽如此(代码显然大部分按原样工作),但问题出现的原因是存储在tupline或tupline2中的值因任何原因而为None。在代码中,使用sql语句

分配这些变量
tupline = cur.fetchone()
tupline2 = cur.fetchone() 

其中一个调用显然没有顺利运行(无论出于何种原因,数据或表已损坏),导致它返回“None”。也许有人正在删除或更换表格?您可以添加一些语义来检查此案例,报告并尝试继续。

的内容
if tupline == None or tupline2 == None:
    print "Why does tupline = None?"
    continue

你应该知道sql .fetchone()方法何时返回none。真的,你应该做一些更好的日志记录,但这可能会让你进入下一步调试。

答案 1 :(得分:0)

正如Paul Seeb所指出的,当代码到达tupline行时,会出现错误消息,因为tupline2Nonedeltay=...。您的代码(成功)在到达tupline行之前访问deltay的元素。所以问题必须是tupline2

我的猜测是数据库不包含tupline2查询的匹配记录;这肯定是要检查的东西。我还注意到,您在tupline2查询之前注释了一个延迟,提示此查询无法可靠地返回记录。我不知道你的程序逻辑,但你可以在tupline2查询之后用这样的行修复问题:

if tupline2 is None:
    continue`

或者您可以重试tupline2查询,直到成功

while True:
    cur.execute(
        """SELECT * FROM bondtest_data 
            WHERE DateTime BETWEEN %(date1)s AND %(date2)s 
            ORDER BY DateTime""",
        {'date1':STRtime2,'date2':STRtime1}
    )
    tupline2 = cur.fetchone()
    if tupline2 is None:
        print "Failed to obtain a record for tupline2; trying again..."
        time.sleep(<some duration>)
    else:
        break

我还注意到,每次进行循环时,您的代码都会创建一个新的数据库连接(db)并且从不显式关闭它。我建议将db = mdb.connect(...)命令移到主循环开头之上,或者在最后db.close()命令之后的某处添加fetchone()。您的代码很可能达到某种连接限制,但这可能会引发一个不同的异常。当db被重新分配并且旧连接超出范围时,您更有可能从某种自动关闭连接中受益。