Django - 什么可能导致数据库挂起?

时间:2016-04-15 12:02:58

标签: django postgresql

我已经设计了一个使用Django框架的本地网络网站,最近遇到了我直到最近才遇到的问题。 我们正在本地网络上进行实验,收集各种测量值,我建立了这个网站,以确保我们在同一个地方收集所有数据。

我设置了一个PostGreSQL数据库并使用django在接收测量值时动态填充它。执行该操作的脚本如下所示:

**ladrLogger.py**
#various imports
import django
from django.db import IntegrityError
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
django.setup()
from logger.models import Measurement, Device, Type , Room, Experiment, ExperimentData

def logDevice(self,port):
    # Callback function executed each time I receive data to log it in the database
    deviceData = port.data   # get the data
    # Do a bunch of tests and checks
    # ....
    # Get all measurement to add to the database
    # measurements is a list of measurement as defined in my django models
    measurements = self.prepareMeasurement(...)
    self.saveMeasurements(measurements)
    print "Saved measurements successfully."

def saveMeasurements(self,meas):
    if not meas:
        return
    elif type(meas) is list:
        for m in meas:
            self.saveMeasurements(m)
    elif type(meas) is Measurement:
        try:    
            meas.save()
        except IntegrityError as e:
            if 'unique constraint' in e.message:
                print "Skipping... Measurement already existed for device " + meas.device.name
            else:
                print "Skipping measurement due to error: " + e.message

def prepareMeasurement(self,nameDevice, typeDevice, time, data):
    ### Takes the characteristics of measurement (device, name and type) and creates the appropriate measurements.
    measurements = []
    m = Measurement()
    m.device = Device.objects.get(name=nameDevice)
    m.date = time
    # Bunch of tests
    # ..... 
    for idv,v in enumerate(value):
        if v in data:
            m = Measurement()
            m.device = something
            m.date = something else
            m.value = bla
            m.quantity = blabla
            measurements.append(m)
    return measurements

# Bunch of other methods

请注意,此脚本始终在运行,并等待执行logDevice回调的更多测量。

编辑:基于YARP的基于自定义的库负责回调处理。创建回调的代码如下所示:

portid = self.createPort(quer.group(1),True,True) #creates a port  
pyarp.connect(desc[0], self.fullPortPath(portid)) #establishes connection to talking port 
self.listenToPort(portid, lambda port: self.logDevice(port)) #tells him to execute that callback when he receives messages'   

回调完全在后台处理。

另一方面,我有我的django网站,有各种视图显示设备,测量,绘图等等。 我遇到的问题是我正在记录我的测量结果(有时候每秒大约几(2-3)次,通常更少)我可以看到记录似乎很好。但是当我调用我的观点时,例如询问设备x的最新测量值,我得到一个旧的测量值。代码的一个例子:

def latestTemp(request,device_id):
    # Creates a csv file with the latest temperature measured
    #### for now does not check what measurements are actually available
    dev = get_object_or_404(Device, pk=device_id)
    tz = pytz.timezone('Europe/Zurich')
    # Create the HttpResponse object with the appropriate CSV header.
    response = HttpResponse(content_type='text/csv')
    response['Content-Disposition'] = 'attachment; filename="%s.csv"' %dev.name
    #get measurement
    lastMeas = Measurement.objects.filter(device=dev, quantity=Type.objects.get(quantity='Temperature')).latest('date')
    writer = csv.writer(response)
    # Form list of required timesteps
    date = lastMeas.date.astimezone(tz)
    writer.writerow([date.strftime('%Y-%m-%d'),date.strftime('%H:%M'),lastMeas.value])
    return response

编辑(更精确): 我已经记录了几个小时的数据,但网站只向我展示了几个小时后的东西。随着我不断要求这些数据,它变得越来越近,好像它已被缓存到某个地方并且现在变得缓慢是网站可见的地方,直到一切都恢复正常。另一方面,如果我终止了日志记录过程,数据似乎永远丢失了。但奇怪的是logDevice方法完成了,我可以看到meas.save()命令已经执行了。我还尝试为Django信号post.save添加一个监听器,并且我正确地捕获它们。

几乎没有信息:
- 我正在使用postgresql后端
- 我在专用的Mac机器上运行所有这些 - 让我知道其他任何有用的知识

我的问题是:
- 你有没有可能发生的任何理由(以前不会发生这种情况,所以我猜它可能与数据库变大,4Gb现在有关) - 作为一个附带问题,但可能是相关的,我怀疑我在数据库中推送新元素的方式并不是很好,因为代码完全独立于django网站本身运行。有关如何改进的任何建议?我认为ladrLogger代码可以向创建新元素的专用视图发送请求,但这可能更重,无用。

编辑:添加我的models.py

class Room(models.Model):
    fullName = models.CharField(max_length=20, unique=True)
    shortName = models.CharField(max_length=5, unique=True, default = "000")
    nickName = models.CharField(max_length=20, default="Random Room")
    def __unicode__(self):
        return self.fullName

class Type(models.Model):
    quantity = models.CharField(max_length=100, default="Temperature", unique = True)
    unit = models.CharField(max_length=5, default="C", blank=True)
    VALUE_TYPES = (
               ('float', 'float'),
               ('boolean', 'boolean'),
               ('integer', 'integer'),
               ('string', 'string'),
                          )
    value_type = models.CharField(max_length=20, choices=VALUE_TYPES, default = "float")
    def __unicode__(self):
        return self.quantity

class Device(models.Model):
    name = models.CharField(max_length=30, default="Unidentified Device",unique=True)
    room = models.ForeignKey(Room)
    description = models.CharField(max_length=500, default="", blank=True,)
    indigoId = models.CharField(max_length=30,default="000")

    def __unicode__(self):
        #r = Room.objects.get(pk = self.room)
        return self.name #+ ' in room ' + r.name

    def latestMeasurement(self,*args):
        if len(args)==0:
            #No argument so just return latest argument
            meas = Measurement.objects.filter(device=self).latest('date')
        else:
            #Use first argument as the type
            meas = Measurement.objects.filter(device=self, quantity=args[0]).latest('date')
        if not meas:
            return None
        else:    
            return meas

    def typeList(self):
        return Type.objects.filter(measurement__device=self).distinct()


class Measurement(models.Model):
    device = models.ForeignKey(Device)
    date = models.DateTimeField(db_index=True)
    value = models.CharField(max_length=100,default="")
    quantity = models.ForeignKey(Type)

    class Meta:
        unique_together = ('date','device','quantity',)
        index_together = ['date', 'device']

    def __unicode__(self):
        t = self.quantity
        return str(self.value) + " " + self.quantity.unit
        # return str(self.value)

0 个答案:

没有答案