Python中的数据局部性优化

时间:2015-11-03 10:17:46

标签: python numpy memory

问题在于如何在Python中为这个类分配内存,而问题更多的是出于好奇而不是现实生活价值。

以下后处理类处理大量数据(numpy数组)。这是实际代码的简化版本,以使问题可读。基类的共享数据现在替换为characteristic_length变量。

Exception in thread "JavaFX Application Thread" java.lang.RuntimeException: Button.disable : A bound value cannot be set.
at javafx.beans.property.BooleanPropertyBase.set(BooleanPropertyBase.java:140)
at javafx.scene.Node.setDisable(Node.java:1543)
at com.virtusa.tempo.server.ui.controllers.ProjectManagerController$3.handle(ProjectManagerController.java:131)
at com.virtusa.tempo.server.ui.controllers.ProjectManagerController$3.handle(ProjectManagerController.java:124)
at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86)
at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238)
at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191)
at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74)
at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54)
at javafx.event.Event.fireEvent(Event.java:198)
at javafx.scene.Scene$ClickGenerator.postProcess(Scene.java:3471)
at javafx.scene.Scene$ClickGenerator.access$8100(Scene.java:3399)
at javafx.scene.Scene$MouseHandler.process(Scene.java:3767)
at javafx.scene.Scene$MouseHandler.access$1500(Scene.java:3486)
at javafx.scene.Scene.impl_processMouseEvent(Scene.java:1762)
at javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2495)
at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:350)
at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:275)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$350(GlassViewEventHandler.java:385)
at com.sun.javafx.tk.quantum.GlassViewEventHandler$$Lambda$355/919089086.get(Unknown Source)
at com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(QuantumToolkit.java:404)
at com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:384)
at com.sun.glass.ui.View.handleMouseEvent(View.java:555)
at com.sun.glass.ui.View.notifyMouse(View.java:927)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.lambda$null$145(WinApplication.java:101)
at com.sun.glass.ui.win.WinApplication$$Lambda$39/1633781598.run(Unknown Source)
at java.lang.Thread.run(Thread.java:745)

作为处理内存的python方式的陌生人,我很好奇是否将characteristic_length放在子类中会大大提高性能吗?

基于python文档,我认为派生类实例有一个指向定义基类的地址的指针。如果子类中的查找失败,则在基类中继续。这听起来像使用基类属性来存储派生类共享的数据是非常昂贵的。

在实际应用程序中,共享数据有点大numpy数组,但对所有派生类都是通用的。懒惰,我想避免分别读取每个子类的数据。有关如何有效解决这个问题的任何指示?

1 个答案:

答案 0 :(得分:1)

最简单的解决方案是使numpy数组成为全局数组。如果您想阻止用户修改该数组,可以将writable flag设置为False

或者你可以使它成为Reader类的class属性。