Question

我在下面的方法中面临outOfMemory异常。有人可以帮忙吗？

此图为7073455节点和36549084关系我正在使用以下参数运行此程序： -Xmx3072M -XX：+ UseConcMarkSweepGC

代码：

public Iterable<Node> calculatePa(GraphDatabaseService graphDb) throws Exception {


float pa = 0.0f;
Transaction tx = graphDb.beginTx(); 
Iterable<Node> allNodes = getAllTypeNodes(graphDb, TERM_NODE);
int counter =0;
HashMap<String, String> starNodemap = new HashMap<String, String>(); 
PrintWriter pw = new PrintWriter(TERM_STAR_NODE_FILE_PATH);
Iterable<Relationship> pVRels;
Iterable<Relationship> removeRels;
Iterable<Relationship> pVRels2;
Node termNode;
Iterator<Node> termItr = allNodes.iterator();
while(termItr.hasNext()){
    termNode = termItr.next();
    int sumPV = 0,starNodeCounter = 0;
    pVRels = termNode.getRelationships( Direction.OUTGOING, RelTypes.PV);
    counter++;
    for (Relationship pvRel : pVRels){              
         sumPV +=Integer.parseInt(pvRel.getProperty(PV).toString());
         starNodeCounter++;

     }

    // to stop executing the rest of the code for this node
     if(starNodeCounter > STAR_NODE_TERM_THRESHOLD){

         if(isDebug){
             pw.println(termNode.getProperty(TERM).toString()+ PIPE    +String.valueOf(starNodeCounter));
             pw.flush();
         }
         removeRels = termNode.getRelationships(RelTypes.PV, Direction.OUTGOING );
         for (Relationship r : removeRels){
         r.delete();
     }
     termNode.delete();

     continue;
}
termNode.setProperty(SUM, sumPV);
pVRels2 = termNode.getRelationships( Direction.OUTGOING, RelTypes.PV);          
for (Relationship pvRel : pVRels2 ){
    pa = Integer.parseInt(pvRel.getProperty(PV).toString())/(float)sumPV;
    pvRel.setProperty(Pa, pa);
}

if(counter == COMMIT_SIZE){
    LOGGER.info("Calculated Pa for :::"+counter+"::: at ::"+(new Date()));
    LOGGER.info(Runtime.getRuntime().freeMemory()/(1024*1024)+"::: is the free memory before");
     tx.success();
     tx.close();
     tx = graphDb.beginTx();    
     LOGGER.info(Runtime.getRuntime().freeMemory()/(1024*1024)+"::: is the free memory after");
     counter =0;
}
}
if(counter!=0){
     tx.success();
}       
pw.close();
tx.close(); 

return allNodes;
}

public  Iterable<Node> getAllTypeNodes(final GraphDatabaseService graphDb , String type)    {           
Iterable<Node> allNodes = null; 
try( Transaction tx = graphDb.beginTx() ){  
allNodes = GlobalGraphOperations.at(graphDb).getAllNodesWithLabel(DynamicLabel.label(type));
//              long n_allNodes = IteratorUtil.count(allNodes);             
//  LOGGER.info("Count of "+type+"s -->"+n_allNodes);                           
tx.success();
}       
return allNodes;        
}

Answer 1

getAllTypeNodes提取的图表百分比是多少？

你能分享OOM例外吗？

你的COMMIT_SIZE是什么？

应该是10k到50k左右（但你必须为rels增加它，否则你可能最终得到x的提交大小，但你的事务中实际上有x * (number of rels)个元素。

使用neo4j嵌入时的内存问题

1 个答案: