Question

此代码可能应该进行代码审查，但我不会在那里得到快速响应（那里只有2个常规问题）。我有以下代码将数据从excel导入我的grails应用程序。问题是我没有在excel文件中测试＆gt; 1000行，所以当我的客户端尝试上传13k行时，我的应用程序就冻结了。我已经检查了stacktrace.log（app正在生产中），但也不例外。系统管理员认为jvm内存不足。我们增加了堆内存的大小。 但是，我想询问是否有更好的方法来实现它。我正在使用apache poi并创建域对象，因为我从excel读取每一行。之后，我将对象列表传递给控制器，该控制器验证并将它们保存在数据库中。 我应该告诉我的客户一次限制导入的项目数量吗？ 是否有更好的方法来编写此代码？

def importData(file, user){
    def rows = []
    def keywords = Keyword.list()
    int inventoryCount = Inventory.findAllByUser(user).size()

    def inventory = new Inventory(name:"Inventory ${inventoryCount +1}", user:user)

    Workbook workbook = WorkbookFactory.create(file)
    Sheet sheet = workbook.getSheetAt(0)

    int rowStart = 1;
    int rowEnd = sheet.getLastRowNum() + 1 ;
    for (int rowNum = rowStart; rowNum < rowEnd; rowNum++) {
        Row r = sheet.getRow(rowNum);

        if(r != null && r?.getCell(0, Row.RETURN_BLANK_AS_NULL)!=null ){
            def rowData =[:]
            int lastColumn = 8;
            for (int cn = 0; cn < lastColumn; cn++) {
                Cell c = r.getCell(cn, Row.RETURN_BLANK_AS_NULL);
                if (c == null) {
                    return new ExcelFormatException("Empty cell not allowed",rowNum+1, cn+1)
                } else {

                    def field = properties[cn+1]
                    if(field.type==c.getCellType()){
                        if(c.getCellType()==text){

                            rowData<<[(field.name):c.getStringCellValue().toString()]
                        }else if(c.getCellType()==numeric){
                            if(field.name.equalsIgnoreCase("price") ){
                                rowData<<[(field.name):c.getNumericCellValue().toDouble()]
                            }else{
                                rowData<<[(field.name):c.getNumericCellValue().toInteger()]
                            }
                        }
                    }else{
                        return new ExcelFormatException("Invalid value found", rowNum+1, cn+1)
                    }
                }

            }

            def item = new InventoryItem(rowData)
            String keyword = retrieveKeyword(item.description, keywords)
            String criticality = keyword?"Critical":"Not known"
            int proposedMin = getProposedMin(item.usagePerYear)
            int proposedMax = getProposedMax(proposedMin, item.price, item.usagePerYear, item?.currentMin)
            String inventoryLevel = getInventoryLevel(item.usagePerYear, item.quantity, proposedMin, item.currentMin) 

            item.proposedMin = proposedMin
            item.proposedMax = proposedMax
            item.inventoryLevel = inventoryLevel
            item.keyword = keyword 
            item.criticality = criticality

            inventory.addToItems(item)
        }

    }

    return inventory
}

上述代码中使用的函数：

def retrieveKeyword(desc, keywords){
    def keyword
    for (key in keywords){
        if(desc.toLowerCase().contains(key.name.toLowerCase())){
            keyword = key.name
            break
        }
    }
    return keyword
}
int getProposedMin(int usage){
    (int) ((((usage/12)/30) *7) + 1)
}


int getProposedMax(int pmin, double price, int usage, int cmin){
    int c = price == 0? 1: ((Math.sqrt((24 * (usage/12)*5)/(0.15*price))) + (pmin - 1))
    if(cmin >= c){
        return pmin
    }
    return c
}

String getInventoryLevel(int usage, int qty, int proposedMin, int currentMin){
    if(qty != 0){
        double c = usage/qty
        if(usage==0)
            return "Excess"
        if(c<0.75){
            return "Inactive"
        }else if(proposedMin<currentMin){
            return "Excess"
        }else if(c>=0.75){
            return "Active"
        }
    }else if(usage==0 && qty == 0){
        return "Not used"
    }else if(usage>3 && qty ==0){
        return "Insufficient"
    }else if(proposedMin > currentMin){
        return "Insufficient"
    }

}

控制器操作：

def importData(){
    if(request.post){
        def file = request.getFile("excelFile")

        //validate file
        def file_types = ["application/vnd.ms-excel","application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"]

        if(!file_types.contains(file.getContentType())){
            render view:"importData", model:[error:"Invalid File type"]
            return
        }

        def inv = excelService.importData(file.getInputStream(),User.get(principal.id))
        if(inv){
            if(inv instanceof ExcelFormatException){
                def err = (ExcelFormatException) inv
                render view:"importData", model:[error:err.message +". Error occurred at: Row: "+err.row+" Col: "+err.col]
                return
            }else{
                render view:"viewData", model:[inventory:inv]
                return
            }
        }
    }


}

Answer 1

在处理批量导入时，Hibernate和GORM需要进行一些调整。两个建议：

遵循此处的技术：http://naleid.com/blog/2009/10/01/batch-import-performance-with-grails-and-mysql（用MySQL编写，但概念与任何RDBMS相关）
请勿使用集合来映射Inventory和InventoryItem之间的关系。从items移除Inventory集合，然后在Inventory课程中添加InventoryItem字段。 Burt Beckwith在此详细介绍了这一点：http://burtbeckwith.com/blog/?p=1029

Answer 2

使用插件会是更好的选择。我使用这个插件 - http://grails.org/plugin/excel-import

在grails中实现excel导入的有效方法

2 个答案: