如何在Salesforce中解析CSV

Question

我有一个CSV，其中包含需要转到多个Salesforce自定义对象的字段。我已经创建了APEX类来处理解析。但是，当我从VisualForce页面调用该类时，我收到消息：

Collection size 3,511 exceeds maximum size of 1,000.

我的APEX课程如下（从关于该主题的2篇博文中拼凑而成）：

public class parseCSV{  
public Blob contentFile { get; set; }    
public String nameFile { get; set; }    
public Integer rowCount { get; set; }    
public Integer colCount { get; set; }    
public List<List<String>> getResults() {        
 List<List<String>> parsedCSV = new List<List<String>>();        
 rowCount = 0;        
 colCount = 0;        
 if (contentFile != null){            
 String fileString = contentFile.toString();            
 parsedCSV = parseCSV(fileString, false);            
 rowCount = parsedCSV.size();            
 for (List<String> row : parsedCSV){                
 if (row.size() > colCount){                    
 colCount = row.size();                
}            
}        
}        
return parsedCSV;    
}  
public static List<List<String>> parseCSV(String contents,Boolean skipHeaders) {        
List<List<String>> allFields = new List<List<String>>();    
// replace instances where a double quote begins a field containing a comma    
// in this case you get a double quote followed by a doubled double quote    
// do this for beginning and end of a field    
contents = contents.replaceAll(',"""',',"DBLQT').replaceall('""",','DBLQT",');    
// now replace all remaining double quotes - we do this so that we can reconstruct    
// fields with commas inside assuming they begin and end with a double quote    
contents = contents.replaceAll('""','DBLQT');    
// we are not attempting to handle fields with a newline inside of them    
// so, split on newline to get the spreadsheet rows    
List<String> lines = new List<String>();    
try {        
lines = contents.split('\n');    
} 
catch (System.ListException e) {        
System.debug('Limits exceeded?' + e.getMessage());    
}    
Integer num = 0;    
for(String line : lines) {        
// check for blank CSV lines (only commas)        
if (line.replaceAll(',','').trim().length() == 0) break;
List<String> fields = line.split(',');          
List<String> cleanFields = new List<String>();        
String compositeField;        
Boolean makeCompositeField = false;        
for(String field : fields) {            
if (field.startsWith('"') && field.endsWith('"')) 
{
            cleanFields.add(field.replaceAll('DBLQT','"'));            
} 
else if (field.startsWith('"')) {                
makeCompositeField = true;                
compositeField = field;            
}
else if (field.endsWith('"')) {
compositeField += ',' + field;                
cleanFields.add(compositeField.replaceAll('DBLQT','"'));                
makeCompositeField = false;            
}
else if (makeCompositeField) {                
compositeField +=  ',' + field;            
}
else {
cleanFields.add(field.replaceAll('DBLQT','"'));            
}        
}                
allFields.add(cleanFields);    
}    
if (skipHeaders) allFields.remove(0);    
return allFields;       
}  
}

如何确保上述类将CSV中的正确列插入到正确的自定义对象中？

由于某些对象包含对其他对象的查找，我如何确保首先将数据加载到查找字段中，然后再加载到子表中？

最后，我如何解决上面的错误消息？是否建议使用VisualForce页面？

非常感谢您的帮助和指导。

Answer 1

出于兴趣，是否需要来自visualforce页面？您可以使用Data Loader加载CSV，甚至可以自己使用批量API构建解决方案。

如果您确实需要从某个页面进行操作，我建议您不要立即拆分新行，而是使用substring和indexOf一次只拉一行从输入字符串开始，随时处理它们。

Answer 2

我尝试处理一个4MB的CSV文件，SalesForce又向我回复了“Regext too complex”错误。我相信这个问题和你的问题类似。

我使用批处理顶点和迭代器修复它。更多信息请访问this link

Answer 3

salesforce的这个示例不处理单引号的情况。

例如，

",",XXX,YYY

当只有3个

时，这将返回4个字段

问题出在：

if (field.startsWith('"') && field.endsWith('"')) 

如果您有以下字段：

"

它以双引号开头和结尾，但这不是测试的意图

将其更改为：

if (field.length() > 1 && field.startsWith('"') && field.endsWith('"')) {

它会起作用。

完整更正的例子是：

    public class parseCSV{  
public Blob contentFile { get; set; }    
public String nameFile { get; set; }    
public Integer rowCount { get; set; }    
public Integer colCount { get; set; }    
public List<List<String>> getResults() {        
 List<List<String>> parsedCSV = new List<List<String>>();        
 rowCount = 0;        
 colCount = 0;        
 if (contentFile != null){            
 String fileString = contentFile.toString();            
 parsedCSV = parseCSV(fileString, false);            
 rowCount = parsedCSV.size();            
 for (List<String> row : parsedCSV){                
 if (row.size() > colCount){                    
 colCount = row.size();                
}            
}        
}        
return parsedCSV;    
}  
public static List<List<String>> parseCSV(String contents,Boolean skipHeaders) {        
List<List<String>> allFields = new List<List<String>>();    
// replace instances where a double quote begins a field containing a comma    
// in this case you get a double quote followed by a doubled double quote    
// do this for beginning and end of a field    
contents = contents.replaceAll(',"""',',"DBLQT').replaceall('""",','DBLQT",');    
// now replace all remaining double quotes - we do this so that we can reconstruct    
// fields with commas inside assuming they begin and end with a double quote    
contents = contents.replaceAll('""','DBLQT');    
// we are not attempting to handle fields with a newline inside of them    
// so, split on newline to get the spreadsheet rows    
List<String> lines = new List<String>();    
try {        
lines = contents.split('\n');    
} 
catch (System.ListException e) {        
System.debug('Limits exceeded?' + e.getMessage());    
}    
Integer num = 0;    
for(String line : lines) {        
// check for blank CSV lines (only commas)        
if (line.replaceAll(',','').trim().length() == 0) break;
List<String> fields = line.split(',');          
List<String> cleanFields = new List<String>();        
String compositeField;        
Boolean makeCompositeField = false;        
for(String field : fields) {            
if (field.length() > 1 && field.startsWith('"') && field.endsWith('"')) {
{
            cleanFields.add(field.replaceAll('DBLQT','"'));            
} 
else if (field.startsWith('"')) {                
makeCompositeField = true;                
compositeField = field;            
}
else if (field.endsWith('"')) {
compositeField += ',' + field;                
cleanFields.add(compositeField.replaceAll('DBLQT','"'));                
makeCompositeField = false;            
}
else if (makeCompositeField) {                
compositeField +=  ',' + field;            
}
else {
cleanFields.add(field.replaceAll('DBLQT','"'));            
}        
}                
allFields.add(cleanFields);    
}    
if (skipHeaders) allFields.remove(0);    
return allFields;       
}  
}

Answer 4

当我尝试将转换后的CSV数据（超过1000条记录）存储到Visualforce页面上的显示列表中时，我也收到了该消息。如果要显示导入的记录（这是一个很好的功能），则需要确保显示的列表少于1000个元素。

要将记录插入Salesforce对象，请执行以下操作：

我使用名为“parseCSV”的静态方法定义了一个名为“Util”的类（见下文）：

public blob importedCSV {get;set;}

list<list<string>> parsedCSV = Util.parseCSV(importedCSV.toString(),true);
list<Custom_Object__c> cObjsToInsert = new list<Custom_Object__c>();
for(list<string> line:parsedCSV){
    Custom_Object__c c = new Custom_Object__c();
    c.Name=line[0];
    c.String_Field__c=line[2];
    c.Currency_Field__c=decimal.valueof(line[3]);
    if(line.size()>4)
        c.Description__c=line[4];
    cObjsToInsert.add(c);
}
insert cObjsToInsert;

通常，当我处理CSV并将数据插入Salesforce对象时，我不会遇到大小错误。我已经制作了导入CSV的Visualforce页面，以某种奇特的方式处理CSV，然后将处理后的数据导回到CSV，Excel或文本（固定宽度）格式。我将在Visualforce页面上加载CSV并将数据作为附件保存在虚拟记录中。然后，我将把用户引导到另一个Visualforce页面，该页面抓取附件，处理数据，并输出到所需的格式。

我不记得我在哪里获得这段代码，但它的功能就像一个魅力。我也为 parseCSV 方法编写了测试代码。

public static list<list<String>> parseCSV(String contents,Boolean skipHeaders) {
    list<list<String>> allFields = new List<List<String>>();

    // replace instances where a double quote begins a field containing a comma
    // in this case you get a double quote followed by a doubled double quote
    // do this for beginning and end of a field
    contents = contents.replaceAll(',"""',',"DBLQT').replaceall('""",','DBLQT",');
    // now replace all remaining double quotes - we do this so that we can reconstruct
    // fields with commas inside assuming they begin and end with a double quote
    contents = contents.replaceAll('""','DBLQT');
    // we are not attempting to handle fields with a newline inside of them
    // so, split on newline to get the spreadsheet rows
    list<String> lines=new list<string>();
    try {
        lines = contents.split('\n');
    } catch (System.ListException e) {
        System.debug('Limits exceeded?  '+e.getMessage());
    }
    integer num=0;
    for(string line :lines) {
        // check for blank CSV lines (only commas)
        if(line.replaceAll(',','').trim().length()==0) 
            break;
        list<String> fields=line.split(',');    
        list<String> cleanFields=new list<String>();
        string compositeField;
        boolean makeCompositeField=false;
        for(string field : fields) {
            if(field.startsWith('"') && field.endsWith('"')) {
                cleanFields.add(field.replaceAll('DBLQT','"').removeStart('"').removeEnd('"'));
            } else if(field.startsWith('"')) {
                makeCompositeField = true;
                compositeField = field;
            } else if(field.endsWith('"')) {
                compositeField += ',' + field;
                cleanFields.add(compositeField.replaceAll('DBLQT','"').removeStart('"').removeEnd('"'));
                makeCompositeField = false;
            } else if(makeCompositeField) {
                compositeField +=  ',' + field;
            } else{
                cleanFields.add(field.replaceAll('DBLQT','"').removeStart('"').removeEnd('"'));
            }
        }  
        allFields.add(cleanFields);
    }
    if(skipHeaders) 
        allFields.remove(0);
    return allFields;       
}//END ParseCSV

@isTest
private static void testParseCSV(){
    string contents='Field1,Field2,Field3\n1,,Smith\n2,Fred,O\'Connor\n3,Destiny,"Awaits, DDS"\n\n';
    list<list<string>> parsedCSV=Util.parseCSV(contents,true);
    //line 1
    system.assertEquals('1',parsedCSV[0][0]);
    system.assertEquals('',parsedCSV[0][1]);
    system.assertEquals('Smith',parsedCSV[0][2]);
    //line 2
    system.assertEquals('2',parsedCSV[1][0]);
    system.assertEquals('Fred',parsedCSV[1][1]);
    system.assertEquals('O\'Connor',parsedCSV[1][2]);
    //line 3
    system.assertEquals('3',parsedCSV[2][0]);
    system.assertEquals('Destiny',parsedCSV[2][1]);
    system.assertEquals('Awaits, DDS',parsedCSV[2][2]);
}//END

我将此代码放在名为“Util”的类中。如果您有blob，请先将其转换为字符串，然后调用该方法。 e.g。

blob b=inputBlob;
list<list<string>> parsedCSV = Util.parseCSV(b.toString(),true);