Question

我有一个JSON文件，其中包含多个包含重复值的文本数组。例如：

{
    "mName": "Carl Sanchez",
    "mEmailID": "csanchez0@msn.com",
    "mPhoneNo": 7954041324,

    "tutorTypes": [
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Coaching Institute Teacher ",
        " Corporate Professional ",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor",
        " Freelancer/Professional Tutor"
    ],
    "disciplines": [
        " Japanese",
        " German ",
        " Japanese",
        " German ",
        " Japanese",
        " Hindi ",
        " Japanese",
        " French "
    ]
}

我想从JSON源中的所有数组中删除重复值（文本值）。在上面的例子中，那就是从数组中删除重复的语言和教师类型。所需的输出将是上述JSON源，只有在适用的情况下删除了重复值。此外，我不想将代码绑定到特定的JSON字段名称，而是通常将任何文本值数组绑定。上例中的所需输出为

{
    "mName": "Carl Sanchez",
    "mEmailID": "csanchez0@msn.com",
    "mPhoneNo": 7954041324,

    "tutorTypes": [
        " Freelancer/Professional Tutor",
        " Coaching Institute Teacher ",
        " Corporate Professional "
    ],
    "disciplines": [
        " Japanese",
        " German ",
        " Hindi ",
        " French "
    ]
}

JSON的输入源是一个文件，我想在一个文件中写入输出。我尝试使用Jackson数据绑定API来完成此任务：

public static void removeDuplicateStringElementsFromAllArrays(String file) throws IOException {

        Writer fileWriter = new BufferedWriter(new FileWriter(new File("out.json")));

        JsonFactory f = new MappingJsonFactory();
        JsonParser jp = f.createJsonParser(new File(file));

        parse(jp, fileWriter);
    }

    private static void parse(JsonParser jp, Writer writer) throws IOException{
        JsonToken current;
        current = jp.nextToken();

        if(current != null){
            System.out.println(current.asString());
            writer.write(current.asString());
        }

        if(current == JsonToken.START_ARRAY){
            if(jp.nextTextValue() != null){
                JsonNode node = jp.readValueAsTree();
                // Trim the String values
                String[] values = ArraysUtil.trimArray("\"" , node.toString().split(","), "\"");
                // Ensure that there is no duplicate value
                values = new HashSet<String>(Arrays.asList(values)).toArray(new String[0]);
                // Finally, concatenate the values back and stash them to file
                String concatValue = String.join(",", values);

                // Write the concatenated values to file
                writer.write(concatValue);
            }
            else{
                parse(jp, writer);
            }
        }
        else{
            // Move on directly
            parse(jp, writer);
        }
    }

我得到几个空值作为输出。我知道为什么会发生这种情况。我认为，当我调用jp.nextTextValue()时，解析器已经移动并构建一个值树可能导致了这一点，但我无法找到任何解决方法。有谁知道，我将如何完成任务。

修改

我想在这里添加一件事 - 我使用的是Jackson-Databind API，因为它是基于Streaming API构建的，这在解析大型JSON源时很有效，这是我的情况。所以，考虑到这一点的解决方案将不胜感激。

Answer 1

创建一个bean Contact.java ，并将要删除重复项的属性声明为Set。

当您序列化JSON时，Set将执行删除重复项的工作。不需要额外的代码。

package com.tmp;

import java.util.Set;

public class Contact {

    String      mName;
    String      mEmailID;
    long        mPhoneNo;

    Set<String> tutorTypes; // to remove duplicates
    Set<String> disciplines; // to remove duplicates

    // setter and getter methods goes here...    
}

删除重复项

package com.tmp;

import java.io.File;
import java.io.IOException;

import com.fasterxml.jackson.databind.ObjectMapper;


/**
 * 
 * @author Ravi P
 */
class Tmp {

    public static void main( String[] args ) throws IOException {

        ObjectMapper mapper = new ObjectMapper();

        Contact contact = mapper.readValue( new File( "D:\\tmp\\file.json" ), Contact.class );

        mapper.writeValue( new File( "D:\\tmp\\file1.json" ), contact );

    }
}

Answer 2

以下是使用Json Simple的示例。请注意，这假定数组存在于根级别，并且不检查每个参数中的嵌套数组。如果要支持

，可以添加递归逻辑

package test.json.jsonsimple;

import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;

import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;

public class App 
{
    @SuppressWarnings("unchecked")
    public static void main( String[] args )
    {
        System.out.println( "Hello World!" );

        JSONParser parser = new JSONParser();

        try {
            JSONObject outmap = new JSONObject();
            Object obj = parser.parse(new FileReader("d:\\in.json"));
            JSONObject jsonObject = (JSONObject) obj;
            for(Object o : jsonObject.entrySet()){
                if(o instanceof Map.Entry){
                    Map.Entry<String, Object> entry = (Map.Entry<String, Object>) o;
                    if(entry !=null ){
                        if(entry.getValue() instanceof JSONArray){
                            Set<String> uniqueValues = removeDuplicates(entry.getValue());
                            outmap.put(entry.getKey(), uniqueValues);
                        }else{
                            outmap.put(entry.getKey(), entry.getValue());
                        }
                    }
                }
            }

            FileWriter file = new FileWriter("d:\\out.json");
            file.write(outmap.toJSONString());
            file.flush();
            file.close();

        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (ParseException e) {
            e.printStackTrace();
        }

    }

    @SuppressWarnings("unchecked")
    private static Set<String> removeDuplicates(Object value) {
        Set<String> outset = new HashSet<String>();
        JSONArray inset = (JSONArray) value;

        if (inset != null) {
            Iterator<String> iterator = inset.iterator();
            while (iterator.hasNext()) {
                outset.add(iterator.next());
            } 
        }
        return outset;
    }
}

使用Jackson删除所有JSON阵列中的重复文本值

2 个答案: