假设我有巨大的LinkedHashMap,我需要根据一些谓词并行地从列表中删除元素。
填写地图
Map<Long, String> map = new LinkedHashMap<>();
for (long i = 0; i < Integer.MAX_VALUE / 1000; i++) {
StringBuilder sb = new StringBuilder();
for (int j = 0; j < 10; j++) {
sb.append(String.valueOf(sb.append(UUID.randomUUID().toString())));
}
map.put(i, sb.toString());
}
尝试使用单线程循环删除元素。
Iterator<Entry<Long, String>> it = maps.entrySet().iterator();
while (it.hasNext()) {
if (it.next().getValue().indexOf(SEARCH_STR) != -1) {
it.remove();
}
}
尝试并行执行(只是一个工作缓慢的例子)
List<Future<?>> futures = new ArrayList<>();
long to = 0, from = 0, size = map.size();
do {
to += 550000;
to = Math.min(to, size);
long _from = from, _to = to;
Future<?> future = executor.submit(()-> {
for (Entry<Long, String> entry : map.subMap(_from, _to).entrySet()) {
if (entry.getValue().indexOf(SEARCH_STR) == -1) {
synchronized (result) {
result.put(entry.getKey(), entry.getValue());
}
}
}
});
from = to;
futures.add(future);
} while (to < size);
for (Future<?> future : futures) {
future.get();
}
使用Java 8 ForkJoinPool
或ParallelStream
可以获得更好的性能吗?下面的代码只是OOM
Map<Long, String> result = originalMap.entrySet().parallelStream()
.filter((k) -> k.getValue().indexOf(SEARCH_STR) != -1)
.collect(Collectors.toMap((k) -> k.getKey(), (v) -> v.getValue()
));