2-D(并发)HashMap:2属性密钥类型? hashmaps的hashmap? [更新]

时间:2009-02-04 19:16:00

标签: java performance hashmap concurrenthashmap

所以我需要一个二维ConcurrentHashMap

它必须尽可能快速,因为我将非常频繁地添加和更新其值。它位于多线程应用程序中,因此可以选择使用ConcurrentHashMap而不仅仅是HashMap。

“x”和“y”索引都是具有已知范围(0到40,000,000)的整数。

我需要知道的是:实现这一目标的最有效方法是什么,以便尽快实现?最明显的路线是做一个文字2-D hashmap:

ConcurrentHashMap<Integer, ConcurrentHashMap<Integer, ValueObj>> foo;

或者我可以创建一个私有类“IntPair”,其中包含两个属性x和y,并将其用作键...但如果我这样做,那么最有效的方法是equals()和{{ 1}?我最终会分配太多新的hashcode()吗?我可以为我分配的每个x / y保留一组IntPair,然后使用纯粹自反的equals(),这样我只是检查完全相同的对象实例吗?


更新

现在我已经仔细研究了Integer.valueOf(int),它使用的特定缓存模型在这里没有意义,因为我正在处理一个带有不可预测条目的非常稀疏的矩阵。我真的需要缓存所有使用的IntPairs,而不是预先指定的子集。

直观地说,在我看来,在一张大地图中查找IntPair,看看我是否已经创建了它,实际上与在大型地图中查找它一样或多或少相同-D“ConcurrentHashMap无论如何,不​​是吗?因此,这里的解决方案似乎只是在每次查找密钥时使用IntPair。是

4 个答案:

答案 0 :(得分:2)

ConcurrentHashMap非常大,所以你可能不想要它们的集合。

短期对象实际上分配速度非常快。你还得创建Integers吗?

您可以实习坐标对象,但仅仅查找的成本可能与创建它们的成本相当。 Integer的真正胜利是,当您在一段时间内保留大量的实例时,会共享相同的实例。

如果性能确实是一个巨大的问题,您可以编写(或使用)将long映射到引用的map类型对象。我不会惊讶地看到那里的自定义地图也有与坐标系相关的功能(比如找到最近的或在一个范围内)。

答案 1 :(得分:2)

这取决于您的(x,y)点在40,000,000 x 40,000,000矩阵中的稀疏程度。我的猜测是矩阵无论如何都会非常稀疏,因此创建大量ConcurrentHashMap s将会很昂贵。

相比之下,您的(不可变的)IntPair建议似乎更具吸引力。正如您所建议的,您甚至可以缓存其中一些以提高性能(请参阅Integer.valueOf(int)以了解如何使用静态嵌套类和静态工厂方法实现此功能)。由于总是需要哈希码,因此您可以在构造函数中预先计算它并将其保存为最终字段。要计算等于,您可以使用高速缓存中对象的标识相等性,否则您需要单独比较x和y。

编辑:这是source codeInteger.valueOf(int)(OpenJDK)。

答案 2 :(得分:0)

回应扎克,是的,矩阵将非常稀疏。

我查看了你链接的页面,毫无疑问,Integer.valueOf(int)的功能是理想的。如果我在IntPair类中开发了类似的静态方法,我可以假设我可以定义equals()来仅检查严格的自反性相等吗?

那就是说,我没有在那个页面中看到它解释了如何使用静态嵌套类和静态工厂方法实现该功能....我只是错过了它?我该怎么做?

谢谢!

答案 3 :(得分:0)

我已经基于标准Java HashMap创建了一个Int2DMap实现。我发现它比使用IntPair更快。但是它需要同步。

import java.io.*;
import java.util.*;

public class Int2DMap implements Map, Serializable {
    private static final int DEFAULT_INITIAL_CAPACITY = 16;
    private static final int MAXIMUM_CAPACITY = 1 << 30;
    private static final float DEFAULT_LOAD_FACTOR = 0.75f;
    protected Entry[] table;
    protected int size;
    protected int threshold;
    protected float loadFactor;
    protected transient volatile int modCount;

    public Int2DMap(int initialCapacity, float loadFactor) {
        if (initialCapacity < 0)
            throw new IllegalArgumentException("Illegal initial capacity: " + initialCapacity);
        if (initialCapacity > MAXIMUM_CAPACITY)
            initialCapacity = MAXIMUM_CAPACITY;
        if (loadFactor <= 0 || Float.isNaN(loadFactor))
            throw new IllegalArgumentException("Illegal load factor: " + loadFactor);
        // Find a power of 2 >= initialCapacity
        int capacity = 1;
        while (capacity < initialCapacity) {
            capacity <<= 1;
        }
        this.loadFactor = loadFactor;
        this.threshold = (int) (capacity * loadFactor);
        this.table = new Entry[capacity];
    }

    public Int2DMap(int initialCapacity) {
        this(initialCapacity, DEFAULT_LOAD_FACTOR);
    }

    public Int2DMap() {
        this(DEFAULT_INITIAL_CAPACITY, DEFAULT_LOAD_FACTOR);
    }

    public boolean containsKey(Object key) {
        int[] xy = (int[]) key;
        return containsKey(xy[0], xy[1]);
    }

    public Object get(Object key) {
        int[] xy = (int[]) key;
        return get(xy[0], xy[1]);
    }

    public Object put(Object key, Object value) {
        int[] xy = (int[]) key;
        return put(xy[0], xy[1], value);
    }

    public Object remove(Object key) {
        int[] xy = (int[]) key;
        return remove(xy[0], xy[1]);
    }

    public int size() {
        return size;
    }

    public boolean isEmpty() {
        return size == 0;
    }

    protected static final int indexFor(int x, int y, int length) {
        return (x * 31 + y) & (length - 1);
    }

    public Object get(int x, int y) {
        for (Entry e = table[indexFor(x, y, table.length)]; e != null; e = e.next) {
            if (e.x == x && e.y == y) {
                return e.value;
            }
        }
        return null;
    }

    public boolean containsKey(int x, int y) {
        return getEntry(x, y) != null;
    }

    protected Entry getEntry(int x, int y) {
        for (Entry e = table[indexFor(x, y, table.length)]; e != null; e = e.next) {
            if (e.x == x && e.y == y) {
                return e;
            }
        }
        return null;
    }

    public Object put(int x, int y, Object value) {
        int i = indexFor(x, y, table.length);
        for (Entry e = table[i]; e != null; e = e.next) {
            if (e.x == x && e.y == y) {
                Object oldValue = e.value;
                e.value = value;
                e.recordAccess(this);
                return oldValue;
            }
        }
        modCount++;
        addEntry(x, y, value, i);
        return null;
    }

    protected void resize(int newCapacity) {
        Entry[] oldTable = table;
        int oldCapacity = oldTable.length;
        if (oldCapacity == MAXIMUM_CAPACITY) {
            threshold = Integer.MAX_VALUE;
            return;
        }
        Entry[] newTable = new Entry[newCapacity];
        transfer(newTable);
        table = newTable;
        threshold = (int) (newCapacity * loadFactor);
    }

    protected void transfer(Entry[] newTable) {
        Entry[] src = table;
        int newCapacity = newTable.length;
        for (int j = 0; j < src.length; j++) {
            Entry e = src[j];
            if (e != null) {
                src[j] = null;
                do {
                    Entry next = e.next;
                    int i = indexFor(e.x, e.y, newCapacity);
                    e.next = newTable[i];
                    newTable[i] = e;
                    e = next;
                } while (e != null);
            }
        }
    }

    public void putAll(Map m) {
        int numKeysToBeAdded = m.size();
        if (numKeysToBeAdded == 0) {
            return;
        }
        if (numKeysToBeAdded > threshold) {
            int targetCapacity = (int) (numKeysToBeAdded / loadFactor + 1);
            if (targetCapacity > MAXIMUM_CAPACITY)
                targetCapacity = MAXIMUM_CAPACITY;
            int newCapacity = table.length;
            while (newCapacity < targetCapacity)
                newCapacity <<= 1;
            if (newCapacity > table.length)
                resize(newCapacity);
        }
        for (Iterator i = m.entrySet().iterator(); i.hasNext();) {
            Map.Entry e = (Map.Entry) i.next();
            put(e.getKey(), e.getValue());
        }
    }

    public Object remove(int x, int y) {
        Entry e = removeEntryForKey(x, y);
        return (e == null ? null : e.value);
    }

    protected Entry removeEntryForKey(int x, int y) {
        int i = indexFor(x, y, table.length);
        Entry prev = table[i];
        Entry e = prev;
        while (e != null) {
            Entry next = e.next;
            Object k;
            if (e.x == x && e.y == y) {
                modCount++;
                size--;
                if (prev == e)
                    table[i] = next;
                else
                    prev.next = next;
                e.recordRemoval(this);
                return e;
            }
            prev = e;
            e = next;
        }
        return e;
    }

    protected Entry removeMapping(Object o) {
        if (!(o instanceof Entry))
            return null;
        Entry entry = (Entry) o;
        int x = entry.x;
        int y = entry.y;
        int i = indexFor(x, y, table.length);
        Entry prev = table[i];
        Entry e = prev;
        while (e != null) {
            Entry next = e.next;
            if (e.x == x && e.y == y) {
                modCount++;
                size--;
                if (prev == e)
                    table[i] = next;
                else
                    prev.next = next;
                e.recordRemoval(this);
                return e;
            }
            prev = e;
            e = next;
        }
        return e;
    }

    public void clear() {
        modCount++;
        Entry[] tab = table;
        for (int i = 0; i < tab.length; i++)
            tab[i] = null;
        size = 0;
    }

    public boolean containsValue(Object value) {
        Entry[] tab = table;
        for (int i = 0; i < tab.length; i++)
            for (Entry e = tab[i]; e != null; e = e.next)
                if (value.equals(e.value))
                    return true;
        return false;
    }

    static class Entry implements Map.Entry {
        final int x;
        final int y;
        Object value;
        Entry next;

        Entry(int x, int y, Object value, Entry next) {
            this.x = x;
            this.y = y;
            this.value = value;
            this.next = next;
        }

        public final Object getKey() {
            return new int[] { x, y };
        }

        public final Object getValue() {
            return value;
        }

        public final Object setValue(Object newValue) {
            Object oldValue = value;
            value = newValue;
            return oldValue;
        }

        public final boolean equals(Object o) {
            if (!(o instanceof Map.Entry))
                return false;
            Map.Entry e = (Map.Entry) o;
            int[] xy = (int[])e.getKey();
            if (x == xy[0] && y == xy[1]) {
                Object v1 = getValue();
                Object v2 = e.getValue();
                if (v1 == v2 || (v1 != null && v1.equals(v2)))
                    return true;
            }
            return false;
        }

        public final int hashCode() {
            return ((31 + x) * 31 + y);
        }

        public final String toString() {
            return "[" + x + ", " + y + "]=" + value;
        }

        /**
         * This method is invoked whenever the value in an entry is overwritten by
         * an invocation of put(k,v) for a key k that's already in the HashMap.
         */
        void recordAccess(Int2DMap m) {
        }

        /**
         * This method is invoked whenever the entry is removed from the table.
         */
        void recordRemoval(Int2DMap m) {
        }
    }

    void addEntry(int x, int y, Object value, int bucketIndex) {
        Entry e = table[bucketIndex];
        table[bucketIndex] = new Entry(x, y, value, e);
        if (size++ >= threshold)
            resize(2 * table.length);
    }


    private abstract class HashIterator implements Iterator {
        Entry next; // next entry to return
        int expectedModCount; // For fast-fail
        int index; // current slot
        Entry current; // current entry

        HashIterator() {
            expectedModCount = modCount;
            if (size > 0) { // advance to first entry
                Entry[] t = table;
                while (index < t.length && (next = t[index++]) == null)
                    ;
            }
        }

        public final boolean hasNext() {
            return next != null;
        }

        final Entry nextEntry() {
            if (modCount != expectedModCount)
                throw new ConcurrentModificationException();
            Entry e = current = next;
            if (e == null)
                throw new NoSuchElementException();
            if ((next = e.next) == null) {
                Entry[] t = table;
                while (index < t.length && (next = t[index++]) == null)
                    ;
            }
            return e;
        }

        public void remove() {
            if (current == null)
                throw new IllegalStateException();
            if (modCount != expectedModCount)
                throw new ConcurrentModificationException();
            int x = current.x;
            int y = current.y;
            current = null;
            Int2DMap.this.removeEntryForKey(x, y);
            expectedModCount = modCount;
        }
    }

    private final class ValueIterator extends HashIterator {
        public Object next() {
            return nextEntry().value;
        }
    }

    private final class KeyIterator extends HashIterator {
        public Object next() {
            return nextEntry().getKey();
        }
    }

    private final class EntryIterator extends HashIterator {
        public Map.Entry next() {
            return nextEntry();
        }
    }

    // Subclass overrides these to alter behavior of views' iterator() method
    Iterator newKeyIterator() {
        return new KeyIterator();
    }

    Iterator newValueIterator() {
        return new ValueIterator();
    }

    Iterator newEntryIterator() {
        return new EntryIterator();
    }

    public Set keySet() {
        return new KeySet();
    }

    private final class KeySet extends AbstractSet {
        public Iterator iterator() {
            return newKeyIterator();
        }

        public int size() {
            return size;
        }

        public boolean contains(Object o) {
            return containsKey(o);
        }

        public boolean remove(Object o) {
            int[] xy = (int[]) o;
            return Int2DMap.this.removeEntryForKey(xy[0], xy[1]) != null;
        }

        public void clear() {
            Int2DMap.this.clear();
        }
    }

    public Collection values() {
        return new Values();
    }

    private final class Values extends AbstractCollection {
        public Iterator iterator() {
            return newValueIterator();
        }

        public int size() {
            return size;
        }

        public boolean contains(Object o) {
            return containsValue(o);
        }

        public void clear() {
            Int2DMap.this.clear();
        }
    }

    public Set entrySet() {
        return new EntrySet();
    }

    private final class EntrySet extends AbstractSet {

        public Iterator iterator() {
            return newEntryIterator();
        }

        public boolean contains(Object o) {
            if (!(o instanceof Map.Entry))
                return false;
            Entry e = (Entry) o;
            Entry candidate = getEntry(e.x, e.y);
            return candidate != null && candidate.equals(e);
        }

        public boolean remove(Object o) {
            return removeMapping(o) != null;
        }

        public int size() {
            return size;
        }

        public void clear() {
            Int2DMap.this.clear();
        }
    }

    public static void main(String[] args) {                
        try {

            Int2DMap map = new Int2DMap();

            map.put(20, 6000, "Test");
            System.out.println(map.size() == 1);

            System.out.println(map.get(20, 6000) != null);

            System.out.println("Test".equals(map.get(20, 6000)));

            for (Iterator iter = map.values().iterator(); iter.hasNext();) {
                System.out.println("Test".equals(iter.next()));
            }

            for (Iterator iter = map.keySet().iterator(); iter.hasNext();) {
                int[] key = (int[])iter.next();
                System.out.println(key[0] == 20 && key[1] == 6000);
            }

            for (Iterator iter = map.entrySet().iterator(); iter.hasNext();) {
                Map.Entry e = (Map.Entry)iter.next();
                System.out.println(e.toString().equals("[20, 6000]=Test"));
            }

            map.remove(20, 6000);
            System.out.println(map.size() == 0 && map.get(20, 6000) == null);


            long start = System.nanoTime();
            int max = 40000000;
            for (int i = 0; i < 500000; i++) {
                int x = (int)(Math.random() * max);
                int y = (int)(Math.random() * max);
                map.put(x, y, "");

                int x2 = (int)(Math.random() * max);
                int y2 = (int)(Math.random() * max);
                Object o = map.get(x2, y2);

            }
            System.out.println(map.size());
            System.out.println((System.nanoTime() - start) / 1000000);


            Map map2 = new HashMap();
            start = System.nanoTime();

            for (int i = 0; i < 500000; i++) {
                String key = "" + (int)(Math.random() * max) + "," + (int)(Math.random() * max);
                map2.put(key, "");

                String key2 = "" + (int)(Math.random() * max) + "," + (int)(Math.random() * max);
                Object o = map2.get(key2);

            }
            System.out.println(map2.size());
            System.out.println((System.nanoTime() - start) / 1000000);


        } catch (Throwable t) {
            t.printStackTrace();
        }
    }
}