我正在尝试确定任何给定TYPE_INT_ARGB BufferedImage
将使用多少堆,以便对于正在进行某些图像处理的程序,我可以根据我们提供的图像大小设置合理的最大堆它
我将以下程序编写为测试,然后我用它来确定在没有OutOfMemoryError
的情况下运行的最小堆:
import java.awt.image.BufferedImage;
public class Test {
public static void main(String[] args) {
final int w = Integer.parseInt(args[0]);
final int h = Integer.parseInt(args[1]);
final BufferedImage img =
new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
System.out.println((4*w*h) >> 20);
}
}
(打印的值是存储int[]
像素数据的BufferedImage
的预期大小。)我期望找到的是所需的最大堆类似于{{ 1}},其中x + c
是数据数组的大小,x
是一个常量,由加载的类的大小,c
对象等组成。这就是我找到了(所有值都以MB为单位):
4*w*h min max heap ----- ------------ 5 - 10 15 20 31 40 61 80 121 160 241
BufferedImage
非常适合观察。 (请注意,我发现5MB图像没有最小值。)我不明白我所看到的。这些额外的字节是什么?
答案 0 :(得分:4)
在1.6.0_16和1.6.0_20之间的某个地方,Oracle的VM似乎存在一个错误。您甚至可以减少分配int数组的问题,因为问题不仅与BufferedImage有关。
使用1.6.0_16,我需要至少413 MB的堆来分配一个带有100,000,000个元素的int数组,这看起来很合理。对于1.6.0_20,相同的操作需要至少573 MB的堆空间,尽管在分配数组后实际只使用了400,000,000个字节。
答案 1 :(得分:1)
在进一步调查中,问题似乎是堆中的旧生成无法充分扩展以容纳图像的数据阵列,尽管堆中有足够的可用内存。
有关如何扩展旧代的详细信息,请参阅this question。
答案 2 :(得分:0)
问题在于BufferedImage对象以未压缩的格式将图像存储在内存中。有一个有效的解决方案:您可以将映像存储在硬盘驱动器上,而不必担心堆大小或物理内存限制。它最多可以存储2,147,483,647像素(或46,340 x 46,340像素)。 BigBufferedImage 解决了此问题。
创建az清空BigBufferedImage:
BigBufferedImage image = BigBufferedImage.create(
tempDir, width, height, type);
将现有图像加载到BigBufferedImage:
BigBufferedImage image = BigBufferedImage.create(
imagePath, tempDir, type);
渲染图像的一部分:
part = image.getSubimage(x, y, width, height);
BigBufferedImage的实现:
package com.pulispace.mc.ui.panorama.util;
/*
* This class is part of MCFS (Mission Control - Flight Software) a development
* of Team Puli Space, official Google Lunar XPRIZE contestant.
* This class is released under Creative Commons CC0.
* @author Zsolt Pocze, Dimitry Polivaev
* Please like us on facebook, and/or join our Small Step Club.
* http://www.pulispace.com
* https://www.facebook.com/pulispace
* http://nyomdmegteis.hu/en/
*/
import java.awt.Point;
import java.awt.Rectangle;
import java.awt.color.ColorSpace;
import java.awt.image.BandedSampleModel;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.ComponentColorModel;
import java.awt.image.DataBuffer;
import java.awt.image.Raster;
import java.awt.image.RenderedImage;
import java.awt.image.SampleModel;
import java.awt.image.WritableRaster;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.Hashtable;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.imageio.ImageIO;
import javax.imageio.ImageReadParam;
import javax.imageio.ImageReader;
import javax.imageio.stream.ImageInputStream;
import sun.nio.ch.DirectBuffer;
public class BigBufferedImage extends BufferedImage {
private static final String TMP_DIR = System.getProperty("java.io.tmpdir");
public static final int MAX_PIXELS_IN_MEMORY = 1024 * 1024;
public static BufferedImage create(int width, int height, int imageType) {
if (width * height > MAX_PIXELS_IN_MEMORY) {
try {
final File tempDir = new File(TMP_DIR);
return createBigBufferedImage(tempDir, width, height, imageType);
} catch (IOException e) {
throw new RuntimeException(e);
}
} else {
return new BufferedImage(width, height, imageType);
}
}
public static BufferedImage create(File inputFile, int imageType) throws IOException {
try (ImageInputStream stream = ImageIO.createImageInputStream(inputFile);) {
Iterator<ImageReader> readers = ImageIO.getImageReaders(stream);
if (readers.hasNext()) {
try {
ImageReader reader = readers.next();
reader.setInput(stream, true, true);
int width = reader.getWidth(reader.getMinIndex());
int height = reader.getHeight(reader.getMinIndex());
BufferedImage image = create(width, height, imageType);
int cores = Math.max(1, Runtime.getRuntime().availableProcessors() / 2);
int block = Math.min(MAX_PIXELS_IN_MEMORY / cores / width, (int) (Math.ceil(height / (double) cores)));
ExecutorService generalExecutor = Executors.newFixedThreadPool(cores);
List<Callable<ImagePartLoader>> partLoaders = new ArrayList<>();
for (int y = 0; y < height; y += block) {
partLoaders.add(new ImagePartLoader(
y, width, Math.min(block, height - y), inputFile, image));
}
generalExecutor.invokeAll(partLoaders);
generalExecutor.shutdown();
return image;
} catch (InterruptedException ex) {
Logger.getLogger(BigBufferedImage.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
return null;
}
private static BufferedImage createBigBufferedImage(File tempDir, int width, int height, int imageType)
throws FileNotFoundException, IOException {
FileDataBuffer buffer = new FileDataBuffer(tempDir, width * height, 4);
ColorModel colorModel = null;
BandedSampleModel sampleModel = null;
switch (imageType) {
case TYPE_INT_RGB:
colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
new int[]{8, 8, 8, 0},
false,
false,
ComponentColorModel.TRANSLUCENT,
DataBuffer.TYPE_BYTE);
sampleModel = new BandedSampleModel(DataBuffer.TYPE_BYTE, width, height, 3);
break;
case TYPE_INT_ARGB:
colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
new int[]{8, 8, 8, 8},
true,
false,
ComponentColorModel.TRANSLUCENT,
DataBuffer.TYPE_BYTE);
sampleModel = new BandedSampleModel(DataBuffer.TYPE_BYTE, width, height, 4);
break;
default:
throw new IllegalArgumentException("Unsupported image type: " + imageType);
}
SimpleRaster raster = new SimpleRaster(sampleModel, buffer, new Point(0, 0));
BigBufferedImage image = new BigBufferedImage(colorModel, raster, colorModel.isAlphaPremultiplied(), null);
return image;
}
private static class ImagePartLoader implements Callable<ImagePartLoader> {
private final int y;
private final BufferedImage image;
private final Rectangle region;
private final File file;
public ImagePartLoader(int y, int width, int height, File file, BufferedImage image) {
this.y = y;
this.image = image;
this.file = file;
region = new Rectangle(0, y, width, height);
}
@Override
public ImagePartLoader call() throws Exception {
Thread.currentThread().setPriority((Thread.MIN_PRIORITY + Thread.NORM_PRIORITY) / 2);
try (ImageInputStream stream = ImageIO.createImageInputStream(file);) {
Iterator<ImageReader> readers = ImageIO.getImageReaders(stream);
if (readers.hasNext()) {
ImageReader reader = readers.next();
reader.setInput(stream, true, true);
ImageReadParam param = reader.getDefaultReadParam();
param.setSourceRegion(region);
BufferedImage part = reader.read(0, param);
Raster source = part.getRaster();
WritableRaster target = image.getRaster();
target.setRect(0, y, source);
}
}
return ImagePartLoader.this;
}
}
private BigBufferedImage(ColorModel cm, SimpleRaster raster, boolean isRasterPremultiplied, Hashtable<?, ?> properties) {
super(cm, raster, isRasterPremultiplied, properties);
}
public void dispose() {
((SimpleRaster) getRaster()).dispose();
}
public static void dispose(RenderedImage image) {
if (image instanceof BigBufferedImage) {
((BigBufferedImage) image).dispose();
}
}
private static class SimpleRaster extends WritableRaster {
public SimpleRaster(SampleModel sampleModel, FileDataBuffer dataBuffer, Point origin) {
super(sampleModel, dataBuffer, origin);
}
public void dispose() {
((FileDataBuffer) getDataBuffer()).dispose();
}
}
private static final class FileDataBufferDeleterHook extends Thread {
static {
Runtime.getRuntime().addShutdownHook(new FileDataBufferDeleterHook());
}
private static final HashSet<FileDataBuffer> undisposedBuffers = new HashSet<>();
@Override
public void run() {
final FileDataBuffer[] buffers = undisposedBuffers.toArray(new FileDataBuffer[0]);
for (FileDataBuffer b : buffers) {
b.disposeNow();
}
}
}
private static class FileDataBuffer extends DataBuffer {
private final String id = "buffer-" + System.currentTimeMillis() + "-" + ((int) (Math.random() * 1000));
private File dir;
private String path;
private File[] files;
private RandomAccessFile[] accessFiles;
private MappedByteBuffer[] buffer;
public FileDataBuffer(File dir, int size) throws FileNotFoundException, IOException {
super(TYPE_BYTE, size);
this.dir = dir;
init();
}
public FileDataBuffer(File dir, int size, int numBanks) throws FileNotFoundException, IOException {
super(TYPE_BYTE, size, numBanks);
this.dir = dir;
init();
}
private void init() throws FileNotFoundException, IOException {
FileDataBufferDeleterHook.undisposedBuffers.add(this);
if (dir == null) {
dir = new File(".");
}
if (!dir.exists()) {
throw new RuntimeException("FileDataBuffer constructor parameter dir does not exist: " + dir);
}
if (!dir.isDirectory()) {
throw new RuntimeException("FileDataBuffer constructor parameter dir is not a directory: " + dir);
}
path = dir.getPath() + "/" + id;
File subDir = new File(path);
subDir.mkdir();
buffer = new MappedByteBuffer[banks];
accessFiles = new RandomAccessFile[banks];
files = new File[banks];
for (int i = 0; i < banks; i++) {
File file = files[i] = new File(path + "/bank" + i + ".dat");
final RandomAccessFile randomAccessFile = accessFiles[i] = new RandomAccessFile(file, "rw");
buffer[i] = randomAccessFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, getSize());
}
}
@Override
public int getElem(int bank, int i) {
return buffer[bank].get(i) & 0xff;
}
@Override
public void setElem(int bank, int i, int val) {
buffer[bank].put(i, (byte) val);
}
@Override
protected void finalize() throws Throwable {
dispose();
}
private void disposeNow() {
final MappedByteBuffer[] disposedBuffer = this.buffer;
this.buffer = null;
disposeNow(disposedBuffer);
}
public void dispose() {
final MappedByteBuffer[] disposedBuffer = this.buffer;
this.buffer = null;
new Thread() {
@Override
public void run() {
disposeNow(disposedBuffer);
}
}.start();
}
private void disposeNow(final MappedByteBuffer[] disposedBuffer) {
FileDataBufferDeleterHook.undisposedBuffers.remove(this);
if (disposedBuffer != null) {
for (MappedByteBuffer b : disposedBuffer) {
((DirectBuffer) b).cleaner().clean();
}
}
if (accessFiles != null) {
for (RandomAccessFile file : accessFiles) {
try {
file.close();
} catch (IOException e) {
e.printStackTrace();
}
}
accessFiles = null;
}
if (files != null) {
for (File file : files) {
file.delete();
}
files = null;
}
if (path != null) {
new File(path).delete();
path = null;
}
}
}
}