我正在尝试开发自定义的低开销映像管道，以使用直接套接字连接来处理映像。
我无法在服务器端重新创建图像，但是无法打开图像。
我有一个客户端，它从屏幕上截取屏幕截图并通过套接字连接发送这些屏幕截图。
要发送的数据格式为ds = data size，指示JSON标头大小，然后为8个字符（实际标头大小为16个字节，以字节为单位），然后作为具有文件/应用程序数据的标头数据的JSON数据，紧随其后的是图像字节。
（有些背景，无需阅读：）我没有将图像字节包含在实际的JSON主体中，因为这需要将图像字节转换为非常低效的base64并引起有关25秒即可到达服务器。
例如：
ds222 {
"abcd_id":"1234567890",
"image_file_format":"png",
"image_encoding":"RGB_565",
"width_in_pixels":1024,
"height_in_pixels":2048,
"timestamp":"2019-01-31T20:48:59+00:00",
"data_array_size_in_bytes":"208670"
}
这是客户端测试程序，它捕获快照并发送数据：
package SocketImageClient;
public class ImageClient {
private static final SimpleDateFormat yyyyMMddHHmmssSSSdataFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
private static final String[] HEADER_FOR_INCOMING_DATA_INFO = {"d", "s", " ", " ", " ", " ", " ", " ", " ", " ", " ", " "};
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
//String ipAddress = Inet4Address.getLocalHost().getHostAddress();
String ipAddress = "127.0.0.1";
int port = 10002;
int counter = 0;
Robot r = null;
int width=0;
int height=0;
String mime= "jpg";
byte[] imageInByte = null;
while (counter < 5)
{
try {
r = new Robot();
} catch (AWTException ex) {
Logger.getLogger(ImageClient.class.getName()).log(Level.SEVERE, null, ex);
}
// Used to get ScreenSize and capture image
Rectangle capture = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage Image = r.createScreenCapture(capture);
try {
width = Image.getWidth();
height = Image.getHeight();
//get the bytes of the buffered image
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write( Image, "jpg", baos );
baos.flush();
imageInByte = baos.toByteArray();
baos.close();
} catch (IOException ex) {
Logger.getLogger(ImageClient.class.getName()).log(Level.SEVERE, null, ex);
}
sendImageDataTcp(ipAddress,port,"jpg",mime, width,height, yyyyMMddHHmmssSSSdataFormat.format(System.currentTimeMillis()) ,imageInByte);
counter++;
try {
Thread.sleep(150);
} catch (InterruptedException ex) {
Logger.getLogger(ImageClient.class.getName()).log(Level.SEVERE, null, ex);
}
counter++;
}
}
private static void sendImageDataTcp(String ipAddress, int port, String fileFormat, String MIME, int widthPixels, int heightPixels, String timeStamp, byte[] imageBytes) {
//String ipAddress = Inet4Address.getLocalHost().getHostAddress();
Socket socket = null;
OutputStream os = null;
try {
socket = new Socket(ipAddress, port);
if (socket != null) {
os = socket.getOutputStream();
//make sure byte-enconding is UTF8 for actual JSON structure
//OutputStreamWriter out = new OutputStreamWriter(os, StandardCharsets.UTF_8);
//test JSON
String headerJson = "{\n"
+ " \"abcd_id\":\"1234567890\",\n"
+ " \"file_format\":\"jpg\",\n"
+ " \"image_compression\":\"RGB_565\",\n"
+ " \"width_in_pixels\":1024,\n"
+ " \"height_in_pixels\":2048,\n"
+ " \"timestamp\":\"2019-01-31T20:48:59+00:00\",\n"
+ " \"data_array_size_in_bytes\":\""+imageBytes.length+"\"\n"
+ "}";
//empty header
String[] header_for_incoming_data_info = HEADER_FOR_INCOMING_DATA_INFO;
//note we wan to length of the bytes not JSONObject length.
String headerStringPlusSize = "ds" + headerJson.toString().getBytes().length;
//set header based on the string headerStringPlusSize
for (int i = 0; i < headerStringPlusSize.length(); i++) {
header_for_incoming_data_info[i] = String.valueOf(headerStringPlusSize.charAt(i));
}
//entire entireMessage = header + json data
StringBuilder entireMessage = new StringBuilder();
for (int c = 0; c < header_for_incoming_data_info.length; c++) {
entireMessage.append(header_for_incoming_data_info[c]);
}
entireMessage.append(headerJson.toString());
//write JSON HEADER
os.write(entireMessage.toString().getBytes());
//write image bytes
os.write(imageBytes);
} else {
//Log.e(TAG, "The the image transfer server cannot be reached!");
System.out.println("The the image transfer server cannot be reached!");
}
} catch (java.net.ConnectException exc) {
//Log.e(TAG, "The the image transfer server cannot be reached!");
System.out.println("The the image transfer server cannot be reached!");
exc.printStackTrace();
} catch (IOException e) {
//Log.e(TAG, "ex:" + e.getMessage());
System.out.println("ex:" + e.getMessage());
e.printStackTrace();
} /*catch (JSONException e) {
e.printStackTrace();
}*/ finally {
if (os != null) {
try {
os.flush();
os.close();
} catch (IOException ex) {
//Log.e(TAG, "ex:" + ex.getMessage());
System.out.println("ex:" + ex.getMessage());
ex.printStackTrace();
}
}
}
}//end client
}
这是套接字服务器，它接收字节并尝试重新创建映像：
package socketserver;
public class SocketServer {
/**
* How to run the program via if you have .class & via command line java -cp
* ./classes socketserver.SocketServer 10001
*
* @param args the command line arguments
*/
public static void main(String[] args) {
ServerSocket serverSocket = null;
System.out.println("1)Run the program from command prompt OR shell like by issuing this command.java -jar SocketServerGradle.jar <PORT_NUMBER>");
System.out.println("Note: <PORT_NUMBER> must replace by actual port number. Like java -jar SocketServerGradle.jar 10001");
System.out.println("\n");
System.out.println("2)Any other message format, will result in errors.");
if (args[0] != null && args[0].length() > 0) {
//a port number must be numeric
int portNumber = Integer.valueOf(args[0]);
try {
serverSocket = new ServerSocket(portNumber);
System.out.println("\n");
System.out.println("***abcd to adrian image server started****");
new Thread(new ServiceHandler(serverSocket)).start();
} catch (Exception ex) {
Logger.getLogger(SocketServer.class.getName()).log(Level.SEVERE, null, ex);
}
}//end args input validation
else {
System.out.println("1)Run the program from command prompt OR shell like by issuing this command.java -jar SocketServerGradle.jar <PORT_NUMBER>");
System.out.println("Note: <PORT_NUMBER> must replace by actual port number. Like java -jar SocketServerGradle.jar 10001");
System.out.println("\n");
System.out.println("2)Any other message format, will result in errors.");
}
}//end main
private static class ServiceHandler implements Runnable {
private ServerSocket serverSocket;
public ServiceHandler(ServerSocket _serverSocket) {
serverSocket = _serverSocket;
}
public void run() {
Socket socket = null;
BufferedReader bf = null;
//stream suitable for reading raw bytes
InputStream inputStream;
//stream suitable for reading characters
InputStreamReader inputStreamReader;
while (true) {
try {
socket = serverSocket.accept();
} catch (java.net.BindException exc) {
Logger.getLogger(SocketServer.class.getName()).log(Level.SEVERE, null, exc);
if (exc.getMessage().contains("Address") && exc.getMessage().contains("already") && exc.getMessage().contains("use")) {
System.out.println("Port already being used, make sure to kill previous instance of program.");
errorLog(exc.getMessage());
}
} catch (IOException ex) {
errorLog(ex.getMessage());
ex.printStackTrace();
}
try {
if (socket != null) {
inputStream = socket.getInputStream();
inputStreamReader = new InputStreamReader(inputStream);
int read = 0;
int counter = 0;
long size = 0;
//read the 1st 10 character (20 bytes) to get the header
StringBuilder header = new StringBuilder();
while ((read = inputStreamReader.read()) != -1 && counter < 11) {
header.append((char) read);
counter++;
}
//ok inputStream the header valid, if the 1st two characters are ds then yes.
if (header.length() > 1 && header.substring(0, 2).equalsIgnoreCase("ds")) {
System.out.println("Valid Header");
//now lets get the message size, which comes after the 2nd char until the 12th char. RThe size will be parsed into Long numeric value.
try {
String sizedata = header.substring(2, 11).trim();
size = Long.valueOf(sizedata);
System.out.println("HEADER size in bytes:" + size);
} catch (NumberFormatException exc) {
exc.printStackTrace();
errorLog(exc.getMessage());
}
//now that we have the message size lets get the entire header which inputStream in JSON
StringBuffer message = new StringBuffer();
long byteCounter = 0;
while ((read = inputStreamReader.read()) != -1 && byteCounter < size) {
message.append((char) read);
byteCounter++;
}
ImagePayload imagePayload = parseJson(message.toString());
if (imagePayload != null) {
String logMsg = "parsed JSON & created:" + imagePayload;
System.out.println(logMsg);
logInfo(logMsg);
//after the JSON , get teh actual size of the image bytes that are sent after the JSON header
int actualImageBytesSize = imagePayload.getDataArraySizeInBytes();
System.out.println("Image size in bytes:" + actualImageBytesSize);
//get the image bytes
int imageBytesCounter = 0;
ByteArrayOutputStream byteos = new ByteArrayOutputStream();
byte [] buffer = new byte[1024];
int currentBytesRead=0;
int totalBytesRead=0;
while ((currentBytesRead = inputStream.read(buffer)) != -1) {
byteos.write(buffer,0,currentBytesRead);
totalBytesRead= totalBytesRead+ currentBytesRead;
}
//log info
String info = "total bytes Read:"+totalBytesRead+"expected number of bytes:"+actualImageBytesSize;
System.out.println(info);
logInfo(info);
//get the byte array and write the image to disk
byteos.flush();
byte[] imageData = byteos.toByteArray();
BufferedImage bufferedImage = ImageIO.read(new ByteArrayInputStream(imageData));
try
{
//write
File imageFile = new File("." + File.separator + imagePayload.getabcdId()+"."+imagePayload.getImageFileFormat());
//ERROR IS HERE -> attempt #1
if (!imageFile.exists()) {
imageFile.createNewFile();
}
ImageIO.write(bufferedImage,imagePayload.getImageFileFormat(), imageFile );
//ERROR IS HERE attempt # 2 - although the file is written its it cant be opened. Windows creates JPG file but cant open it.
/*FileOutputStream fos = new FileOutputStream(imageFile);
fos.write(imageData);
fos.close();*/
}
catch (java.lang.IllegalArgumentException exc)
{
exc.printStackTrace();
}
catch (IOException exc)
{
exc.printStackTrace();
}
} else {
String logErr = "Unable! to parse JSON::" + message.toString();
System.out.println(logErr);
logInfo(logErr);
}
} else {
System.err.println("invalid header!");
logInfo("invalid header!");
}
}
} catch (StringIndexOutOfBoundsException exc) {
exc.printStackTrace();
errorLog(exc.getMessage());
} catch (IOException ex) {
errorLog(ex.getMessage());
ex.printStackTrace();
}
}//end while
}
}
private static ImagePayload parseJson(String message) {
ImagePayload imagePayload = null;
try {
//parse JSON
ObjectMapper mapper = new ObjectMapper();
JsonFactory factory = mapper.getFactory();
com.fasterxml.jackson.core.JsonParser parser = factory.createParser(message.toString());
//Jackson JSON Tree Model
JsonNode jacksonNode = mapper.readTree(parser);
if (jacksonNode != null) {
JsonNode abcdIdNode = jacksonNode.get("abcd_id");
JsonNode imageFileFormatNode = jacksonNode.get("file_format");
JsonNode imageEncodingNode = jacksonNode.get("image_compression");
JsonNode imageWidthInPixelsNode = jacksonNode.get("width_in_pixels");
JsonNode imageHeightInPixelsNode = jacksonNode.get("height_in_pixels");
JsonNode dataArraySizeInBytesNode = jacksonNode.get("data_array_size_in_bytes");
JsonNode timestampNode = jacksonNode.get("timestamp");
String abcdId = abcdIdNode.textValue();
String imageFileFormat = imageFileFormatNode.textValue();
String imageCompression= imageEncodingNode.textValue();
int width = imageWidthInPixelsNode.intValue();
int height = imageHeightInPixelsNode.intValue();
int dataArraySizeInBytes = dataArraySizeInBytesNode.asInt();
String timestamp = timestampNode.asText();
imagePayload = new ImagePayload(abcdId, imageFileFormat, imageCompression, width, height, dataArraySizeInBytes, timestamp);
} else {
String errorMsg = "Error parsing json, JsonNode=" + jacksonNode + ", message:" + message;
System.err.println(errorMsg);
errorLog(errorMsg);
}
} catch (IOException ex) {
String err = "ImagePayload parseJson(), message:"+message+",exc:"+ex.getMessage();
System.out.println(err);
ex.printStackTrace();
errorLog(err);
}
catch (NullPointerException exc)
{
String err = "ImagePayload parseJson(), message:"+message+",exc:"+exc.getMessage();
System.out.println(err);
exc.printStackTrace();
errorLog(err);
}
return imagePayload;
}
private static void logInfo(String message) {
try {
File logFile = new File("." + File.separator + "abcd_adrian_IMG_INFO_LOG.log");
if (!logFile.exists()) {
logFile.createNewFile();
}
message = "\n" + message;
Files.write(Paths.get(logFile.getAbsolutePath()), message.getBytes(), StandardOpenOption.APPEND);
} catch (IOException ex) {
ex.printStackTrace();
}
}
private static void errorLog(String message) {
try {
File logFile = new File("." + File.separator + "abcd_adrian_IMG_ERROR_LOG.log");
if (!logFile.exists()) {
logFile.createNewFile();
}
message = "\n" + message;
Files.write(Paths.get(logFile.getAbsolutePath()), message.getBytes(), StandardOpenOption.APPEND);
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
尝试＃1 CMD输出（相关摘要）：
*** abcd启动到adrian图像服务器**** 有效标题
HEADER大小（以字节为单位）：219
解析的JSON并创建：ImagePayload {abcdId = 1234567890，ImageFileFormat = jpg，ImageEncoding = RGB_565，WidthInPixels = 1024，HeightInPixels = 2048，DataArraySizeInBytes = 132561，Timestamp = 2019-01-31T20：48：59 + 00 ：00}
图片大小（以字节为单位）：132561 总字节数：124369预期的字节数：132561
java.lang.IllegalArgumentException ：图片==空！ 在javax.imageio.ImageTypeSpecifier.createFromRenderedImage（ImageTypeSpecifier.java:925） 在javax.imageio.ImageIO.getWriter（ImageIO.java:1592） 在javax.imageio.ImageIO.write（ImageIO.java:1520） 在socketserver.SocketServer $ ServiceHandler.run（SocketServer.java:216） 在java.lang.Thread.run（Thread.java:748）
我尝试将格式硬编码为JPEG，jpeg，PNG，png等，但是却抛出了同样的IllegalArgumentException异常。
我还注意到，正如您在日志中看到的那样，套接字可能未读取全部字节？
总字节数：124369预期的字节数：132561
尝试＃2 CMD输出（相关摘要）：
***从abcd到adrian图像服务器的启动****
有效标题 标头大小（以字节为单位）：219 解析JSON并创建：ImagePayload {abcdId = 1234567890，ImageFileFormat = jpg，ImageEncoding = RGB_565，WidthInPixels = 1024，HeightInPixels = 2048，DataArraySizeInBytes = 129122，Timestamp = 2019-01-31T20：48：59 + 00：00} 图片大小（以字节为单位）：129122
总字节数：121161预期的字节数：129122
尝试＃2，而不要使用：
ImageIO.write(bufferedImage,imagePayload.getImageFileFormat(), imageFile );
我曾经使用过：
FileOutputStream fos = new FileOutputStream(imageFile);
fos.write(imageData);
fos.close();
这时我已经能够将文件写入驱动器，但是我无法打开它。当我单击Windows OS文件时，显示“看来我们不支持该文件。格式”。尽管该文件在文件系统上被识别为JPG。
有什么想法为什么不能读取整个字节，或者我如何解决此代码以便一次发送我的JSON标头和图像字节？
更新，基于注释：
由于Stackoverflow的限制，我为客户端添加了更新的代码段，基本上，代码是JSON + image []字节
new DataOutputStream(os).writeUTF(headerJson);
//write image bytes
os.write(imageBytes);
相关服务器摘要：
if (socket != null) {
inputStream = socket.getInputStream();
DataInputStream dis = new DataInputStream(inputStream);
String message = dis.readUTF();
//now that we have the message size lets get the entire header which inputStream in JSON
ImagePayload imagePayload = parseJson(message);
if (imagePayload != null) {
String logMsg = "parsed JSON & created:" + imagePayload;
System.out.println(logMsg);
logInfo(logMsg);
//after the JSON , get teh actual size of the image bytes that are sent after the JSON header
int actualImageBytesSize = imagePayload.getDataArraySizeInBytes();
//get the image bytes
int imageBytesCounter = 0;
ByteArrayOutputStream byteos = new ByteArrayOutputStream();
byte [] buffer = new byte[1024];
int currentBytesRead=0;
int totalBytesRead=0;
while ((currentBytesRead = inputStream.read(buffer)) != -1) {
byteos.write(buffer,0,currentBytesRead);
totalBytesRead= totalBytesRead+ currentBytesRead;
}
//log info
String info = "total bytes Read:"+totalBytesRead+",expected number of bytes:"+actualImageBytesSize;
System.out.println(info);
logInfo(info);
//get the byte array and write the image to disk
byteos.flush();
byte[] imageData = byteos.toByteArray();
BufferedImage bufferedImage = ImageIO.read(new ByteArrayInputStream(imageData));
try
{
//write
File imageFile = new File("." + File.separator + imagePayload.getWamsId()+"."+imagePayload.getImageFileFormat());
if (!imageFile.exists()) {
imageFile.createNewFile();
}
//Attempt #1
ImageIO.write(bufferedImage,"jpeg", imageFile );
//attempt # 2 - although the file is written its it cant be opened
/*FileOutputStream fos = new FileOutputStream(imageFile);
fos.write(imageData);
fos.close();*/
}
catch (java.lang.IllegalArgumentException exc)
{
exc.printStackTrace();
}
catch (IOException exc)
{
exc.printStackTrace();
}
} else {
String logErr = "Unable! to parse JSON::" + message.toString();
System.out.println(logErr);
logInfo(logErr);
}
}
该程序现在可以正常运行，但仅适用于Windows。当我将服务器代码移至Linux Amazon AWS时，服务器运行正确获取JSON并创建了映像，但无法再次打开该映像？为什么只有在Windows上运行服务器时才能打开image（JPG）？
我仅在linux上遇到此异常：
total bytes Read:15805440,expected number of bytes:15805440
java.lang.IllegalArgumentException: image == null!
at javax.imageio.ImageTypeSpecifier.createFromRenderedImage(Unknown Source)
at javax.imageio.ImageIO.getWriter(Unknown Source)
at javax.imageio.ImageIO.write(Unknown Source)
at socketserver.SocketServer$ServiceHandler.run(SocketServer.java:186)
at java.lang.Thread.run(Unknown Source)
total bytes Read:15805440,expected number of bytes:15805440
java.lang.IllegalArgumentException: image == null!
at javax.imageio.ImageTypeSpecifier.createFromRenderedImage(Unknown Source)