Question

这是我陷入的一种奇怪的行为，我找不到任何关于为什么会这样的暗示。我在这个例子中使用了estimate method of SizeEstimator from Spark，但我没有在代码中发现任何故障，所以我想知道为什么 - 如果它们提供了很好的内存估计 - 为什么我有这个：

val buf1 = new ArrayBuffer[(Int,Double)]
var i = 0
while (i < 3) {
   buf1 += ((i,i.toDouble))
   i += 1
}
System.out.println(s"Raw size with doubles: ${SizeEstimator.estimate(buf1)}")
val ite1 = buf1.toIterator
var size1: Long = 0l
while (ite1.hasNext) {
   val cur = ite1.next()
   size1 += SizeEstimator.estimate(cur)
}
System.out.println(s"Size with doubles: $size1")

val buf2 = new ArrayBuffer[(Int,Float)]
i = 0
while (i < 3) {
   buf2 += ((i,i.toFloat))
   i += 1
}
System.out.println(s"Raw size with floats: ${SizeEstimator.estimate(buf2)}")
val ite2 = buf2.toIterator
var size2: Long = 0l
while (ite2.hasNext) {
   val cur = ite2.next()
   size2 += SizeEstimator.estimate(cur)
 }
 System.out.println(s"Size with floats: $size2")

控制台输出打印：

Raw size with doubles: 200
Size with doubles: 96
Raw size with floats: 272
Size with floats: 168

所以我的问题很天真：在这种情况下，为什么花车往往会占用更多的内存而不是双打？当我将它转换为迭代器时，为什么会变得更糟（第一种情况，当转换为迭代器时，有75％的比率变为50％！）。

（为了获得更多上下文，我在试图通过将Double更改为Float来“优化”Spark应用程序时遇到了这个问题，并发现它实际上需要更多的浮点数而不是双倍的内存。。）

P.S。：这不是因为缓冲区的小尺寸（这里是3），如果我把100而不是：

Raw size with doubles: 3752
Size with doubles: 3200
Raw size with floats: 6152
Size with floats: 5600

和浮点数仍然消耗更多的内存......但是这个比率已经稳定了，所以看起来转换到迭代器的不同比率似乎必然是由于我猜的一些开销。

编辑： Product2实际上只适用于Int，Long和Double：

trait Product2[@specialized(Int, Long, Double) +T1, @specialized(Int, Long, Double) +T2] extends Any with Product

有人知道为什么Float没有被考虑在内吗？ Short都不会导致奇怪的行为......

Answer 1

这是因为@specialized Double Float (Int,Double)，但int专用double。

这意味着(Int,Float)将显示为具有2个原始java类型int和java.lang.Float字段的结构，而type Exception report message Error instantiating servlet class LoginServlet description The server encountered an internal error that prevented it from fulfilling this request. exception javax.servlet.ServletException: Error instantiating servlet class LoginServlet org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745) root cause java.lang.ClassNotFoundException: LoginServlet org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720) org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) java.lang.Thread.run(Thread.java:745) note The full stack trace of the root cause is available in the Apache Tomcat/7.0.56 logs.将显示为<%@page contentType="text/html" pageEncoding="UTF-8"%> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>APIS</title> <link rel="stylesheet" href="public/css/bootstrap.min.css"> <link rel="stylesheet" href="public/css/style.css"> <%@page import= "com.servlet.LoginServlet" %> </head> <body> <ul class="nav navbar-nav"> <li><a href="index.jsp"><span class="glyphicon glyphicon-home"></span>   Home</a></li> <li class="active"><a href="login.jsp"><span class="glyphicon glyphicon-lock"></span>   Login</a></li> <% if (session.getAttribute("agencyname") != null) { if (session.getAttribute("agencyname").toString().equals("slc")) { %> <li><a href="register.jsp"><span class="glyphicon glyphicon-user"></span>   New Register</a></li> <% } %> <li><a href="resetPassword.jsp"><span class="glyphicon glyphicon-refresh"></span>   Reset Password</a></li> <% } %> </ul> <form class="login-form" action="LoginServlet" method="post"> <h2 class="form-signin-heading" style="margin-left:20px;margin-bottom:30px;">Sign In</h2> <div class="form-group"> <input type="text" class="form-control" id="exampleInputEmail1" placeholder="UserName" name="UserName"> </div> <div class="form-group"> <input type="password" class="form-control" id="exampleInputPassword1" placeholder="Password" name="Password"> </div> <input type="submit" class="btn btn-success btn-block" value="Submit"> </form> </body> <script src="public/js/jquery.min.js"></script> <script src="public/js/bootstrap.min.js"></script> </html>的结构和包装器类型package com.servlet; import com.controller.UserDAO; import com.model.OgaUser; import java.io.IOException; import java.io.PrintWriter; import javax.servlet.ServletException; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import javax.servlet.http.HttpSession; @WebServlet(name = "LoginServlet", urlPatterns = {"/LoginServlet"}) public class LoginServlet extends HttpServlet { protected void processRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.setContentType("text/html;charset=UTF-8"); try (PrintWriter out = response.getWriter()) { /* TODO output your page here. You may use following sample code. */ out.println("<!DOCTYPE html>"); out.println("<html>"); out.println("<head>"); out.println("<title>Servlet LoginServlet</title>"); out.println("</head>"); out.println("<body>"); out.println("<h1>Servlet LoginServlet at " + request.getContextPath() + "</h1>"); out.println("</body>"); out.println("</html>"); } } // <editor-fold defaultstate="collapsed" desc="HttpServlet methods. Click on the + sign on the left to edit the code."> @Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { processRequest(request, response); } @Override protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { processRequest(request, response); System.out.println("******************************"); System.out.println("In doPost method"); String username = request.getParameter("UserName"); String password = request.getParameter("Password"); //OgaUser ou = new OgaUser(); String loginResult = UserDAO.login(username, password); if (loginResult != null) { HttpSession session = request.getSession(); if (loginResult.equalsIgnoreCase("slc")) { session.setAttribute("agency", "slc"); session.setAttribute("username","Harsha"); response.sendRedirect("index.jsp"); } else { response.sendRedirect("index.jsp"); } } else { } } @Override public String getServletInfo() { return "Short description"; }// </editor-fold> }字段

更多讨论Tuple2

Scala - 在这种情况下，为什么Double比Floats消耗更少的内存？

1 个答案: