Question

我正在尝试了解IO等待加载谷歌云SQL。我使用D1计划，它应该允许100个并发查询。我没有在我的仪表板中看到有关并发查询的任何统计信息，但在本地我有一个64线程池，所以我永远不会超过64个并发查询（不时有2个以上）。另外，我的IO操作每秒高峰为20。

转向更大的计划只会有所帮助。

查询非常简单 - 选择1，主要选择随机选择小顺序。

所有我的64个主题都停留在socketRead（）的google SQL IO上，如下所示：

at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
- locked <5f74f841> (a com.mysql.jdbc.util.ReadAheadInputStream)
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3049)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2809)
- locked <7856a45c> (a com.mysql.jdbc.JDBC4Connection)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2758)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:894)
- locked <7856a45c> (a com.mysql.jdbc.JDBC4Connection)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:732)
at org.jboss.jca.adapters.jdbc.CheckValidConnectionSQL.isValidConnection(CheckValidConnectionSQL.java:74)
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnectionFactory.isValidConnection(BaseWrapperManagedConnectionFactory.java:1173)
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.checkValid(BaseWrapperManagedConnection.java:461)
at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnectionFactory.getInvalidConnections(BaseWrapperManagedConnectionFactory.java:999)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.validateConnections(SemaphoreArrayListManagedConnectionPool.java:870)
at org.jboss.jca.core.connectionmanager.pool.validator.ConnectionValidator$ConnectionValidatorRunner.run(ConnectionValidator.java:277)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)

您能告诉我们如何理解瓶颈吗？

The IO graph

编辑：此外，这是卡住后最终出现的超时异常：

The last packet successfully received from the server was 13,969 milliseconds ago.  The last packet sent successfully to the server was 13,969 milliseconds ago.
        at sun.reflect.GeneratedConstructorAccessor107.newInstance(Unknown Source) [:1.6.0_45]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) [rt.jar:1.6.0_45]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513) [rt.jar:1.6.0_45]
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603)
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
        at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
        at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107)
        at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462)
        at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.6.0_45]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45]
        at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45]
        at org.hibernate.engine.jdbc.internal.proxy.AbstractStatementProxyHandler.continueInvocation(AbstractStatementProxyHandler.java:122)
        ... 49 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
        at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052)
        at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503)
        ... 62 more

EDIT2：消费者是一个谷歌云计算实例，与数据库并列。

Answer 1

Using filesort表示需要进行排序。如果数据量大于sort_buffer_size，则将创建临时文件。 Cloud SQL中的临时文件很慢，因此避免它们对于获得良好的性能/

至关重要

sort_buffer_size的默认值为2M。它既是全局变量又是会话变量，因此可以在需要更多内存的查询中进行更改。

mysql> SHOW VARIABLES LIKE 'sort_buffer_size';
+------------------+---------+
| Variable_name    | Value   |
+------------------+---------+
| sort_buffer_size | 2097152 |
+------------------+---------+
1 row in set (0.00 sec)

mysql>

每次sort_buffer_size太小，Created_tmp_files状态变量都会递增。

mysql> SHOW GLOBAL STATUS LIKE 'Created_tmp_files';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| Created_tmp_files | 663   |
+-------------------+-------+
1 row in set (0.01 sec)

mysql>

有关Using filesort的更多信息，请参阅Baron Schwartz's post on MySQL Performance Blog。深入讨论＆＃39; ORDER BY＆＃39;可以在Sergey Petrunia’s blog post找到。

我写了一个小程序（mysql-sorttest）来专门测试MySQL的排序性能。

在默认配置（1000行）中，数据适合内存（测试后Created_tmp_files不会增加）因此查询速度很快：

$ ./sort -dsn="root:root@tcp(x.x.x.x:3306)/sorttest" prepare
2014/08/24 12:22:03 Start number of rows: 0
2014/08/24 12:22:06 End number of rows: 1000
$ ./sort -dsn="root:root@tcp(x.x.x.x:3306)/sorttest" run
2014/08/24 12:22:10 Sleep 10s
2014/08/24 12:22:10 Start oneRun
2014/08/24 12:22:20 Done 3000 requests
2014/08/24 12:22:20 Total requests: 300.00/s
$

对于5000行，数据不适合内存，速度大幅下降：

$ ./sort -dsn="root:root@tcp(x.x.x.x:3306)/sorttest" -table_size=5000 prepare
2014/08/24 12:22:53 Start number of rows: 1000
2014/08/24 12:23:08 End number of rows: 5000
$ ./sort -dsn="root:root@tcp(x.x.x.x:3306)/sorttest" run
2014/08/24 12:23:15 Sleep 10s
2014/08/24 12:23:15 Start oneRun
2014/08/24 12:23:25 Done 91 requests
2014/08/24 12:23:25 Total requests: 9.10/s
$

对ORDER BY RAND()的评论：如numerous places中所示，避免这种类型的查询会很好，因为它会强制MySQL为每一行生成随机数，然后对其进行排序。这是一个CPU密集型操作，并且使用足够的线程执行该操作最终将占用所有CPU（或者可能在mysqld中达到其他一些限制）。

调试IO限制

1 个答案: