月度归档:2016年06月

优化系列 | DELETE子查询改写优化

0、导读

有个采用子查询的DELETE执行得非常慢,改写成SELECT后执行却很快,最后把这个子查询DELETE改写成JOIN优化过程

1、问题描述

朋友遇到一个怪事,一个用子查询的DELETE,执行效率非常低。把DELETE改成SELECT后执行起来却很快,百思不得其解。

下面就是这个用了子查询的DELETE了:

[yejr@imysql.com]mydb > EXPLAIN delete from trade_info where id in (

select id from (

select a.id from trade_info a, order_info b, user c where

b.buyer = c.id and c.itv_account=’90000248′ and a.order_id = b.id) temp)\G

delete1

几个表的DDL是这样的:

delete2

上面这个SQL的执行耗时是:31.74秒

Query OK, 5 rows affected (31.74 sec)

如果我们把DELETE改写成SELECT的话,执行耗时仅是:0秒,来对比看下执行计划:

[yejr@imysql.com]mydb >EXPLAIN select id from trade_info where

id in (

select id from (

select a.id from trade_info a, order_info b, user c where

b.buyer = c.id and c.itv_account=’90000248′ and a.order_id = b.id) temp)\G

delete3

可以看到,trade_info 表从的全表扫描(type=ALL)变成了基于主键的等值查询(type=eq_ref),计划扫描数据量也从571万变成了1条,而且还可以避免回表,这2个SQL对比代价相差巨大。

2、优化思路

既然这个SQL把DELETE改成SELECT后执行效率就可以获得很大提升,除此外没特别区别,可能是查询优化器方面有些不足,导致无法直接优化,就得另想办法了。

我们的思路是把基于子查询的DELETE简化改写成多表JOIN后DELETE(一般来说,子查询效率比较低的话,可以考虑改写成JOIN),多表DELETE的语法课参考:https://dev.mysql.com/doc/refman/5.7/en/delete.html#idm140469624466800,例如这样的:

DELETE t1 FROM t1 LEFT JOIN t2 ON t1.id=t2.id WHERE t2.id IS NULL;

参照上面的形式,改写之后的SQL变成了下面这样:

DELETE trade_info

FROM

trade_info,

(

SELECT

a.id

FROM

trade_info a

JOIN order_info b ON a.order_id = b.id

JOIN user c ON b.buyer = c.id

WHERE

c.itv_account = ‘90000248’

) t2 where trade_info.id = t2.id;

delete4

可以看到新的SQL执行效率相对就高很多了,不需要再扫描571万条记录,执行耗时只需:0.01秒。

Query OK, 5 rows affected (0.01 sec)

3、其他建议

虽然MySQL 5.6及以上的版本对子查询做了优化,但从本案例的结果来看,在一些情况下还是不如意。

因此,如果发现有些子查询SQL效率比较差的话,可以尝试改写成JOIN形式,看看是否有所提升。此外,也要勇于怀疑查询优化器个别情况下存在不足,想办法绕过这些坑。

 

关于MySQL的方方面面大家想了解什么,可以直接留言回复,我会从中选择一些热门话题进行分享。 同时希望大家多多转发,多一些阅读量是老叶继续努力分享的绝佳助力,谢谢大家 :)

最后打个广告,运维圈人士专属铁观音茶叶微店上线了,访问:http://yejinrong.com 直达

FAQ系列 | 从MySQL 5.6到5.7复制错误解决

0、导读

在MySQL 5.7下采用多源复制方式,从5.6复制数据过来,会有问题吗?

1、问题描述

Q群里有位朋友想尝鲜5.7的多源复制,于是用MySQL 5.7版本作为slave,把MySQL 5.6作为master,想要将数据进行汇总,发现此路不通。

他在my.cnf中设置了2个选项,开启并发复制:

slave_parallel_type    = LOGICAL_CLOCK

slave_parallel_workers = 4

启动复制线程后,结果在错误日志中不断有类似下面的信息:

Transaction is tagged with inconsistent logical timestamps: sequence_number (823267087) <= last_committed (1301275374434324336)

执行 SHOW SLAVE STATUS\G 查看状态:

*************************** 1. row ***************************

Slave_IO_State: Waiting for master to send event

Last_Errno: 1756

Last_Error: … The slave coordinator and worker threads are stopped, possibly leaving data in inconsistent state. A restart should restore consistency automatically, although using non-transactional storage for data or info tables or DDL queries could lead to problems. In such cases you have to examine your data (see documentation for details).

Last_SQL_Errno: 1756

Last_SQL_Error: … The slave coordinator and worker threads are stopped, possibly leaving data in inconsistent state. A restart should restore consistency automatically, although using non-transactional storage for data or info tables or DDL queries could lead to problems. In such cases you have to examine your data (see documentation for details).

Last_IO_Error_Timestamp:

Last_SQL_Error_Timestamp: 160523 18:49:05

*************************** 2. row ***************************

Slave_IO_State: Waiting for master to send event

Last_Errno: 1756

Last_Error: … The slave coordinator and worker threads are stopped, possibly leaving data in inconsistent state. A restart should restore consistency automatically, although using non-transactional storage for data or info tables or DDL queries could lead to problems. In such cases you have to examine your data (see documentation for details).

Skip_Counter: 0

Last_SQL_Errno: 1756

Last_SQL_Error: … The slave coordinator and worker threads are stopped, possibly leaving data in inconsistent state. A restart should restore consistency automatically, although using non-transactional storage for data or info tables or DDL queries could lead to problems. In such cases you have to examine your data (see documentation for details).

2、原因分析

上面这些错误提示可以看到,主要原因是:slave端采用了基于 LOGICAL_CLOCK 类型的并行复制,但master端的binlog格式并不支持这种方式,所以slave端无法正确读取binlog并行apply。

3、解决方案

虽然仍旧可以采用MySQL 5.7作为slave,但就无法开启基于 LOGICAL_CLOCK 类型的并行复制了。需要改回传统模式就可以了:

slave_parallel_type    = DATABASE

此外,在MySQL复制方案中,强烈建议不要让主从大版本不一样,很容易出现各种各样的问题。

 

关于MySQL的方方面面大家想了解什么,可以直接留言回复,我会从中选择一些热门话题进行分享。 同时希望大家多多转发,多一些阅读量是老叶继续努力分享的绝佳助力,谢谢大家 :)

最后打个广告,运维圈人士专属铁观音茶叶微店上线了,访问:http://yejinrong.com 直达