能快速查删重复记录需明确重复定义并有唯一标识列;查重须按业务逻辑选字段,避免MySQL 8.0+报错;删重推荐ROW_NUMBER()窗口函数,MySQL 5.7用带偏序条件的自连接;操作后必须加唯一约束防复发。

能快速查删重复记录,前提是明确“重复”的定义,并且表里有唯一标识列(比如 id)。没这两样,所谓“快速”就是空中楼阁。
怎么查:先确认哪几个字段算重复
查错比删错更常出问题。别一上来就 SELECT * + GROUP BY,MySQL 8.0+ 会直接报 ERROR 1055。必须按业务逻辑选字段:
- 单字段重复(如
phone):SELECT phone, COUNT(*) FROM users GROUP BY phone HAVING COUNT(*) > 1 - 多字段联合重复(如订单号+用户ID):
SELECT order_no, user_id, COUNT(*) FROM orders GROUP BY order_no, user_id HAVING COUNT(*) > 1 - 完全重复(所有业务字段相同):
SELECT name, email, COUNT(*) FROM users GROUP BY name, email HAVING COUNT(*) > 1—— 注意不能含主键id,否则每行都唯一
含 NULL 值时,GROUP BY 会把它们归为一组,但 = 判断不成立,得额外加 IS NULL 条件补查。
怎么删:MySQL 8.0+ 用 ROW_NUMBER() 最稳
窗口函数写法清晰、可读性强,还能正确处理 NULL 和并发锁问题:
WITH duplicates AS ( SELECT id, ROW_NUMBER() OVER (PARTITION BY name, email ORDER BY id) AS rn FROM users ) DELETE FROM users WHERE id IN (SELECT id FROM duplicates WHERE rn > 1);
关键点:
-
PARTITION BY后字段必须和查重时完全一致 -
ORDER BY id保证最小id留下;想留最新记录就改用ORDER BY id DESC - 执行前务必先跑一遍
SELECT验证要删的id是否合理
MySQL 5.7 怎么安全删:自连接必须带偏序条件
不能写 DELETE FROM t WHERE id IN (SELECT ...),MySQL 会报错。只能靠自连接,但极易误删:
DELETE t1 FROM users t1 INNER JOIN users t2 WHERE t1.name = t2.name AND t1.email = t2.email AND t1.id > t2.id;
漏掉 t1.id > t2.id 就全表清空。其他坑:
- 必须用
INNER JOIN,LEFT JOIN会留一堆NULL导致删不干净 - 所有业务字段都要在
WHERE中显式比对,少一个就可能漏删或错删 - 大表慎用,容易锁表卡住,建议加
WHERE id BETWEEN ...分批删
删完必须加唯一约束,否则等于白干
删只是擦灰,不加约束就是不关窗。加之前先验证:
SELECT name, email, COUNT(*) FROM users GROUP BY name, email HAVING COUNT(*) > 1;
结果为空才能执行:
ALTER TABLE users ADD UNIQUE KEY uk_name_email (name, email);
如果已有重复,ALTER TABLE 会失败。另外,INSERT IGNORE 不是清理工具——它只防新插入冲突,对存量数据完全无效。


















