AskOverflow.Dev

AskOverflow.Dev Logo AskOverflow.Dev Logo

AskOverflow.Dev Navigation

  • 主页
  • 系统&网络
  • Ubuntu
  • Unix
  • DBA
  • Computer
  • Coding
  • LangChain

Mobile menu

Close
  • 主页
  • 系统&网络
    • 最新
    • 热门
    • 标签
  • Ubuntu
    • 最新
    • 热门
    • 标签
  • Unix
    • 最新
    • 标签
  • DBA
    • 最新
    • 标签
  • Computer
    • 最新
    • 标签
  • Coding
    • 最新
    • 标签
主页 / dba / 问题 / 262646
Accepted
adam78
adam78
Asked: 2020-03-26 00:48:52 +0800 CST2020-03-26 00:48:52 +0800 CST 2020-03-26 00:48:52 +0800 CST

Mysql 使用多对多表连接优化执行缓慢的查询

  • 772

我有以下查询与多个多对多联结表的联接:

profile_language
profile_industry
profile_contract_type
profile_contract_hour
profile_qualification

执行大约需要 3-4 秒。当我尝试排除多对多联结表时,查询在 0.4 秒内执行。

select distinct `profiles`.*
      , `locations`.`name` as `location_name`
       , `candidate_view`.`last_viewed`
         , CASE WHEN candidate_shortlist.profile_id IS NOT NULL THEN true ELSE false END AS shortlisted
     , CASE WHEN unlocked_profiles.profile_id IS NOT NULL THEN true ELSE false END AS unlocked 
from `profiles` 
inner join `jobseekers` on `jobseekers`.`id` = `profiles`.`jobseeker_id`
inner join `locations` on `locations`.`id` = `profiles`.`location_id` 
inner join `profile_language` on `profile_language`.`profile_id` = `profiles`.`id` 
inner join `profile_industry` on `profile_industry`.`profile_id` = `profiles`.`id` 
left join `profile_contract_type` on `profile_contract_type`.`profile_id` = `profiles`.`id` 
left join `profile_contract_hour` on `profile_contract_hour`.`profile_id` = `profiles`.`id` 
left join `profile_qualification` on `profile_qualification`.`profile_id` = `profiles`.`id` 
left join (SELECT MAX(created_at) AS last_viewed
                , profile_id
             FROM candidate_views
            WHERE recruiter_id = 43 
             GROUP BY profile_id ) AS candidate_view on `candidate_view`.`profile_id` = `profiles`.`id` 
left JOIN (SELECT order_items.purchaseable_id as profile_id
             FROM orders
       INNER JOIN order_items on order_items.order_id = orders.id
       INNER JOIN recruiters on recruiters.id = orders.recruiter_id
            WHERE recruiters.company_id = 37
              AND order_items.purchaseable_type = "App\\Profile" ) AS unlocked_profiles on `unlocked_profiles`.`profile_id` = `profiles`.`id` 
left join `candidate_shortlist` on `candidate_shortlist`.`profile_id` = `profiles`.`id` and `candidate_shortlist`.`recruiter_id` = 43 
    where `profiles`.`searchable` = 1 
      and `profiles`.`deleted_at` is NULL 
 order by `profiles`.`id` desc limit 25 offset 0

这是解释信息:

+----+-------------+-----------------------+------------+--------+-------------------------------------------------------------------+-------------+---------+----------------------------------+------+----------+----------------------------------------------+
| id | select_type | table                 | partitions | type   | possible_keys                                                     | key         | key_len | ref                              | rows | filtered | Extra                                        |
+----+-------------+-----------------------+------------+--------+-------------------------------------------------------------------+-------------+---------+----------------------------------+------+----------+----------------------------------------------+
|  1 | PRIMARY     | profiles              | NULL       | ALL    | PRIMARY,profiles_jobseeker_id_unique,profiles_location_id_foreign | NULL        | NULL    | NULL                             | 2826 |     1.00 | Using where; Using temporary; Using filesort |
|  1 | PRIMARY     | jobseekers            | NULL       | eq_ref | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.jobseeker_id |    1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | locations             | NULL       | eq_ref | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.location_id  |    1 |   100.00 | NULL                                         |
|  1 | PRIMARY     | profile_contract_type | NULL       | ref    | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.id           |    1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | profile_contract_hour | NULL       | ref    | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.id           |    1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | profile_qualification | NULL       | ref    | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.id           |    1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | <derived2>            | NULL       | ref    | <auto_key0>                                                       | <auto_key0> | 4       | testjobsdb.profiles.id           |    2 |   100.00 | NULL                                         |
|  1 | PRIMARY     | profile_language      | NULL       | ref    | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.id           |    2 |   100.00 | Using index                                  |
|  1 | PRIMARY     | order_items           | NULL       | ALL    | order_items_order_id_foreign                                      | NULL        | NULL    | NULL                             |    9 |   100.00 | Using where                                  |
|  1 | PRIMARY     | orders                | NULL       | eq_ref | PRIMARY,orders_recruiter_id_foreign                               | PRIMARY     | 4       | testjobsdb.order_items.order_id  |    1 |   100.00 | NULL                                         |
|  1 | PRIMARY     | recruiters            | NULL       | eq_ref | PRIMARY,recruiters_company_id_foreign                             | PRIMARY     | 4       | testjobsdb.orders.recruiter_id   |    1 |   100.00 | Using where                                  |
|  1 | PRIMARY     | candidate_shortlist   | NULL       | eq_ref | PRIMARY,candidate_shortlist_profile_id_foreign                    | PRIMARY     | 8       | const,testjobsdb.profiles.id     |    1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | profile_industry      | NULL       | ref    | PRIMARY                                                           | PRIMARY     | 4       | testjobsdb.profiles.id           |    4 |   100.00 | Using index; Distinct                        |
|  2 | DERIVED     | candidate_views       | NULL       | ref    | candidate_views_profile_id_foreign,Index 4                        | Index 4     | 4       | const                            |   21 |   100.00 | Using where; Using index                     |
+----+-------------+-----------------------+------------+--------+-------------------------------------------------------------------+-------------+---------+----------------------------------+------+----------+----------------------------------------------+

请注意,联结表是在 php 中构建动态搜索查询所必需的,并且在上面的示例中没有显示,但是如果输入了搜索参数,它们将被添加到 where 子句中,例如:

and `profile_contract_type`.`contract_type_id` in (1,2,3,4)

此外,当我更改查询以进行计数时,它甚至需要更长的时间,大约 4-5 秒,例如

select count(distinct `profiles`.`id`) as aggregate from `profiles`...

如何优化此查询。任何帮助表示赞赏。

mysql query-performance
  • 2 2 个回答
  • 1227 Views

2 个回答

  • Voted
  1. Best Answer
    Rick James
    2020-04-04T21:44:29+08:002020-04-04T21:44:29+08:00

    过度标准化。我的意思是模式具有在不需要时被规范化的数据。这通常会导致冗长的查询和较差的性能。

    与其为类型、小时、资格、行业、语言等设置单独的表,不如为每个表设置一列。

    也有

    profiles:  INDEX(deleted_at, searchable, id)
    

    可能会有更多建议;修复归一化后回来。

    更多的

    假设表是 InnoDB 并且id是PRIMARY KEY, ...

    SELECT COUNT(DISTINCT id) FROM profiles
    

    效率低下有两个原因。 DISTINCT是不必要的,因为id值无论如何都是不同的。它所能做的就是计算行数。但这最好通过SELECT COUNT(*) FROM profiles. 此时,优化器选择最小的索引并遍历该 BTree。

    在一个复杂的查询中,正是这样WHERE .. ORDER BY .. LIMIT:

        where `profiles`.`searchable` = 1 
          and `profiles`.`deleted_at` is NULL 
     order by `profiles`.`id` desc limit 25 offset 0
    

    那么优化器会考虑使用我建议的索引。但是,它将检查可能涉及多少行。如果超过大约 20% 的表需要被访问,它会在索引上踢出并简单地进行表扫描。

    LEFT JOIN与子查询...

    旧版本的 MySQL 在优化方面做得很糟糕

    LEFT JOIN ( SELECT ... ) AS x ON ...
    LEFT JOIN ( SELECT ... ) AS y ON ...
    

    在您的情况下,这些似乎每个都返回 1 行(或者可能NULL)。这是对所有版本都有帮助的优化。代替

    SELECT  ...,
            candidate_view,
            ...
        FROM ...
        LEFT JOIN ( SELECT ... ) AS candidate_view ON ...
        ...
    

    将该派生表移动到主SELECT列表中:

    SELECT  ...,
            ( SELECT  MAX(created_at)
                FROM  candidate_views
                WHERE  recruiter_id = 43
                  AND  profile_id = profiles.id  -- correlated subquery
            ) AS last_viewed,
            ...
        FROM ...
        ...
    

    和INDEX(recruiter_id, profile_id, created_at)

    您会得到相同的结果,包括NULL案例,但它通常运行得更快。也这样做unlocked_profiles。

    • 1
  2. NikitaSerbskiy
    2020-03-26T05:49:11+08:002020-03-26T05:49:11+08:00

    尝试先选择所需的配置文件 ID,然后加入其他列:

    select `profiles`.*
          , `locations`.`name` as `location_name`
           , `candidate_view`.`last_viewed`
             , CASE WHEN candidate_shortlist.profile_id IS NOT NULL THEN true ELSE false END AS shortlisted
         , CASE WHEN unlocked_profiles.profile_id IS NOT NULL THEN true ELSE false END AS unlocked 
    from (select distinct `profiles`.id
            from `profiles` 
            inner join `jobseekers` on `jobseekers`.`id` = `profiles`.`jobseeker_id`
            inner join `locations` on `locations`.`id` = `profiles`.`location_id` 
            inner join `profile_language` on `profile_language`.`profile_id` = `profiles`.`id` 
            inner join `profile_industry` on `profile_industry`.`profile_id` = `profiles`.`id` 
            left join `profile_contract_type` on `profile_contract_type`.`profile_id` = `profiles`.`id` 
            left join `profile_contract_hour` on `profile_contract_hour`.`profile_id` = `profiles`.`id` 
            left join `profile_qualification` on `profile_qualification`.`profile_id` = `profiles`.`id` 
            where `profiles`.`searchable` = 1 
          and `profiles`.`deleted_at` is NULL 
     order by `profiles`.`id` desc limit 25 offset 0
        ) AS p
    inner join `profiles` on p.id = `profiles`.id
    inner join `locations` on `locations`.`id` = `profiles`.`location_id` 
    left join (SELECT MAX(created_at) AS last_viewed
                    , profile_id
                 FROM candidate_views
                WHERE recruiter_id = 43 
                 GROUP BY profile_id ) AS candidate_view on `candidate_view`.`profile_id` = `profiles`.`id` 
    left JOIN (SELECT order_items.purchaseable_id as profile_id
                 FROM orders
           INNER JOIN order_items on order_items.order_id = orders.id
           INNER JOIN recruiters on recruiters.id = orders.recruiter_id
                WHERE recruiters.company_id = 37
                  AND order_items.purchaseable_type = "App\\Profile" ) AS unlocked_profiles on `unlocked_profiles`.`profile_id` = `profiles`.`id` 
    left join `candidate_shortlist` on `candidate_shortlist`.`profile_id` = `profiles`.`id` and `candidate_shortlist`.`recruiter_id` = 43 
    order by `profiles`.`id` desc 
    
    • 0

相关问题

  • 是否有任何 MySQL 基准测试工具?[关闭]

  • 我在哪里可以找到mysql慢日志?

  • 如何优化大型数据库的 mysqldump?

  • 什么时候是使用 MariaDB 而不是 MySQL 的合适时机,为什么?

  • 组如何跟踪数据库架构更改?

Sidebar

Stats

  • 问题 205573
  • 回答 270741
  • 最佳答案 135370
  • 用户 68524
  • 热门
  • 回答
  • Marko Smith

    连接到 PostgreSQL 服务器:致命:主机没有 pg_hba.conf 条目

    • 12 个回答
  • Marko Smith

    如何让sqlplus的输出出现在一行中?

    • 3 个回答
  • Marko Smith

    选择具有最大日期或最晚日期的日期

    • 3 个回答
  • Marko Smith

    如何列出 PostgreSQL 中的所有模式?

    • 4 个回答
  • Marko Smith

    列出指定表的所有列

    • 5 个回答
  • Marko Smith

    如何在不修改我自己的 tnsnames.ora 的情况下使用 sqlplus 连接到位于另一台主机上的 Oracle 数据库

    • 4 个回答
  • Marko Smith

    你如何mysqldump特定的表?

    • 4 个回答
  • Marko Smith

    使用 psql 列出数据库权限

    • 10 个回答
  • Marko Smith

    如何从 PostgreSQL 中的选择查询中将值插入表中?

    • 4 个回答
  • Marko Smith

    如何使用 psql 列出所有数据库和表?

    • 7 个回答
  • Martin Hope
    Jin 连接到 PostgreSQL 服务器:致命:主机没有 pg_hba.conf 条目 2014-12-02 02:54:58 +0800 CST
  • Martin Hope
    Stéphane 如何列出 PostgreSQL 中的所有模式? 2013-04-16 11:19:16 +0800 CST
  • Martin Hope
    Mike Walsh 为什么事务日志不断增长或空间不足? 2012-12-05 18:11:22 +0800 CST
  • Martin Hope
    Stephane Rolland 列出指定表的所有列 2012-08-14 04:44:44 +0800 CST
  • Martin Hope
    haxney MySQL 能否合理地对数十亿行执行查询? 2012-07-03 11:36:13 +0800 CST
  • Martin Hope
    qazwsx 如何监控大型 .sql 文件的导入进度? 2012-05-03 08:54:41 +0800 CST
  • Martin Hope
    markdorison 你如何mysqldump特定的表? 2011-12-17 12:39:37 +0800 CST
  • Martin Hope
    Jonas 如何使用 psql 对 SQL 查询进行计时? 2011-06-04 02:22:54 +0800 CST
  • Martin Hope
    Jonas 如何从 PostgreSQL 中的选择查询中将值插入表中? 2011-05-28 00:33:05 +0800 CST
  • Martin Hope
    Jonas 如何使用 psql 列出所有数据库和表? 2011-02-18 00:45:49 +0800 CST

热门标签

sql-server mysql postgresql sql-server-2014 sql-server-2016 oracle sql-server-2008 database-design query-performance sql-server-2017

Explore

  • 主页
  • 问题
    • 最新
    • 热门
  • 标签
  • 帮助

Footer

AskOverflow.Dev

关于我们

  • 关于我们
  • 联系我们

Legal Stuff

  • Privacy Policy

Language

  • Pt
  • Server
  • Unix

© 2023 AskOverflow.DEV All Rights Reserve