0%

SQL优化-子查询结果集过多

SQL优化-子查询结果集过多

这个优化案例是一个实际的业务场景,朋友公司从商业数据库迁移到PG中,测试中发现性能存在问题,同等数据量在商业数据库中不到秒级就出结果,但是在PG中缺需要几分钟甚至更久。

经过优化对SQL的改写效果还是不错,这里与大家分享下。

现象

最大的一张主表T98_SYT_ACCT_STAT_D_1有5.8亿,其余的都是百万级
原始SQL语句如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
select
org.Internal_Org_Id as Stat_Org_Id
,(case when org.Internal_Org_Id = '999999888'
then '合计'
else org.Org_Name
end ) as Stat_Org_Name
,(case when org.Internal_Org_Id = '999999888'
then 1
else 2
end) as ORDERID
,sum(data.jon_acct_num) as jon_acct_num
,sum(data.jon_cross_num) as jon_cross_num
,sum(data.jon_pay_num) as jon_pay_num
,sum(data.total_jon_num) as total_jon_num
,sum(data.des_acct_num) as des_acct_num
,sum(data.des_cross_num) as des_cross_num
,sum(data.des_pay_num) as des_pay_num
,sum(data.total_des_num) as total_des_num
,sum(data.accjon_acct_num) as accjon_acct_num
,sum(data.accjon_cross_num) as accjon_cross_num
,sum(data.accjon_pay_num) as accjon_pay_num
,sum(data.total_accjon_num) as total_accjon_num
,sum(data.accjon_acct_amt) as accjon_acct_amt
,sum(data.accjon_cross_amt) as accjon_cross_amt
,sum(data.accjon_acct1_amt) as accjon_acct1_amt
,sum(data.accjon_acct0_amt) as accjon_acct0_amt
,sum(data.accjon_pay_amt) as accjon_pay_amt
,sum(data.total_accjon_amt) as total_accjon_amt
from cimfbview.t98_int_org_app_rela_h org
left join
(
select
syt.Stat_Org_Id as Stat_Org_Id
,sum(syt.jon_acct_num) as jon_acct_num
,sum(syt.jon_cross_num) as jon_cross_num
,sum(syt.jon_pay_num) as jon_pay_num
,sum(syt.total_jon_num) as total_jon_num
,sum(syt.des_acct_num) as des_acct_num
,sum(syt.des_cross_num) as des_cross_num
,sum(syt.des_pay_num) as des_pay_num
,sum(syt.total_des_num) as total_des_num
,sum(syt.accjon_acct_num) as accjon_acct_num
,sum(syt.accjon_cross_num) as accjon_cross_num
,sum(syt.accjon_pay_num) as accjon_pay_num
,sum(syt.total_accjon_num) as total_accjon_num
,sum(syt.accjon_acct_amt) as accjon_acct_amt
,sum(syt.accjon_cross_amt) as accjon_cross_amt
,sum(syt.accjon_acct1_amt) as accjon_acct1_amt
,sum(syt.accjon_acct0_amt) as accjon_acct0_amt
,sum(syt.accjon_pay_amt) as accjon_pay_amt
,sum(syt.total_accjon_amt) as total_accjon_amt
from T98_SYT_ACCT_STAT_D_1 syt
where syt.Stat_Period_Cd = '1'
and syt.Summ_Date = to_date('20161001','yyyymmdd')
and syt.Stat_Org_Attr_Cd = '9'
group by 1
) data
on org.Internal_Org_Id=data.Stat_Org_Id
where (org.Internal_Org_Id = '999999888'
OR ORG.Parent_Int_Org_Id = '999999888'
or '999999888' =
(
case
when '1' = '1' and '2' in ('2','3') then nation_org_id
when '1' = '2' and '2' in ('3','4','5') then prov_org_id
when '1' = '3' and '2' in ('4','5') then city_org_id
when '1' = '4' and '2' = '5' then county_org_id
else Internal_Org_Id
end
)
and org.Int_Org_level_cd in ('1','2')
)
and org.Int_Org_Stru_Type_Cd='1'
and org.Int_Org_Type_Cd in ('01','07','10','05','99')
and org.Start_Dt <= to_date('20161001','yyyymmdd')
and org.End_Dt > to_date('20161001','yyyymmdd')
group by 1,2,3;

查询比较慢我们首先查看执行计划:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
GroupAggregate  (cost=14409317.97..14409320.04 rows=18 width=622) (actual time=472510.542..472510.766 rows=37 loops=1)
Group Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COALESCE
(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
-> Sort (cost=14409317.97..14409318.02 rows=18 width=622) (actual time=472510.519..472510.542 rows=37 loops=1)
Sort Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COA
LESCE(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
Sort Method: quicksort Memory: 34kB
-> Merge Left Join (cost=14404111.77..14409317.60 rows=18 width=622) (actual time=472016.735..472510.437 rows=37 loops=
1)
Merge Cond: (a.internal_org_id = syt.stat_org_id)
-> Sort (cost=68934.45..68934.49 rows=18 width=76) (actual time=452.103..452.133 rows=37 loops=1)
Sort Key: a.internal_org_id
Sort Method: quicksort Memory: 29kB
-> Gather (cost=1000.42..68934.07 rows=18 width=76) (actual time=131.356..451.966 rows=37 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Nested Loop Left Join (cost=0.42..67932.27 rows=8 width=76) (actual time=58.910..398.641 rows=12 l
oops=3)
-> Parallel Seq Scan on t98_int_org_app_rela_h a (cost=0.00..67864.63 rows=8 width=51) (actual
time=58.773..398.006 rows=12 loops=3)
Filter: ((int_org_stru_type_cd = '1'::bpchar) AND (start_dt <= to_date('20161001'::text, 'y
yyymmdd'::text)) AND (end_dt > to_date('20161001'::text, 'yyyymmdd'::text)) AND (int_org_type_cd = ANY ('{01,07,10,05,99}'::bpchar
[])) AND ((internal_org_id = '999999888'::bpchar) OR (parent_int_org_id = '999999888'::bpchar) OR (('999999888'::bpchar = CASE WHE
N ('2'::text = ANY ('{2,3}'::text[])) THEN nation_org_id ELSE internal_org_id END) AND (int_org_level_cd = ANY ('{1,2}'::bpchar[])
))))
Rows Removed by Filter: 549182
-> Index Scan using idx_3 on t04_sys_organization org3 (cost=0.42..8.44 rows=1 width=45) (actua
l time=0.041..0.042 rows=1 loops=37)
Index Cond: ((a.new_org_id = internal_org_id) AND (sys_id = 'S20'::character(3)))
Filter: (province_cd = 71)
-> GroupAggregate (cost=14335177.32..14339939.28 rows=35483 width=586) (actual time=471564.595..472002.661 rows=5
6831 loops=1)
Group Key: syt.stat_org_id
-> Sort (cost=14335177.32..14335317.84 rows=56208 width=100) (actual time=471564.558..471606.793 rows=56832
loops=1)
Sort Key: syt.stat_org_id
Sort Method: quicksort Memory: 9529kB
-> Gather (cost=1000.00..14330742.93 rows=56208 width=100) (actual time=70985.797..471057.045 rows=56
833 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Seq Scan on t98_syt_acct_stat_d_1 syt (cost=0.00..14324122.13 rows=23420 width=100)
(actual time=71007.859..471086.192 rows=18944 loops=3)
Filter: ((stat_period_cd = '1'::bpchar) AND (stat_org_attr_cd = '9'::bpchar) AND (summ_date
= to_date('20161001'::text, 'yyyymmdd'::text)))
Rows Removed by Filter: 186677461
Planning time: 2.688 ms
Execution time: 472512.890 ms

通过执行计划看到t98_syt_acct_stat_d_1表并没有走到索引(其实表上是有索引),通过调整random_page_cost,seq_page_cost让优化器更偏向于走索引,得到的执行计划如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
GroupAggregate  (cost=9913849.82..9913851.89 rows=18 width=622) (actual time=166908.251..166908.659 rows=37 loops=1)
Group Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COALESCE
(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
-> Sort (cost=9913849.82..9913849.86 rows=18 width=622) (actual time=166908.218..166908.259 rows=37 loops=1)
Sort Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COA
LESCE(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
Sort Method: quicksort Memory: 34kB
-> Nested Loop Left Join (cost=1.43..9913849.44 rows=18 width=622) (actual time=138.763..166907.954 rows=37 loops=1)
-> Merge Left Join (cost=1.00..9913805.16 rows=18 width=627) (actual time=138.717..166905.751 rows=37 loops=1)
Merge Cond: (a.internal_org_id = syt.stat_org_id)
-> Index Scan using idx_2 on t98_int_org_app_rela_h a (cost=0.43..74380.19 rows=18 width=51) (actual time=1
4.101..2392.532 rows=37 loops=1)
Index Cond: (int_org_stru_type_cd = '1'::bpchar)
Filter: ((start_dt <= to_date('20161001'::text, 'yyyymmdd'::text)) AND (end_dt > to_date('20161001'::te
xt, 'yyyymmdd'::text)) AND (int_org_type_cd = ANY ('{01,07,10,05,99}'::bpchar[])) AND ((internal_org_id = '999999888'::bpchar) OR
(parent_int_org_id = '999999888'::bpchar) OR (('999999888'::bpchar = CASE WHEN ('2'::text = ANY ('{2,3}'::text[])) THEN nation_org
_id ELSE internal_org_id END) AND (int_org_level_cd = ANY ('{1,2}'::bpchar[])))))
Rows Removed by Filter: 415239
-> GroupAggregate (cost=0.58..9838981.24 rows=35483 width=586) (actual time=69.743..164300.902 rows=56831 l
oops=1)
Group Key: syt.stat_org_id
-> Index Scan using idx_1 on t98_syt_acct_stat_d_1 syt (cost=0.58..9834359.79 rows=56208 width=100) (
actual time=45.638..163209.973 rows=56832 loops=1)
Index Cond: ((summ_date = to_date('20161001'::text, 'yyyymmdd'::text)) AND (stat_period_cd = '1':
:bpchar) AND (stat_org_attr_cd = '9'::bpchar))
-> Index Scan using idx_3 on t04_sys_organization org3 (cost=0.42..2.45 rows=1 width=45) (actual time=0.041..0.04
3 rows=1 loops=37)
Index Cond: ((a.new_org_id = internal_org_id) AND (sys_id = 'S20'::character(3)))
Filter: (province_cd = 71)
Planning time: 2.604 ms
Execution time: 166909.087 ms

可以看到通过走索引后时间减少了一半多,仔细观察执行计划在此处COST值消耗较大并且后续还有一个Merge Left Join,子查询数据返回太多导致后面做Merge时效率也很低。t98_syt_acct_stat_d_1这个表数据量最大5.8亿

1
2
3
4
Index Scan using idx_1 on t98_syt_acct_stat_d_1 syt  (cost=0.58..9834359.79 rows=56208 width=100) (
actual time=45.638..163209.973 rows=56832 loops=1)
Index Cond: ((summ_date = to_date('20161001'::text, 'yyyymmdd'::text)) AND (stat_period_cd = '1':
:bpchar) AND (stat_org_attr_cd = '9'::bpchar))

解决方案

我们需要较少子查询返回的结果集,观察SQL语句中的这部分

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
on org.Internal_Org_Id=data.Stat_Org_Id
where (org.Internal_Org_Id = '999999888'
OR ORG.Parent_Int_Org_Id = '999999888'
or '999999888' =
(
case
when '1' = '1' and '2' in ('2','3') then nation_org_id
when '1' = '2' and '2' in ('3','4','5') then prov_org_id
when '1' = '3' and '2' in ('4','5') then city_org_id
when '1' = '4' and '2' = '5' then county_org_id
else Internal_Org_Id
end
)
and org.Int_Org_level_cd in ('1','2')
)
and org.Int_Org_Stru_Type_Cd='1'
and org.Int_Org_Type_Cd in ('01','07','10','05','99')
and org.Start_Dt <= to_date('20161001','yyyymmdd')
and org.End_Dt > to_date('20161001','yyyymmdd')

org.Internal_Org_Id=data.Stat_Org_Id , 后面的where条件是org表的过滤条件,我们将此过滤条件放到子查询中减少子查询返回的结果集,这样可以提高效率
改写后的SQL及执行计划:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
 select
org.Internal_Org_Id as Stat_Org_Id --机构编号
,(case when org.Internal_Org_Id = '999999888'
then '合计'
else org.Org_Name
end ) as Stat_Org_Name --机构代码
,(case when org.Internal_Org_Id = '999999888'
then 1
else 2
end) as ORDERID
,sum(data.jon_acct_num) as jon_acct_num
,sum(data.jon_cross_num) as jon_cross_num
,sum(data.jon_pay_num) as jon_pay_num
,sum(data.total_jon_num) as total_jon_num
,sum(data.des_acct_num) as des_acct_num
,sum(data.des_cross_num) as des_cross_num
,sum(data.des_pay_num) as des_pay_num
,sum(data.total_des_num) as total_des_num
,sum(data.accjon_acct_num) as accjon_acct_num
,sum(data.accjon_cross_num) as accjon_cross_num
,sum(data.accjon_pay_num) as accjon_pay_num
,sum(data.total_accjon_num) as total_accjon_num
,sum(data.accjon_acct_amt) as accjon_acct_amt
,sum(data.accjon_cross_amt) as accjon_cross_amt
,sum(data.accjon_acct1_amt) as accjon_acct1_amt
,sum(data.accjon_acct0_amt) as accjon_acct0_amt
,sum(data.accjon_pay_amt) as accjon_pay_amt
,sum(data.total_accjon_amt) as total_accjon_amt
from cimfbview.t98_int_org_app_rela_h org
left join
(
select
syt.Stat_Org_Id as Stat_Org_Id
,sum(syt.jon_acct_num) as jon_acct_num
,sum(syt.jon_cross_num) as jon_cross_num
,sum(syt.jon_pay_num) as jon_pay_num
,sum(syt.total_jon_num) as total_jon_num
,sum(syt.des_acct_num) as des_acct_num
,sum(syt.des_cross_num) as des_cross_num
,sum(syt.des_pay_num) as des_pay_num
,sum(syt.total_des_num) as total_des_num
,sum(syt.accjon_acct_num) as accjon_acct_num
,sum(syt.accjon_cross_num) as accjon_cross_num
,sum(syt.accjon_pay_num) as accjon_pay_num
,sum(syt.total_accjon_num) as total_accjon_num
,sum(syt.accjon_acct_amt) as accjon_acct_amt
,sum(syt.accjon_cross_amt) as accjon_cross_amt
,sum(syt.accjon_acct1_amt) as accjon_acct1_amt
,sum(syt.accjon_acct0_amt) as accjon_acct0_amt
,sum(syt.accjon_pay_amt) as accjon_pay_amt
,sum(syt.total_accjon_amt) as total_accjon_amt
from T98_SYT_ACCT_STAT_D_1 syt
where syt.Stat_Period_Cd = '1'
and syt.Summ_Date = to_date('20161001','yyyymmdd')
and syt.Stat_Org_Attr_Cd = '9'
and syt.Stat_Org_Id in (
select distinct(Internal_Org_Id) from t98_int_org_app_rela_h org where org.Internal_Org_Id = '999999888'
OR ORG.Parent_Int_Org_Id = '999999888'
or '999999888' =
(
case
when '1' = '1' and '2' in ('2','3') then nation_org_id
when '1' = '2' and '2' in ('3','4','5') then prov_org_id
when '1' = '3' and '2' in ('4','5') then city_org_id
when '1' = '4' and '2' = '5' then county_org_id
else Internal_Org_Id
end
)
and org.Int_Org_level_cd in ('1','2')
and org.Int_Org_Stru_Type_Cd='1'
and org.Int_Org_Type_Cd in ('01','07','10','05','99')
and org.Start_Dt <= to_date('20161001','yyyymmdd')
and org.End_Dt > to_date('20161001','yyyymmdd')

)
group by 1
) data
on org.Internal_Org_Id=data.Stat_Org_Id
where (org.Internal_Org_Id = '999999888'
OR ORG.Parent_Int_Org_Id = '999999888'
or '999999888' =
(
case
when '1' = '1' and '2' in ('2','3') then nation_org_id
when '1' = '2' and '2' in ('3','4','5') then prov_org_id
when '1' = '3' and '2' in ('4','5') then city_org_id
when '1' = '4' and '2' = '5' then county_org_id
else Internal_Org_Id
end
)
and org.Int_Org_level_cd in ('1','2')
)
and org.Int_Org_Stru_Type_Cd='1'
and org.Int_Org_Type_Cd in ('01','07','10','05','99')
and org.Start_Dt <= to_date('20161001','yyyymmdd')
and org.End_Dt > to_date('20161001','yyyymmdd')
group by 1,2,3;

GroupAggregate (cost=140463.35..140465.42 rows=18 width=622) (actual time=548.831..549.113 rows=37 loops=1)
Group Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COALESCE
(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
-> Sort (cost=140463.35..140463.40 rows=18 width=622) (actual time=548.789..548.817 rows=37 loops=1)
Sort Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COA
LESCE(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
Sort Method: quicksort Memory: 34kB
-> Nested Loop Left Join (cost=69906.79..140462.98 rows=18 width=622) (actual time=520.434..548.664 rows=37 loops=1)
Join Filter: (a.internal_org_id = data.stat_org_id)
Rows Removed by Join Filter: 669
-> Gather (cost=1000.42..68934.07 rows=18 width=76) (actual time=63.704..89.506 rows=37 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Nested Loop Left Join (cost=0.42..67932.27 rows=8 width=76) (actual time=61.970..389.321 rows=12 loops=3
)
-> Parallel Seq Scan on t98_int_org_app_rela_h a (cost=0.00..67864.63 rows=8 width=51) (actual time=6
1.885..388.864 rows=12 loops=3)
Filter: ((int_org_stru_type_cd = '1'::bpchar) AND (start_dt <= to_date('20161001'::text, 'yyyymmd
d'::text)) AND (end_dt > to_date('20161001'::text, 'yyyymmdd'::text)) AND (int_org_type_cd = ANY ('{01,07,10,05,99}'::bpchar[])) A
ND ((internal_org_id = '999999888'::bpchar) OR (parent_int_org_id = '999999888'::bpchar) OR (('999999888'::bpchar = CASE WHEN ('2'
::text = ANY ('{2,3}'::text[])) THEN nation_org_id ELSE internal_org_id END) AND (int_org_level_cd = ANY ('{1,2}'::bpchar[])))))
Rows Removed by Filter: 549182
-> Index Scan using idx_3 on t04_sys_organization org3 (cost=0.42..8.44 rows=1 width=45) (actual time
=0.027..0.029 rows=1 loops=37)
Index Cond: ((a.new_org_id = internal_org_id) AND (sys_id = 'S20'::character(3)))
Filter: (province_cd = 71)
-> Materialize (cost=68906.37..71450.71 rows=292 width=586) (actual time=12.309..12.389 rows=19 loops=37)
-> Subquery Scan on data (cost=68906.37..71449.25 rows=292 width=586) (actual time=455.410..457.807 rows=40
loops=1)
-> GroupAggregate (cost=68906.37..71446.33 rows=292 width=586) (actual time=455.407..457.739 rows=40
loops=1)
Group Key: syt.stat_org_id
-> Nested Loop (cost=68906.37..71416.40 rows=292 width=100) (actual time=455.322..457.390 rows=
41 loops=1)
-> Unique (cost=68905.79..68907.25 rows=291 width=10) (actual time=455.259..456.424 rows=
44 loops=1)
-> Sort (cost=68905.79..68906.52 rows=292 width=10) (actual time=455.255..455.810 r
ows=710 loops=1)
Sort Key: org.internal_org_id
Sort Method: quicksort Memory: 58kB
-> Gather (cost=1000.00..68893.83 rows=292 width=10) (actual time=0.683..453.
986 rows=725 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Seq Scan on t98_int_org_app_rela_h org (cost=0.00..67864.63
rows=122 width=10) (actual time=2.719..448.507 rows=242 loops=3)
Filter: ((internal_org_id = '999999888'::bpchar) OR (parent_int_org
_id = '999999888'::bpchar) OR (('999999888'::bpchar = CASE WHEN ('2'::text = ANY ('{2,3}'::text[])) THEN nation_org_id ELSE intern
al_org_id END) AND (int_org_level_cd = ANY ('{1,2}'::bpchar[])) AND (int_org_stru_type_cd = '1'::bpchar) AND (int_org_type_cd = AN
Y ('{01,07,10,05,99}'::bpchar[])) AND (start_dt <= to_date('20161001'::text, 'yyyymmdd'::text)) AND (end_dt > to_date('20161001'::
text, 'yyyymmdd'::text))))
Rows Removed by Filter: 548953
-> Index Scan using idx_1 on t98_syt_acct_stat_d_1 syt (cost=0.58..8.60 rows=1 width=100)
(actual time=0.016..0.017 rows=1 loops=44)
Index Cond: ((stat_org_id = org.internal_org_id) AND (summ_date = to_date('20161001':
:text, 'yyyymmdd'::text)) AND (stat_period_cd = '1'::bpchar) AND (stat_org_attr_cd = '9'::bpchar))
Planning time: 2.997 ms
Execution time: 565.724 ms

执行时间缩小到不到1秒,通过执行计划看到SQL执行顺序发生了变化,t98_syt_acct_stat_d_1不是最先被执行的,而是先过滤完internal_org_id,再根据internal_org_id与t98_syt_acct_stat_d_1上的stat_org_id做 Nested Loop
还有一种方式可以将子查询提出来改成连接,这种方式SQL语句看起来比较清爽, 效率也不错

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
select
org.Internal_Org_Id as Stat_Org_Id --机构编号
,(case when org.Internal_Org_Id = '999999888'
then '合计'
else org.Org_Name
end ) as Stat_Org_Name --机构代码
,(case when org.Internal_Org_Id = '999999888'
then 1
else 2
end) as ORDERID
,sum(data.jon_acct_num ) as jon_acct_num
,sum(data.jon_cross_num ) as jon_cross_num
,sum(data.jon_pay_num ) as jon_pay_num
,sum(data.total_jon_num ) as total_jon_num
,sum(data.des_acct_num ) as des_acct_num
,sum(data.des_cross_num ) as des_cross_num
,sum(data.des_pay_num ) as des_pay_num
,sum(data.total_des_num ) as total_des_num
,sum(data.accjon_acct_num ) as accjon_acct_num
,sum(data.accjon_cross_num ) as accjon_cross_num
,sum(data.accjon_pay_num ) as accjon_pay_num
,sum(data.total_accjon_num ) as total_accjon_num
,sum(data.accjon_acct_amt ) as accjon_acct_amt
,sum(data.accjon_cross_amt ) as accjon_cross_amt
,sum(data.accjon_acct1_amt ) as accjon_acct1_amt
,sum(data.accjon_acct0_amt ) as accjon_acct0_amt
,sum(data.accjon_pay_amt ) as accjon_pay_amt
,sum(data.total_accjon_amt ) as total_accjon_amt
from cimfbview.t98_int_org_app_rela_h org
left join T98_SYT_ACCT_STAT_D_1 data
on org.Internal_Org_Id=data.Stat_Org_Id
and data.Stat_Period_Cd = '1'
and data.Summ_Date = to_date('20161001','yyyymmdd')
and data.Stat_Org_Attr_Cd = '9'
where (org.Internal_Org_Id = '999999888'
OR ORG.Parent_Int_Org_Id = '999999888'
or '999999888' =
(
case
when '1' = '1' and '2' in ('2','3') then nation_org_id
when '1' = '2' and '2' in ('3','4','5') then prov_org_id
when '1' = '3' and '2' in ('4','5') then city_org_id
when '1' = '4' and '2' = '5' then county_org_id
else Internal_Org_Id
end
)
and org.Int_Org_level_cd in ('1','2')
)
and org.Int_Org_Stru_Type_Cd='1'
and org.Int_Org_Type_Cd in ('01','07','10','05','99')
and org.Start_Dt <= to_date('20161001','yyyymmdd')
and org.End_Dt > to_date('20161001','yyyymmdd')
group by 1,2,3
order by 1,2,3;


GroupAggregate (cost=69003.44..69005.51 rows=18 width=622) (actual time=401.488..401.925 rows=37 loops=1)
Group Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COALESCE
(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
-> Sort (cost=69003.44..69003.48 rows=18 width=136) (actual time=401.431..401.487 rows=37 loops=1)
Sort Key: a.internal_org_id, (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN '合计'::character varying ELSE COA
LESCE(org3.org_name, a.org_name) END), (CASE WHEN (a.internal_org_id = '999999888'::bpchar) THEN 1 ELSE 2 END)
Sort Method: quicksort Memory: 34kB
-> Gather (cost=1001.00..69003.06 rows=18 width=136) (actual time=61.302..401.261 rows=37 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Nested Loop Left Join (cost=1.00..68001.21 rows=8 width=136) (actual time=56.932..395.159 rows=12 loops=3)
-> Nested Loop Left Join (cost=0.42..67932.27 rows=8 width=76) (actual time=56.841..394.613 rows=12 loops=3
)
-> Parallel Seq Scan on t98_int_org_app_rela_h a (cost=0.00..67864.63 rows=8 width=51) (actual time=5
6.745..394.173 rows=12 loops=3)
Filter: ((int_org_stru_type_cd = '1'::bpchar) AND (start_dt <= to_date('20161001'::text, 'yyyymmd
d'::text)) AND (end_dt > to_date('20161001'::text, 'yyyymmdd'::text)) AND (int_org_type_cd = ANY ('{01,07,10,05,99}'::bpchar[])) A
ND ((internal_org_id = '999999888'::bpchar) OR (parent_int_org_id = '999999888'::bpchar) OR (('999999888'::bpchar = CASE WHEN ('2'
::text = ANY ('{2,3}'::text[])) THEN nation_org_id ELSE internal_org_id END) AND (int_org_level_cd = ANY ('{1,2}'::bpchar[])))))
Rows Removed by Filter: 549182
-> Index Scan using idx_3 on t04_sys_organization org3 (cost=0.42..8.44 rows=1 width=45) (actual time
=0.026..0.027 rows=1 loops=37)
Index Cond: ((a.new_org_id = internal_org_id) AND (sys_id = 'S20'::character(3)))
Filter: (province_cd = 71)
-> Index Scan using idx_1 on t98_syt_acct_stat_d_1 data (cost=0.58..8.60 rows=1 width=100) (actual time=0.0
34..0.036 rows=1 loops=37)
Index Cond: ((a.internal_org_id = stat_org_id) AND (summ_date = to_date('20161001'::text, 'yyyymmdd'::t
ext)) AND (stat_period_cd = '1'::bpchar) AND (stat_org_attr_cd = '9'::bpchar))
Planning time: 4.092 ms
Execution time: 412.283 ms

总结

优化就是减少对数据扫描次数,子查询或连接查询时应尽量将结果集在底层减少数据结果

观察执行计划要看按执行顺序查看,这样才会发现哪一步执行计划出现了问题、返回的数据量最大、COST值消耗最大

数据库都是相通的这里数据库是PG但MySQL优化思路也一样。