[Bug] explain data change on operatorMem, mpp22698 test failed

[congxuebin]

Cloudberry Database version

No response

What happened

Issue the following statements:

SET gp_resqueue_print_operator_memory_limits=on;
SET gp_resqueue_memory_policy_auto_fixed_mem=100;
create table mpp22698(c1 int, c2 int) partition by range(c2) (start(1) end(6) every(1));
select get_operator_mem('explain select * from mpp22698;');

---

You will get the following:

NOTICE: One or more columns in the following table(s) do not have statistics: mpp22698
HINT: For non-partitioned tables, run analyze <table_name>(<column_list>). For partitioned tables, run analyze rootpartition <table_name>(<column_list>). See log for columns missing statistics.
ERROR: IndexError: list index out of range
CONTEXT: Traceback (most recent call last):
PL/Python function "get_operator_mem", line 8, in
list_opermem.add(operatorMemStr.split(':')[1])
PL/Python function "get_operator_mem"

---

But test case expects the following:

get_operator_mem  
--------------------
 ['100 kB']
(1 row)

What you think should happen instead

No response

How to reproduce

As mentioned above

Operating System

centos7

Anything else

No response

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct.

[avamingli]

Hi, I try to repeoduce but failed, get_operator_mem doesn't exist in CBDB,

gpadmin=# select get_operator_mem('explain select * from mpp22698;');
ERROR:  function get_operator_mem(unknown) does not exist
LINE 1: select get_operator_mem('explain select * from mpp22698;');
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

Could you provide a complete case?

[my-ship-it]

@congxuebin

[avamingli]

NOTICE: One or more columns in the following table(s) do not have statistics: mpp22698
HINT: For non-partitioned tables, run analyze <table_name>(<column_list>). For partitioned tables, run analyze rootpartition <table_name>(<column_list>). See log for columns missing statistics.
ERROR: IndexError: list index out of range
CONTEXT: Traceback (most recent call last):
PL/Python function "get_operator_mem", line 8, in
list_opermem.add(operatorMemStr.split(':')[1])
PL/Python function "get_operator_mem"

According to the message: it's not a statistic issue(just a notice and hint).

It shows the real errors are inside PL/Python function "get_operator_mem", a related pr #292 has fixed sth about operatorMem in explain output.
I think this is not a CBDB bug, change out-side-of CBDB's functionget_operator_mem according the the explain output will resolve your issue.

[congxuebin]

The following is DDL of this function. I believe something changed caused this failure.

create or replace function get_operator_mem(query text) returns text as
$$
rv = plpy.execute(query)
list_opermem = set()
for i in range(len(rv)):
cur_line = rv[i]['QUERY PLAN']
if 'operatorMem' in cur_line or 'Function Scan on gp_partition_expansion' in cur_line or 'Function Scan on gp_partition_inverse' in cur_line:
operatorMemStr = cur_line[cur_line.rindex(')'):]
list_opermem.add(operatorMemStr.split(':')[1])
return list(list_opermem)
$$
language plpython3u;

[congxuebin]

@avamingli Oh, ok. Let me look into the function. Will get back to you for confirmation. If it is an incompatible change, I would update test case accordingly.

[congxuebin]

I spoke with Mingli and confirmed this is as designed.

The following is more details I identified:

New behavior:

Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) (operatorMem: 100kB)
...

Old behavior:

Gather Motion 3:1 (slice1; segments: 3) (cost=0.00..431.00 rows=1 width=8) operatorMem: 100kB

That is why Python PL failed.

operatorMemStr = cur_line[cur_line.rindex(')'):]
list_opermem.add(operatorMemStr.split(':')[1])

Gonna change test case to fix it. Closing this issue.