feature: select..INTO OUTFILE..Add parameters：optionally enclosed by '"' escaped by '"' An error

[shangyanwen]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Please confirm if bug report does NOT exists already ?

• I confirm there is no existing issue for this

Describe the problem

 select `LO_ORDERDATE` , `LO_ORDERKEY` , `LO_LINENUMBER` , `LO_CUSTKEY` , `LO_PARTKEY` , `LO_SUPPKEY` , `LO_ORDERPRIORITY` , `LO_SHIPPRIORITY` , `LO_QUANTITY` , `LO_EXTENDEDPRICE` , `LO_ORDTOTALPRICE` , `LO_DISCOUNT` , `LO_REVENUE` , `LO_SUPPLYCOST` , `LO_TAX` , `LO_COMMITDATE` , `LO_SHIPMODE` , `C_NAME` , `C_ADDRESS` , `C_CITY` , `C_NATION` , `C_REGION` , `C_PHONE` , `C_MKTSEGMENT` , `S_NAME` , `S_ADDRESS` , `S_CITY` , `S_NATION` , `S_REGION` , `S_PHONE` , `P_NAME` , `P_MFGR` , `P_CATEGORY` , `P_BRAND` , `P_COLOR` , `P_TYPE` , `P_SIZE` , `P_CONTAINER` from lineorder l INNER JOIN customer c ON (c.C_CUSTKEY = l.LO_CUSTKEY)  INNER JOIN supplier s ON (s.S_SUPPKEY = l.LO_SUPPKEY)  INNER JOIN part p ON  (p.P_PARTKEY = l.LO_PARTKEY) where lo_orderdate >=19980101 and lo_orderdate <19990101 INTO OUTFILE "/data/ssb_data/select_19980101_02.csv" fields terminated by ',' optionally enclosed by '"' escaped by '"' lines terminated by '\r\n';

ERROR 1105 (HY000): Tianmu other specific error

Expected behavior

The data is exported normally. Procedure

#note：optionally enclosed by '"' escaped by '"'  ，These two parameters are not together. If used alone, the data is normally exported

How To Reproduce

select `LO_ORDERDATE` , `LO_ORDERKEY` , `LO_LINENUMBER` , `LO_CUSTKEY` , `LO_PARTKEY` , `LO_SUPPKEY` , `LO_ORDERPRIORITY` , `LO_SHIPPRIORITY` , `LO_QUANTITY` , `LO_EXTENDEDPRICE` , `LO_ORDTOTALPRICE` , `LO_DISCOUNT` , `LO_REVENUE` , `LO_SUPPLYCOST` , `LO_TAX` , `LO_COMMITDATE` , `LO_SHIPMODE` , `C_NAME` , `C_ADDRESS` , `C_CITY` , `C_NATION` , `C_REGION` , `C_PHONE` , `C_MKTSEGMENT` , `S_NAME` , `S_ADDRESS` , `S_CITY` , `S_NATION` , `S_REGION` , `S_PHONE` , `P_NAME` , `P_MFGR` , `P_CATEGORY` , `P_BRAND` , `P_COLOR` , `P_TYPE` , `P_SIZE` , `P_CONTAINER` from lineorder l INNER JOIN customer c ON (c.C_CUSTKEY = l.LO_CUSTKEY)  INNER JOIN supplier s ON (s.S_SUPPKEY = l.LO_SUPPKEY)  INNER JOIN part p ON  (p.P_PARTKEY = l.LO_PARTKEY) where lo_orderdate >=19980101 and lo_orderdate <19990101 INTO OUTFILE "/data/ssb_data/select_19980101_02.csv" fields terminated by ',' optionally enclosed by '"' escaped by '"' lines terminated by '\r\n';

Environment

Ver 14.14 Distrib 5.7.36-StoneDB, for Linux (x86_64) using EditLine wrapper

Are you interested in submitting a PR to solve the problem?

• Yes, I will!

[lujiashun]

ACK

[lujiashun]

The enclosed character is the same with escaped character. Tianmu dose not support this feature. Need to further study for it;

common::TianmuError Engine::GetIOP
...  
if (io_params->EscapeCharacter() != 0 &&
      io_params->Delimiter().find(io_params->EscapeCharacter()) != std::string::npos)
    return common::TianmuError(common::ErrorCode::WRONG_PARAMETER,
                               "Field terminator containing the escape character not supported.");

[lujiashun]

The output file is in the function

size_t DEforTxt::WriteString(const types::BString &str, int len) {
  int res_len = 0;
  if (escape_character_) {
    escaped_.erase();
    for (size_t i = 0; i < str.size(); i++) {
      if (str[i] == str_qualifier_ || (!str_qualifier_ && str[i] == delimiter_))
        escaped_.append(1, escape_character_);
      escaped_.append(1, str[i]);
    }
......

[lujiashun]

HOW MYSQL DEALS WITH THE FILED RELATED WITH ESCAPED CHAR AND ENCOLOSE CHAR?

1 current byte is escape char
1.1 escape char is different from enclosed char, next byte is escape char;
1.1.1 if multi-byte char length is 1, put it to output; (DONE)
1.1.2 if multi-byte char length is more than 1,put it to buffer;(CONTINUE)
1.2 escape char is the same with enclosed char, next byte is escape/enclosed char;
1.2.1 if multi-byte char length is 1, put it to output; (DONE)
1.1.2 if multi-byte char length is more than 1,put it to buffer;(continue)
1.3 escape char is the same with enclosed char, next byte is escape/enclosed char
put it back to buffer(the current byte and the next byte); (CONTINUE)

2 current byte is the first byte of line-terminator, and the field does not contains enclosed byte;
2.1 if ensure line-terminator, terminate the line;(DONE)
2.2 otherwise, continue;(CONTIUE)

3 current byte is enclose char
3.1 next byte is enclose char; put it to output;(DONE)
3.2 next byte is the first byte of line-terminator, and ensure line-termitor, put it to output;(DONE)
3.3 next byte is the first byte of field-terminator, and ensure field-terminator, put it to output;(DONE)
3.4 otherwise, put it back to buffer;(CONTINUE)

4 begin to process according to multi-bye character:
4.1 single-len, put it to output;(DONE)
4.2 multi-byte-len-character, put it to output in reverse;(DONE)