[IGNITE-7537] SQL COPY command: implement CSV format options

• Type: Improvement
• Status: Reopened
• Priority: Major
• Resolution: Won't Fix
• Component/s: sql
• Assignee: Anton Kurbanov
• Reporter: Kirill Shirokov
• Created: 25/Jan/18 16:10
• Updated: 25/May/23 10:30
• Resolved: 14/Feb/19 15:16

Description

The following options can be implemented for the CSV format:

• Line separator pattern
• Field separator pattern
• Set of quote characters
• Escape sequence start character
• Line comment character

Escape sequences support is important for this feature. There is a draft code for handling escape sequences in a branch called ignite-7372 (see IGNITE-7372 for details).

Syntax example:

COPY
    ...
    FORMAT CSV
        [FIELDSEP='column-separators-regexp']
        [LINESEP='row-separators-regexp']
        [QUOTE='quote-chars']
        [ESCAPE='escape-char']
        [COMMENT='line-comment-char']

We may also want:

• Line comments handling
• Spaces trimming
• Empty lines skipping

Issue Links

• contains: IGNITE-12852 Comma in field is not supported by COPY command
• is blocked by: IGNITE-6917 SQL: implement COPY command for efficient data loading
• is related to: IGNITE-7539 SQL COPY command: implement NULL escape sequence handling
• is related to: IGNITE-17573 SQL COPY command should support all RFC4180 compatible CSV files
• relates to: IGNITE-7372 Implement escape sequences handling in internal SQL parser
• links to: GitHub Pull Request #3537
• links to: GitHub Pull Request #3537