Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-2420

Support string longer than 32000 characters

    Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.4.1
    • Component/s: None
    • Labels:
      None

      Description

      Add a property in creating table 'long_string_columns' to support string columns that will contains more than 32000 characters.
      Inside carbondata, it use an integer instead of short to store the length of bytes content.

        Attachments

          Issue Links

          1.
          long string columns should not be sort column Sub-task Resolved xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 50m
          2.
          long string columns should not be dictionary columns Sub-task Resolved xuchuanyin  
          3.
          long_string_columns should be string columns Sub-task Resolved xuchuanyin  
          4.
          Support long_string_columns in sdk Sub-task Resolved xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h
          5.
          Support long_string_columns property in dataframe writer Sub-task Resolved xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 20m
          6.
          Support varchar datatype in DDL as longstring column Sub-task Open xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2.5h
          7.
          Support page size less than 32000 in CarbondataV3 Sub-task Resolved xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 10m
          8.
          Split to multiple pages if varchar column page exceeds 2GB/snappy limits Sub-task Resolved jiangmanhua

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4.5h
          9.
          Complex datatype support long string Sub-task Open Unassigned  
          10.
          Support config long_string_columns when create datamap Sub-task Resolved jiangmanhua

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h
          11.
          create table with long_string_columns property Sub-task Resolved lianganping

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 8.5h
          12.
          Fix loading problem using global/batch sort fails when table has long string columns Sub-task Resolved jiangmanhua

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3h 50m
          13.
          Fix data convertion problem for Varchar Sub-task Resolved jiangmanhua

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2.5h
          14.
          show long_string_columns in desc table command Sub-task Resolved xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 10m
          15.
          Block alter name/datatype of the long_string_columns Sub-task Closed jiangmanhua  
          16.
          Block alter table property of long_string_columns Sub-task Closed jiangmanhua  
          17.
          Add document for 32k feature Sub-task Resolved xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h
          18.
          Fix data loading problem when table has complex column and long string column Sub-task Resolved jiangmanhua

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3h
          19.
          Support longstring for streaming table Sub-task Open Unassigned  
          20.
          support long string columns with spark FileFormat and SDK with "long_string_columns" TableProperties Sub-task Resolved Ajantha Bhat

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          21.
          Fix bug in writing dataframe to carbon table while the field order is different Sub-task Open xuchuanyin

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                xuchuanyin xuchuanyin
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 75h 10m
                  75h 10m