Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-1429

Add a value based compression for decimal data type when decimal is stored as Int or Long

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Added a value based compression for decimal data type when decimal is stored as Int or Long

      When decimal precision is <= 9, decimal values are stored in 4 bytes but they are not compressed further based on min and max values as compared with other primitive data type compression. Therefore now based on min and max value decimal data falling in Integer range will be further compressed as byte or short.
      When decimal precision is <= 18, decimal values are stored in 8 bytes but they are not compressed further based on min and max values as compared with other primitive data type compression. Therefore now based on min and max value decimal data falling in Long range will be further compressed as byte, short or int.
      Advantage: This will reduce the storage space thereby decreasing the IO time while reading and decompressing the data.

        Issue Links

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              manishgupta_88 Manish Gupta
              Reporter:
              manishgupta_88 Manish Gupta
              Request participants:
              None
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 7h
                7h