Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-1429

Add a value based compression for decimal data type when decimal is stored as Int or Long

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Added a value based compression for decimal data type when decimal is stored as Int or Long

      When decimal precision is <= 9, decimal values are stored in 4 bytes but they are not compressed further based on min and max values as compared with other primitive data type compression. Therefore now based on min and max value decimal data falling in Integer range will be further compressed as byte or short.
      When decimal precision is <= 18, decimal values are stored in 8 bytes but they are not compressed further based on min and max values as compared with other primitive data type compression. Therefore now based on min and max value decimal data falling in Long range will be further compressed as byte, short or int.
      Advantage: This will reduce the storage space thereby decreasing the IO time while reading and decompressing the data.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                manishgupta_88 Manish Gupta
                Reporter:
                manishgupta_88 Manish Gupta
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 7h
                  7h