Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1580

Use 1 byte instead of 8 bytes as column indicator in fact distinct MR job

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.5.2
    • Component/s: None
    • Labels:
      None

      Description

      One small enhancement in fact distinct job; today it uses 8 bytes in the head of the key to indicates the column index as well as cuboid id; actually it can be optimized to 1 byte for column index, and next 8 for cuboid id if the first is a special value; this will reduce the network IO between mappers and reducers.

        Attachments

          Activity

            People

            • Assignee:
              shaofengshi Shao Feng Shi
              Reporter:
              shaofengshi Shao Feng Shi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: