Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-91

Use hdfs v-blocks instead of zero-padding stripes

          Details

          • Type: Improvement
          • Status: Closed
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 1.5.0
          • Component/s: None
          • Labels:
            None

            Description

            HDFS-3689 added variable length blocks to HDFS as a core feature.

            hsync(SyncFlag.END_BLOCK)

            can now end an HDFS block on disk without padding it out with zeros.

            The current space wasted in ORC to padding is 5%.

              Attachments

                Activity

                  People

                  • Assignee:
                    gopalv Gopal V
                    Reporter:
                    gopalv Gopal V
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    6 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: