Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Bytes.toInt shows up quite often in a profiler run.
      It turns out that one source is HFileReaderV2$ScannerV2.getKeyValue().

      Notice that we call the KeyValue(byte[], int) constructor, which forces the constructor to determine its size by reading some of the header information and calculate the size. In this case, however, we already know the size (from the call to readKeyValueLen), so we could just use that.

      In the extreme case of 10000's of columns this noticeably reduces CPU.

        Attachments

        1. 6621-0.96-v4.txt
          2 kB
          Lars Hofhansl
        2. 6621-0.96-v3.txt
          3 kB
          Lars Hofhansl
        3. 6621-0.96-v2.txt
          2 kB
          Lars Hofhansl
        4. 6621-0.96.txt
          0.7 kB
          Lars Hofhansl

          Activity

            People

            • Assignee:
              lhofhansl Lars Hofhansl
              Reporter:
              lhofhansl Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: