Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16818

Avoid multiple copies of binary data during the conversion from Result to Row

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: spark
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In the buildRow() of HBaseRelation, CellUtil.cloneValue will already create a copy of the data. If the data type is BinaryType, another copy is being made within Utils.hbaseFieldToScalaType in Utils.scala. Generally, binary data can be fairly large, so copying may be an expensive operation.

        Attachments

        1. HBASE-16818_v0.patch
          0.8 kB
          Weiqing Yang

          Activity

            People

            • Assignee:
              WeiqingYang Weiqing Yang
              Reporter:
              WeiqingYang Weiqing Yang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment