Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1321

bugs with temp directories, trailing blank fields in HBase bulk load

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 0.6.0
        • Fix Version/s: 0.6.0
        • Component/s: HBase Handler
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          HIVE-1295 had two bugs discovered during testing with production data:

          (1) extra directories may be present in the output directory depending on how the cluster is configured; we need to walk down these to find the column family directory

          (2) if a record ends with fields which are blank strings, the text format omits the corresponding Control-A delimiters, so we need to fill in blanks for these fields (instead of throwing ArrayIndexOutOfBoundsException)

            Issue Links

            relates to

            Improvement - An improvement or enhancement to an existing feature or task. HIVE-1295 facilitate HBase bulk loads from Hive

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              jvs John Sichi added a comment -

              Also need to omit null values (\N) completely.

              Show
              jvs John Sichi added a comment - Also need to omit null values (\N) completely.
              Hide
              Permalink
              jvs John Sichi added a comment -

              Changed test script to exercise blanks and nulls, and did some manual testing to verify that these got into HBase (or not) as expected.

              Show
              jvs John Sichi added a comment - Changed test script to exercise blanks and nulls, and did some manual testing to verify that these got into HBase (or not) as expected.
              Hide
              Permalink
              namit Namit Jain added a comment -

              Committed. Thanks John

              Show
              namit Namit Jain added a comment - Committed. Thanks John

                People

                • Assignee:
                  jvs John Sichi
                  Reporter:
                  jvs John Sichi
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  0 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development