Hadoop Common
  1. Hadoop Common
  2. HADOOP-7651

Hadoop Record compiler generates Java files with erroneous byte-array lengths for fields trailing a 'ustring' field

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: record
    • Tags:
      Hadoop Record Compiler

      Description

      Hadoop Record compiler produces Java files from a DDL file. If a DDL file has a class that contains a 'ustring' field, then the generated 'compareRaw()' function for this record is erroneous in computing the length of remaining bytes after the logic of computing the buffer segment for a 'ustring' field.

      Below is a line in a generated 'compareRaw()' function for a record class with a 'ustring' field :
      s1+=i1; s2+=i2; l1-=i1; l1-=i2;
      This line shoud be corrected by changing the last 'l1' to 'l2':
      s1+=i1; s2+=i2; l1-=i1; l2-=i2;

      To fix this bug, one should correct the 'genCompareBytes()' function in the 'JString.java' file of the package 'org.apache.hadoop.record.compiler' by changing the line below to the ensuing line. There is only one digit difference:

      cb.append("s1+=i1; s2+=i2; l1-=i1; l1-=i2;\n");

      cb.append("s1+=i1; s2+=i2; l1-=i1; l2-=i2;\n");

      This bug is serious as it will always crash unserializing a record with a simple definition like the one below
      class PairStringDouble

      { ustring first; double second; }

      Unserializing a record of this class will throw an exception as the 'second' field does not have 8 bytes for a double value due to the erroneous length computation for the remaining buffer.

      Both Hadoop 0.20 and 0.21 have this bug.

      1. 7651-22.patch
        0.7 kB
        Milind Bhandarkar
      2. 7651-trunk.patch
        0.8 kB
        Milind Bhandarkar

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Milind Bhandarkar
            Reporter:
            Hung-chih Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 1m
              1m
              Remaining:
              Remaining Estimate - 1m
              1m
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development