Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1334

contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: contrib/index
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      $ cd src/contrib/index
      $ ant clean test

      This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames - data structure, when it is being run with different parameters.

      (ArrayIndexOutOfBoundsException raised when inserting elements in doneFileNames, array ).

      Debugging further - there seems to be an additional file called as - hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in addition to those that begins with done* . The presence of the extra file causes the error.

      Attaching a patch that would circumvent this by increasing the array length of shards by 1 .

      But longer term the test fixtures need to be probably revisited to see if the presence of _SUCCESS as a file is a good thing to begin with before we even get to this test case.

      Any comments / suggestions on the same welcome.

      1. MAPREDUCE-1334.patch
        0.8 kB
        Hemanth Yamijala
      2. MAPREDUCE-1334.patch
        1 kB
        Karthik K

        Activity

        Karthik K created issue -
        Karthik K made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-1334.patch [ 12428949 ]
        Karthik K made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Karthik K made changes -
        Attachment MAPREDUCE-1334.patch [ 12429081 ]
        Karthik K made changes -
        Attachment MAPREDUCE-1334.patch [ 12428949 ]
        Karthik K made changes -
        Component/s contrib/index [ 12313220 ]
        Component/s test [ 12312904 ]
        Karthik K made changes -
        Priority Major [ 3 ] Critical [ 2 ]
        Karthik K made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Karthik K made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Karthik K made changes -
        Fix Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.21.0 [ 12314045 ]
        Priority Critical [ 2 ] Major [ 3 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hemanth Yamijala made changes -
        Attachment MAPREDUCE-1334.patch [ 12459017 ]
        Hemanth Yamijala made changes -
        Assignee Kay Kay [ kaykay.unique ]
        Hemanth Yamijala made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hemanth Yamijala made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Karthik K
            Reporter:
            Karthik K
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development