Hadoop Common
  1. Hadoop Common
  2. HADOOP-8765

LocalDirAllocator.ifExists API is broken and unused

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None

      Description

      LocalDirAllocator.ifExists calls AllocatorPerContext.ifExists, which accesses the localDirsPath variable that is uninitialised. Hence, any code that uses this API is likely to fail with a NullPointerException.

      However, this API is currently not used anywhere else in trunk. The earlier usage was in IsolationRunner, that has since been removed via MAPREDUCE-2606.

      Hence, we could remove this API for trunk, and if required, fix it for the 1.x branch.

      1. HADOOP-8765-for-1-0-x.patch
        3 kB
        Hemanth Yamijala
      2. HADOOP-8765.patch
        2 kB
        Hemanth Yamijala

        Activity

        Hide
        Hemanth Yamijala added a comment -

        BTW, all other calls to LocalDirAllocator first call AllocatorPerContext.confChanged, that is responsible for initialising the localDirsPath. Inserting that call fixes the NullPointerException.

        Show
        Hemanth Yamijala added a comment - BTW, all other calls to LocalDirAllocator first call AllocatorPerContext.confChanged, that is responsible for initialising the localDirsPath. Inserting that call fixes the NullPointerException.
        Hide
        Hemanth Yamijala added a comment -

        Attached patch removes the ifExists API and AllocatorPerContext.ifExists API as well. Could someone please review ?

        Show
        Hemanth Yamijala added a comment - Attached patch removes the ifExists API and AllocatorPerContext.ifExists API as well. Could someone please review ?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12543815/HADOOP-8765.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1404//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1404//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543815/HADOOP-8765.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1404//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1404//console This message is automatically generated.
        Hide
        Hemanth Yamijala added a comment -

        -1 tests included. The patch doesn't appear to include any new or modified tests.

        Please justify why no new tests are needed for this patch.

        Also please list what manual steps were performed to verify this patch.

        The patch removes code. Hence, no new tests.

        Show
        Hemanth Yamijala added a comment - -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. The patch removes code. Hence, no new tests.
        Hide
        Hemanth Yamijala added a comment -

        Patch for branch 1.x, where the ifExists API is still used by IsolationRunner.

        While writing a test for this, I realized that mostly the usage pattern of LocalDirAllocator is going to first do a write / read that will end up initializing the localDirsPath variable. Hence, the significance of this patch is only to the extent that it got IsolationRunner working for me and no real impact anywhere else.

        Show
        Hemanth Yamijala added a comment - Patch for branch 1.x, where the ifExists API is still used by IsolationRunner. While writing a test for this, I realized that mostly the usage pattern of LocalDirAllocator is going to first do a write / read that will end up initializing the localDirsPath variable. Hence, the significance of this patch is only to the extent that it got IsolationRunner working for me and no real impact anywhere else.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544036/HADOOP-8765-for-1-0-x.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1409//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544036/HADOOP-8765-for-1-0-x.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1409//console This message is automatically generated.

          People

          • Assignee:
            Hemanth Yamijala
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development