Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14074

DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace.

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.8.0, 3.0.0
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0
    • Component/s: hdfs
    • Labels:
    • Environment:

      hadoop-2.7.3, hadoop-2.8.0

    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Tags:
      HDFS, DiskChecks
    • Flags:
      Patch

      Description

      In ThrottledAsyncChecker class,it members of the completedChecks is WeakHashMap, its definition is as follows:
           this.completedChecks = new WeakHashMap<>();

      and one of its uses is as follows in schedule method:
          if (completedChecks.containsKey(target)) 

      {         // here may be happen garbage collection,and result may be null.        final LastCheckResult<V> result = completedChecks.get(target);                  final long msSinceLastCheck = timer.monotonicNow() - result.completedAt;            。。。。 }

      after  "completedChecks.containsKey(target)",  may be happen garbage collection,  and result may be null.

      the solution is:

      this.completedChecks = new ReferenceMap(1, 1);
      or
      this.completedChecks = new HashMap<>();

       

        Attachments

        1. HDFS-14074-latest.patch
          1 kB
          guangyi lu
        2. HDFS-14074.patch
          2 kB
          guangyi lu
        3. WechatIMG83.jpeg
          88 kB
          guangyi lu

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                luguangyi guangyi lu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 48h
                  48h
                  Remaining:
                  Remaining Estimate - 48h
                  48h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified