Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0
    • Component/s: balancer & mover
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      start balancer failed with NPE
      File this issue to track for QE and dev take a look

      balancer.log:
      2013-03-06 00:19:55,174 ERROR org.apache.hadoop.hdfs.server.balancer.Balancer: java.lang.NullPointerException
      at org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy.getInstance(BlockPlacementPolicy.java:165)
      at org.apache.hadoop.hdfs.server.balancer.Balancer.checkReplicationPolicyCompatibility(Balancer.java:799)
      at org.apache.hadoop.hdfs.server.balancer.Balancer.<init>(Balancer.java:808)
      at org.apache.hadoop.hdfs.server.balancer.Balancer.main(Balancer.java:831)

      Balancer.java
      private void checkReplicationPolicyCompatibility(Configuration conf)
      throws UnsupportedActionException {
      if (!(BlockPlacementPolicy.getInstance(conf, null, null) <== here
      instanceof BlockPlacementPolicyDefault))

      { throw new UnsupportedActionException( "Balancer without BlockPlacementPolicyDefault"); }

      }

      1. HDFS-4558.patch
        0.8 kB
        Junping Du
      2. HDFS-4558-v2.patch
        1 kB
        Junping Du
      3. HDFS-4558-v3.patch
        1 kB
        Junping Du

        Activity

        Hide
        Steve Loughran added a comment -

        What's your block placement policy? Something non-standard?

        Show
        Steve Loughran added a comment - What's your block placement policy? Something non-standard?
        Hide
        Junping Du added a comment -

        The conf in running balancer is not initialized well so left as a null value. I will deliver a patch to fix it.

        Show
        Junping Du added a comment - The conf in running balancer is not initialized well so left as a null value. I will deliver a patch to fix it.
        Hide
        Junping Du added a comment -

        This is a pretty simple one, so not include a unit test. I verified it fix my issue (the same with here) in running balancer against branch-1.

        Show
        Junping Du added a comment - This is a pretty simple one, so not include a unit test. I verified it fix my issue (the same with here) in running balancer against branch-1.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        It seems that this is a 1.2.0 only issue. 1.1.2 should not have this problem.

        Show
        Tsz Wo Nicholas Sze added a comment - It seems that this is a 1.2.0 only issue. 1.1.2 should not have this problem.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Junping,

        How about we simply remove the default constructor and create a new conf in main(..)?

        Show
        Tsz Wo Nicholas Sze added a comment - Hi Junping, How about we simply remove the default constructor and create a new conf in main(..)?
        Hide
        Junping Du added a comment -

        Yes. It should be 1.2.0 issue.
        That's a better idea as main is the only caller for this default constructor which is not necessary now. I will update a patch soon.

        Show
        Junping Du added a comment - Yes. It should be 1.2.0 issue. That's a better idea as main is the only caller for this default constructor which is not necessary now. I will update a patch soon.
        Hide
        Junping Du added a comment -

        Update as Nicholas suggestion in v2 patch. BTW, The default constructor of balancer consumed by main can work before because its config is initialized later in ToolRunner. After adding check replication policy, its conf cannot be null in construct stage, so here we remove the default constructor.

        Show
        Junping Du added a comment - Update as Nicholas suggestion in v2 patch. BTW, The default constructor of balancer consumed by main can work before because its config is initialized later in ToolRunner. After adding check replication policy, its conf cannot be null in construct stage, so here we remove the default constructor.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Thanks Junping, the conf also needs to be passed to ToolRunner. Otherwise, we will create two configurations.

        Show
        Tsz Wo Nicholas Sze added a comment - Thanks Junping, the conf also needs to be passed to ToolRunner. Otherwise, we will create two configurations.
        Hide
        Junping Du added a comment -

        Ooops. You are right, Nicholas. v3 is the right one.

        Show
        Junping Du added a comment - Ooops. You are right, Nicholas. v3 is the right one.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this. Thanks, Junping!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Junping!
        Hide
        Junping Du added a comment -

        Thanks for review and comments. Nicholas!

        Show
        Junping Du added a comment - Thanks for review and comments. Nicholas!
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.

          People

          • Assignee:
            Junping Du
            Reporter:
            Wenwu Peng
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development