Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      I am conducting research on log related bugs. I tried to make a tool to fix repetitive yet simple patterns of bugs that are related to logs. Typos in log messages are one of the reoccurring bugs. Therefore, I made a tool find typos in log statements. During my experiments, I managed to find the following typos in Hadoop YARN:

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java, LOG.info("AsyncDispatcher is draining to stop igonring any new events."),
      igonring should be ignoring

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.java, LOG.info(authorizerClass.getName() + " is instiantiated."),
      instiantiated should be instantiated

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/FileSystemApplicationHistoryStore.java, LOG.info("Completed reading history information of all conatiners"+ " of application attempt " + appAttemptId),
      conatiners should be containers

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java, LOG.info("Neither virutal-memory nor physical-memory monitoring is " +"needed. Not running the monitor-thread"),
      virutal should be virtual

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/reservation/AbstractReservationSystem.java, LOG.info("Intialized plan {} based on reservable queue {}" plan.toString() planQueueName),
      Intialized should be Initialized

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java, LOG.info("Initializing " + queueName + "\n" +"capacity = " + queueCapacities.getCapacity() +" [= (float) configuredCapacity / 100 ]" + "\n" +"asboluteCapacity = " + queueCapacities.getAbsoluteCapacity() +" [= parentAbsoluteCapacity * capacity ]" + "\n" +"maxCapacity = " + queueCapacities.getMaximumCapacity() +" [= configuredMaxCapacity ]" + "\n" +"absoluteMaxCapacity = " + queueCapacities.getAbsoluteMaximumCapacity() +" [= 1.0 maximumCapacity undefined " +"(parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]" +"\n" +"userLimit = " + userLimit +" [= configuredUserLimit ]" + "\n" +"userLimitFactor = " + userLimitFactor +" [= configuredUserLimitFactor ]" + "\n" +"maxApplications = " + maxApplications +" [= configuredMaximumSystemApplicationsPerQueue or" +" (int)(configuredMaximumSystemApplications * absoluteCapacity)]" +"\n" +"maxApplicationsPerUser = " + maxApplicationsPerUser +" [= (int)(maxApplications * (userLimit / 100.0f) * " +"userLimitFactor) ]" + "\n" +"usedCapacity = " + queueCapacities.getUsedCapacity() +" [= usedResourcesMemory / " +"(clusterResourceMemory * absoluteCapacity)]" + "\n" +"absoluteUsedCapacity = " + absoluteUsedCapacity +" [= usedResourcesMemory / clusterResourceMemory]" + "\n" +"maxAMResourcePerQueuePercent = " + maxAMResourcePerQueuePercent +" [= configuredMaximumAMResourcePercent ]" + "\n" +"minimumAllocationFactor = " + minimumAllocationFactor +" [= (float)(maximumAllocationMemory - minimumAllocationMemory) / " +"maximumAllocationMemory ]" + "\n" +"maximumAllocation = " + maximumAllocation +" [= configuredMaxAllocation ]" + "\n" +"numContainers = " + numContainers +" [= currentNumContainers ]" + "\n" +"state = " + state +" [= configuredState ]" + "\n" +"acls = " + aclsString +" [= configuredAcls ]" + "\n" +"nodeLocalityDelay = " + nodeLocalityDelay + "\n" +"labels=" + labelStrBuilder.toString() + "\n" +"reservationsContinueLooking = " +reservationsContinueLooking + "\n" +"preemptionDisabled = " + getPreemptionDisabled() + "\n" +"defaultAppPriorityPerQueue = " + defaultAppPriorityPerQueue),
      asbolute should be absolute

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking),
      asbolute should be absolute

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java, LOG.info(queueName +" capacity=" + this.queueCapacities.getCapacity() +" asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +" maxCapacity=" + this.queueCapacities.getMaximumCapacity() +" asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() +" state=" + state +" acls=" + aclsString +" labels=" + labelStrBuilder.toString() + "\n" +" reservationsContinueLooking=" + reservationsContinueLooking), asbolute should be absolute

      In file /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java, LOG.info("Calling allocate on removed " +"or non existant application " + appAttemptId),
      existant should be existent

        Attachments

        1. YARN-5642.002.patch
          10 kB
          Mehran Hassani
        2. YARN-5642.001.patch
          10 kB
          Mehran Hassani

          Activity

            People

            • Assignee:
              MehranHassani Mehran Hassani
              Reporter:
              MehranHassani Mehran Hassani
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: