Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9359

Avoid code duplication in Resources for calculation methods



    • Type: Improvement
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:


      This is a follow-up for YARN-9318, dealing with code duplication issueas, as discussed with Daniel Templeton earlier.

      Resources has many very similar calculation methods like addTo, subtractFrom, multiply, etc.
      These are having extractable code as common, the only difference could be the calculation they perform on the passed Resource object(s).

      These methods either receive one or two Resource objects and make some calculations on these.
      One caveat that needs some attention is that some of them do clone the Resource and do the calculation on the cloned resource and return the result (leaving the passed Resource alone) and some of them perform the calculation on the passed Resource object itself.

      The common code could be extracted like this:

      private static Resource applyFunctionOnValues(Resource lhs,
            Function<Long, Long> valueFunction) {
          int numResources = ResourceUtils.getNumberOfCountableResourceTypes();
          for (int i = 0; i < numResources; i++) {
            try {
              ResourceInformation lhsValue = lhs.getResourceInformation(i);
              Long modifiedValue = valueFunction.apply(lhsValue.getValue());
              lhs.setResourceValue(i, modifiedValue);
            } catch (ResourceNotFoundException ye) {
              LOG.warn("Resource is missing:" + ye.getMessage());
          return lhs;

      And an example code could be like this:

      public static Resource multiplyAndRoundUp(Resource lhs, double by) {
          return applyFunctionOnValues(clone(lhs),
              (value) -> (long) Math.ceil(value * by));




            • Assignee:
              smileLee WEI-HSIAO-LEE
              snemeth Szilard Nemeth
            • Votes:
              0 Vote for this issue
              4 Start watching this issue


              • Created: