Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3530

ATS throws exception on trying to filter results without otherinfo.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: timelineserver
    • Labels:
      None
    • Target Version/s:

      Description

      Scenario:
      Am attempting to make data loading faster by fetching otherinfo on demand. As shown in the attached image, the patch adds a 'Load Counters' checkbox. It would be disabled by default, and on clicking, the counter data also would be loaded.
      Issue:
      Things are good when otherinfo is loaded.
      But ATS throws exception on trying to filter on status or applicationId without otherinfo in fields list.
      In other words, using fields=events,primaryfilters with secondaryFilter=status:RUNNING will return

      { "exception": "WebApplicationException", "message": "java.lang.NullPointerException", "javaClassName": "javax.ws.rs.WebApplicationException" }

      from the server.

        Issue Links

          Activity

          Hide
          zjshen Zhijie Shen added a comment -

          The problem is that the fields are set not to include other info, such that that the entity has null other info map. The secondary filter is matched after the entity is loaded. It tries to compare with null other info map, and results in NPE.

          The fix is to always include primary filters and other info into the fields if secondary filters are non-empty for matching, and remove them from the entity before returning it back to the caller.

          Show
          zjshen Zhijie Shen added a comment - The problem is that the fields are set not to include other info, such that that the entity has null other info map. The secondary filter is matched after the entity is loaded. It tries to compare with null other info map, and results in NPE. The fix is to always include primary filters and other info into the fields if secondary filters are non-empty for matching, and remove them from the entity before returning it back to the caller.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 53s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          -1 whitespace 0m 0s The patch has 98 line(s) that end in whitespace.
          +1 javac 7m 48s There were no new javac warning messages.
          +1 javadoc 9m 55s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 4m 5s The applied patch generated 3 additional checkstyle issues.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 0m 45s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 yarn tests 2m 46s Tests passed in hadoop-yarn-server-applicationhistoryservice.
              42m 46s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12727360/YARN-3530.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0ebe84d
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7461/artifact/patchprocess/whitespace.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7461/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/7461/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7461/testReport/
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7461//console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 53s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. -1 whitespace 0m 0s The patch has 98 line(s) that end in whitespace. +1 javac 7m 48s There were no new javac warning messages. +1 javadoc 9m 55s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 4m 5s The applied patch generated 3 additional checkstyle issues. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 45s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 2m 46s Tests passed in hadoop-yarn-server-applicationhistoryservice.     42m 46s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727360/YARN-3530.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0ebe84d whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7461/artifact/patchprocess/whitespace.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7461/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/7461/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7461/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7461//console This message was automatically generated.
          Hide
          zjshen Zhijie Shen added a comment -

          Sreenath Somarajapuram/Prakash Ramachandran, would you please take a look at the patch?

          Show
          zjshen Zhijie Shen added a comment - Sreenath Somarajapuram / Prakash Ramachandran , would you please take a look at the patch?
          Hide
          pramachandran Prakash Ramachandran added a comment -

          LGTM.

          Show
          pramachandran Prakash Ramachandran added a comment - LGTM.
          Hide
          xgong Xuan Gong added a comment -

          +1 LGTM. Will commit

          Show
          xgong Xuan Gong added a comment - +1 LGTM. Will commit
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7686 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7686/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7686 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7686/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          xgong Xuan Gong added a comment -

          Committed into trunk/branch-2. Thanks, zhijie

          Show
          xgong Xuan Gong added a comment - Committed into trunk/branch-2. Thanks, zhijie
          Hide
          Sreenath Sreenath Somarajapuram added a comment -

          Thank you Zhijie Shen.

          Show
          Sreenath Sreenath Somarajapuram added a comment - Thank you Zhijie Shen .
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2109 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2109/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2109 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2109/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/168/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/168/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/177/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/177/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #911 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/911/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #911 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/911/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/178/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/178/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2127 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2127/)
          YARN-3530. ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2127 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2127/ ) YARN-3530 . ATS throws exception on trying to filter results without (xgong: rev 7f07c4d81023e3bf4bf8980e64cc9420ec31cf55) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/LeveldbTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/TimelineStoreTestUtils.java

            People

            • Assignee:
              zjshen Zhijie Shen
              Reporter:
              Sreenath Sreenath Somarajapuram
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development