Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6738

LevelDBCacheTimelineStore should reuse ObjectMapper instances

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Using TezUI sometimes times out...and the cause of it was that the query was quite large; and the leveldb handler seems like recreates the ObjectMapper for every read - this is unfortunate; since the ObjectMapper have to rescan the class annotations which may take some time.

      Keeping the objectmapper reduces the ATS call time from 17 seconds to 3 seconds for me...which was enough to get my tez-ui working again

      1. Screen Shot 2017-06-23 at 2.43.06 PM.png
        83 kB
        Zoltan Haindrich
      2. YARN-6738.1.patch
        1 kB
        Zoltan Haindrich
      3. YARN-6738.2.patch
        1 kB
        Zoltan Haindrich

        Activity

        Hide
        kgyrtkirk Zoltan Haindrich added a comment -

        Thank you for adding me to the contributor list... and for reviewing and committing this change!

        Show
        kgyrtkirk Zoltan Haindrich added a comment - Thank you for adding me to the contributor list... and for reviewing and committing this change!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11934 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11934/)
        YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper (jlowe: rev 63ce1593c5b78eb172773e7498d9c321debe81e8)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/LevelDBCacheTimelineStore.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11934 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11934/ ) YARN-6738 . LevelDBCacheTimelineStore should reuse ObjectMapper (jlowe: rev 63ce1593c5b78eb172773e7498d9c321debe81e8) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/LevelDBCacheTimelineStore.java
        Hide
        jlowe Jason Lowe added a comment -

        Thanks to Zoltan Haindrich for the contribution and to Junping Du for additional review! I committed this to trunk, branch-2, and branch-2.8.

        Show
        jlowe Jason Lowe added a comment - Thanks to Zoltan Haindrich for the contribution and to Junping Du for additional review! I committed this to trunk, branch-2, and branch-2.8.
        Hide
        djp Junping Du added a comment -

        +1. Patch LGTM as well.

        Show
        djp Junping Du added a comment - +1. Patch LGTM as well.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for the report and patch!

        +1 lgtm. I'll commit this tomorrow if there are no objections.

        Show
        jlowe Jason Lowe added a comment - Thanks for the report and patch! +1 lgtm. I'll commit this tomorrow if there are no objections.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 15m 44s trunk passed
        +1 compile 0m 17s trunk passed
        +1 checkstyle 0m 12s trunk passed
        +1 mvnsite 0m 20s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 16s the patch passed
        +1 javac 0m 16s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 17s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 34s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 1m 8s hadoop-yarn-server-timeline-pluginstorage in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        21m 57s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6738
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874302/YARN-6738.2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 27e6dd9e91e1 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / abdea26
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16237/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16237/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 15m 44s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 20s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 17s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 34s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 1m 8s hadoop-yarn-server-timeline-pluginstorage in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 21m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6738 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874302/YARN-6738.2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 27e6dd9e91e1 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / abdea26 Default Java 1.8.0_131 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16237/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage Console output https://builds.apache.org/job/PreCommit-YARN-Build/16237/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kgyrtkirk Zoltan Haindrich added a comment -

        #2) fix findbugs issue

        I can't really add a test for this...because this patch only enables to reuse ObjectMapper-s

        Show
        kgyrtkirk Zoltan Haindrich added a comment - #2) fix findbugs issue I can't really add a test for this...because this patch only enables to reuse ObjectMapper-s
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 14m 1s trunk passed
        +1 compile 0m 16s trunk passed
        +1 checkstyle 0m 12s trunk passed
        +1 mvnsite 0m 18s trunk passed
        +1 findbugs 0m 23s trunk passed
        +1 javadoc 0m 12s trunk passed
        +1 mvninstall 0m 14s the patch passed
        +1 compile 0m 14s the patch passed
        +1 javac 0m 14s the patch passed
        -0 checkstyle 0m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3)
        +1 mvnsite 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 28s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 0m 58s hadoop-yarn-server-timeline-pluginstorage in the patch passed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        19m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6738
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874262/YARN-6738.1.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fc6b045d31fc 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e5db9af
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16228/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16228/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16228/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 14m 1s trunk passed +1 compile 0m 16s trunk passed +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 18s trunk passed +1 findbugs 0m 23s trunk passed +1 javadoc 0m 12s trunk passed +1 mvninstall 0m 14s the patch passed +1 compile 0m 14s the patch passed +1 javac 0m 14s the patch passed -0 checkstyle 0m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) +1 mvnsite 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 28s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 0m 58s hadoop-yarn-server-timeline-pluginstorage in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 19m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6738 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874262/YARN-6738.1.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fc6b045d31fc 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e5db9af Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16228/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16228/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage Console output https://builds.apache.org/job/PreCommit-YARN-Build/16228/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

          People

          • Assignee:
            kgyrtkirk Zoltan Haindrich
            Reporter:
            kgyrtkirk Zoltan Haindrich
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development