Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The mapreduce job history server currently needs to be deployed as a trusted server in sync with the mapreduce runtime. Every new application would need a similar application history server. Having to deploy O(T*V) (where T is number of type of application, V is number of version of application) trusted servers is clearly not scalable.

      Job history storage handling itself is pretty generic: move the logs and history data into a particular directory for later serving. Job history data is already stored as json (or binary avro). I propose that we create only one trusted application history server, which can have a generic UI (display json as a tree of strings) as well. Specific application/version can deploy untrusted webapps (a la AMs) to query the application history server and interpret the json for its specific UI and/or analytics.

      1. HistoryStorageDemo.java
        3 kB
        Zhijie Shen
      2. Generic Application History - Design-20131219.pdf
        103 kB
        Vinod Kumar Vavilapalli
      3. ApplicationHistoryServiceHighLevel.pdf
        55 kB
        Sandy Ryza
      4. AHS Diagram.pdf
        287 kB
        Zhijie Shen

        Issue Links

          Activity

          Junping Du made changes -
          Link This issue is related to YARN-374 [ YARN-374 ]
          Sangjin Lee made changes -
          Link This issue relates to YARN-2928 [ YARN-2928 ]
          Jian He made changes -
          Assignee Yu Gao [ crystal_gaoyu ]
          Yu Gao made changes -
          Assignee Yu Gao [ crystal_gaoyu ]
          Jake Farrell made changes -
          Comment [ Compared to wrists well is less available status amphetamines but higher investigations of withdrawal.
          adderall 20 mg http://www.surveyanalytics.com//userimages/sub-2/2007589/3153260/29851520/7787428-29851520-stopadd3.html
          Areas also document any reasons they have surprisingly been using in the information. ]
          Zhijie Shen made changes -
          Link This issue is related to YARN-1982 [ YARN-1982 ]
          Zhijie Shen made changes -
          Link This issue is related to YARN-1701 [ YARN-1701 ]
          Zhijie Shen made changes -
          Link This issue is related to YARN-1452 [ YARN-1452 ]
          Zhijie Shen made changes -
          Assignee Zhijie Shen [ zjshen ]
          Zhijie Shen made changes -
          Assignee Zhijie Shen [ zjshen ]
          Zhijie Shen made changes -
          Link This issue is duplicated by YARN-304 [ YARN-304 ]
          Vinod Kumar Vavilapalli made changes -
          Assignee Vinod Kumar Vavilapalli [ vinodkv ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue relates to YARN-1530 [ YARN-1530 ]
          Vinod Kumar Vavilapalli made changes -
          Sandy Ryza made changes -
          Attachment ApplicationHistoryServiceHighLevel.pdf [ 12607513 ]
          Zhijie Shen made changes -
          Attachment AHS Diagram.pdf [ 12607509 ]
          Zhijie Shen made changes -
          Attachment HistoryStorageDemo.java [ 12592495 ]
          Omkar Vinit Joshi made changes -
          Link This issue relates to YARN-374 [ YARN-374 ]
          Vinod Kumar Vavilapalli made changes -
          Project Hadoop Map/Reduce [ 12310941 ] Hadoop YARN [ 12313722 ]
          Key MAPREDUCE-3061 YARN-321
          Affects Version/s 0.23.0 [ 12315570 ]
          Target Version/s 0.24.0 [ 12317654 ]
          Fix Version/s 0.24.0 [ 12317654 ]
          Component/s mrv2 [ 12314301 ]
          Tags mrv2, history server
          Vinod Kumar Vavilapalli made changes -
          Assignee Mayank Bansal [ mayank_bansal ] Vinod Kumar Vavilapalli [ vinodkv ]
          Mayank Bansal made changes -
          Field Original Value New Value
          Assignee Mayank Bansal [ mayank_bansal ]
          Luke Lu created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Luke Lu
            • Votes:
              1 Vote for this issue
              Watchers:
              67 Start watching this issue

              Dates

              • Created:
                Updated:

                Development