Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: REEF Driver, REEF Evaluator
    • Labels:

      Description

      Currently REEF Driver and Evaluator start only as an application, that is, the only entry point is static main(). We need to refactor the code so we can start REEF components from another Java or C# application in the same VM.

        Attachments

          Issue Links

          1.
          Refactor logic related to Evaluator state Sub-task Resolved Sergiy Matusevych  
          2.
          Refactor REEF Driver logic for code readability and add more javadocs Sub-task Resolved Sergiy Matusevych  
          3.
          Refactor HandlerContainer to improve safety, readability, and performance Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 48h
          Remaining Estimate - 48h
          4.
          Make RuntimeClock more testable and create unit tests to check graceful and forceful shutdown of RuntimeClock Sub-task Closed Sergiy Matusevych

          0%

          Original Estimate - 4h
          Remaining Estimate - 4h
          5.
          Graceful shutdown of the RuntimeClock is not so graceful: unconditionally clears the event queue Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 48h
          Remaining Estimate - 48h
          6.
          Improve readability of code and logs in REEF launcher Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 4h
          Remaining Estimate - 4h
          7.
          Split REEFLauncher into executable part and the Clock-running environment Sub-task Resolved Sergiy Matusevych  
          8.
          Implement proper management of all threads running in REEF and its applications Sub-task In Progress Sergiy Matusevych  
          9.
          Get rid of global exception handlers and other process-global features in REEF Sub-task In Progress Sergiy Matusevych  
          10.
          Add the ability to check the status of the REEF job launched directly via REEFEnvironment Driver. Sub-task Resolved Sergiy Matusevych  
          11.
          Create more unit tests to verify error handling in the driver launched via REEFEnvironment Sub-task Resolved Sergiy Matusevych  
          12.
          Assign name and thread group to each thread that manages local containers on the Driver side Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 4h
          Remaining Estimate - 4h
          13.
          Refactor RunnableProcess and move State and related logic into a separate enum Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 2h
          Remaining Estimate - 2h
          14.
          Check for singletons per REEFEnvironment, not per entire JVM process Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 24h
          Remaining Estimate - 24h
          15.
          Implement timeout for FailClient.runInProcess() and other similar unit tests Sub-task Open Sergiy Matusevych

          0%

          Original Estimate - 48h
          Remaining Estimate - 48h
          16.
          Improve logging in REEF job shutdown Sub-task Resolved Sergiy Matusevych  
          17.
          Improve logging in REEF Remote Manager and related classes Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 24h
          Remaining Estimate - 24h
          18.
          Implement graceful shutdown of Wake executor services Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 168h
          Remaining Estimate - 168h
          19.
          Close the driver restart manager on driver shutdown Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 1h
          Remaining Estimate - 1h
          20.
          Implement proper logging of threads still running at the end of the REEF process Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 24h
          Remaining Estimate - 24h
          21.
          Give REEF threads better names and improve logging during the closing phase of REEF application Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 48h
          Remaining Estimate - 48h
          22.
          Gracefully shut down all threads at the end of the REEF job Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 336h
          Remaining Estimate - 336h
          23.
          RejectedExecutionException thrown when closing the acceptor in NettyMessageTransport Sub-task Resolved Sergiy Matusevych  
          24.
          REEF-on-REEF example Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 168h
          Remaining Estimate - 168h
          25.
          Allow REEF Driver to run in unmanaged AM mode Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 24h
          Remaining Estimate - 24h
          26.
          Create a variant of YarnClientConfiguration for Unmanaged AM Sub-task Resolved Sergiy Matusevych

          0%

          Original Estimate - 48h
          Remaining Estimate - 48h

            Activity

              People

              • Assignee:
                motus Sergiy Matusevych
                Reporter:
                motus Sergiy Matusevych
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 1,023h
                  1,023h
                  Remaining:
                  Remaining Estimate - 1,023h
                  1,023h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified