Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-326

Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I'd like to propose we have a standard interface for hadoop components, the things that get started or stopped when you bring up a namenode. currently, some of these classes have a stop() or shutdown() method, with no standard name/interface, but no way of seeing if they are live, checking their health of shutting them down reliably. Indeed, there is a tendency for the spawned threads to not want to die; to require the entire process to be killed to stop the workers.

      Having a standard interface would make it easier for

      • management tools to manage the different things
      • monitoring the state of things
      • subclassing

      The latter is interesting as right now TaskTracker and JobTracker start up threads in their constructor; that's very dangerous as subclasses may have their methods called before they are full initialised. Adding this interface would be the right time to clean up the startup process so that subclassing is less risky.

        Attachments

        1. AbstractHadoopComponent.java
          3 kB
          Steve Loughran
        2. hadoop-3628.patch
          49 kB
          Steve Loughran
        3. hadoop-3628.patch
          55 kB
          Steve Loughran
        4. hadoop-3628.patch
          70 kB
          Steve Loughran
        5. hadoop-3628.patch
          55 kB
          Steve Loughran
        6. hadoop-3628.patch
          96 kB
          Steve Loughran
        7. hadoop-3628.patch
          121 kB
          Steve Loughran
        8. hadoop-3628.patch
          147 kB
          Steve Loughran
        9. hadoop-3628.patch
          152 kB
          Steve Loughran
        10. hadoop-3628.patch
          139 kB
          Steve Loughran
        11. hadoop-3628.patch
          143 kB
          Steve Loughran
        12. hadoop-3628.patch
          142 kB
          Steve Loughran
        13. hadoop-lifecycle.sxw
          143 kB
          Steve Loughran
        14. hadoop-lifecycle.pdf
          316 kB
          Steve Loughran
        15. hadoop-3628.patch
          154 kB
          Steve Loughran
        16. hadoop-3628.patch
          165 kB
          Steve Loughran
        17. hadoop-3628.patch
          164 kB
          Steve Loughran
        18. hadoop-lifecycle-tomw.sxw
          144 kB
          Steve Loughran
        19. hadoop-3628.patch
          164 kB
          Steve Loughran
        20. hadoop-3628.patch
          167 kB
          Steve Loughran
        21. hadoop-3628.patch
          162 kB
          Steve Loughran
        22. HADOOP-3628-18.patch
          126 kB
          Steve Loughran
        23. HADOOP-3628-19.patch
          347 kB
          Steve Loughran
        24. hadoop-lifecycle.pdf
          321 kB
          Steve Loughran
        25. HDFS-326.patch
          17 kB
          Steve Loughran

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                3 Vote for this issue
                Watchers:
                32 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: