Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-326

Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None
    • None

    Description

      I'd like to propose we have a standard interface for hadoop components, the things that get started or stopped when you bring up a namenode. currently, some of these classes have a stop() or shutdown() method, with no standard name/interface, but no way of seeing if they are live, checking their health of shutting them down reliably. Indeed, there is a tendency for the spawned threads to not want to die; to require the entire process to be killed to stop the workers.

      Having a standard interface would make it easier for

      • management tools to manage the different things
      • monitoring the state of things
      • subclassing

      The latter is interesting as right now TaskTracker and JobTracker start up threads in their constructor; that's very dangerous as subclasses may have their methods called before they are full initialised. Adding this interface would be the right time to clean up the startup process so that subclassing is less risky.

      Attachments

        1. AbstractHadoopComponent.java
          3 kB
          Steve Loughran
        2. hadoop-3628.patch
          162 kB
          Steve Loughran
        3. hadoop-3628.patch
          167 kB
          Steve Loughran
        4. hadoop-3628.patch
          164 kB
          Steve Loughran
        5. hadoop-3628.patch
          164 kB
          Steve Loughran
        6. hadoop-3628.patch
          165 kB
          Steve Loughran
        7. hadoop-3628.patch
          154 kB
          Steve Loughran
        8. hadoop-3628.patch
          142 kB
          Steve Loughran
        9. hadoop-3628.patch
          143 kB
          Steve Loughran
        10. hadoop-3628.patch
          139 kB
          Steve Loughran
        11. hadoop-3628.patch
          152 kB
          Steve Loughran
        12. hadoop-3628.patch
          147 kB
          Steve Loughran
        13. hadoop-3628.patch
          121 kB
          Steve Loughran
        14. hadoop-3628.patch
          96 kB
          Steve Loughran
        15. hadoop-3628.patch
          55 kB
          Steve Loughran
        16. hadoop-3628.patch
          70 kB
          Steve Loughran
        17. hadoop-3628.patch
          55 kB
          Steve Loughran
        18. hadoop-3628.patch
          49 kB
          Steve Loughran
        19. HADOOP-3628-18.patch
          126 kB
          Steve Loughran
        20. HADOOP-3628-19.patch
          347 kB
          Steve Loughran
        21. hadoop-lifecycle.pdf
          321 kB
          Steve Loughran
        22. hadoop-lifecycle.pdf
          316 kB
          Steve Loughran
        23. hadoop-lifecycle.sxw
          143 kB
          Steve Loughran
        24. hadoop-lifecycle-tomw.sxw
          144 kB
          Steve Loughran
        25. HDFS-326.patch
          17 kB
          Steve Loughran

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              3 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: