Flume
  1. Flume
  2. FLUME-1006

PID creation of starting flume agents

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: v1.1.0
    • Fix Version/s: None
    • Component/s: Configuration
    • Labels:

      Description

      New class to let new flume agents write their own pid. Thats we need in future to start flume agents from init, for monitoring too.

      1. getPID.java
        1 kB
        Alexander Alten-Lorenz
      2. getThread.java
        1 kB
        Alexander Alten-Lorenz

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        36s 1 Alexander Alten-Lorenz 28/Feb/12 10:28
        Alexander Alten-Lorenz made changes -
        Assignee Alexander Lorenz-Alten [ alo.alt ]
        Hide
        Bruno Mahé added a comment -

        Not sure if we all refer to the same thing when we talk about init scripts but from an Apache Bigtop (incubating) point of view, here is what we do:
        The init script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/flume-node.init
        The wrapper script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/install_flume.sh#L116

        It feels a little bit hackish right now due to the way flume was spawning itself in 1.0.0, but some patches were sent and the situation should be much better for the coming version of Apache Flume (incubating).
        But in any case, please keep things simple and stupid

        Show
        Bruno Mahé added a comment - Not sure if we all refer to the same thing when we talk about init scripts but from an Apache Bigtop (incubating) point of view, here is what we do: The init script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/flume-node.init The wrapper script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/install_flume.sh#L116 It feels a little bit hackish right now due to the way flume was spawning itself in 1.0.0, but some patches were sent and the situation should be much better for the coming version of Apache Flume (incubating). But in any case, please keep things simple and stupid
        Hide
        Alexander Alten-Lorenz added a comment -

        Hi Brock,

        to manage flume later we need a working PID environment, at the moment I found nothing similar. First flume have to start from a valid init script, here we need the PID of the process himself (getPID). After, when flume runs, flume could have a lot of channels in a configuration. For all of that channels (single threads) we need the PID of the thread too, to manage the flows (kill, status etc). For that I thought two separate classes are will be fine, the first get the PID of the flume-init process back (as .pid via /var/run/ or /proc/), the second a PID of a running thread for easier management. Its a idea, not more

        thanks,
        Alex

        Show
        Alexander Alten-Lorenz added a comment - Hi Brock, to manage flume later we need a working PID environment, at the moment I found nothing similar. First flume have to start from a valid init script, here we need the PID of the process himself (getPID). After, when flume runs, flume could have a lot of channels in a configuration. For all of that channels (single threads) we need the PID of the thread too, to manage the flows (kill, status etc). For that I thought two separate classes are will be fine, the first get the PID of the flume-init process back (as .pid via /var/run/ or /proc/), the second a PID of a running thread for easier management. Its a idea, not more thanks, Alex
        Hide
        Brock Noland added a comment -

        Hi Alexander,

        Are you looking for the pid to be written out in the log or in a .pid file?

        Brock

        Show
        Brock Noland added a comment - Hi Alexander, Are you looking for the pid to be written out in the log or in a .pid file? Brock
        Alexander Alten-Lorenz made changes -
        Attachment getThread.java [ 12516528 ]
        Hide
        Alexander Alten-Lorenz added a comment -

        get the PID from a java process via reflection, since flume starts as a thread safe process here we could printout the PID for managing / monitoring

        Show
        Alexander Alten-Lorenz added a comment - get the PID from a java process via reflection, since flume starts as a thread safe process here we could printout the PID for managing / monitoring
        Alexander Alten-Lorenz made changes -
        Attachment getPID.java [ 12516309 ]
        Alexander Alten-Lorenz made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Alexander Alten-Lorenz added a comment -

        Submitted for review

        Show
        Alexander Alten-Lorenz added a comment - Submitted for review
        Alexander Alten-Lorenz created issue -

          People

          • Assignee:
            Alexander Alten-Lorenz
            Reporter:
            Alexander Alten-Lorenz
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development