Flume
  1. Flume
  2. FLUME-1006

PID creation of starting flume agents

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: v1.1.0
    • Fix Version/s: None
    • Component/s: Configuration
    • Labels:

      Description

      New class to let new flume agents write their own pid. Thats we need in future to start flume agents from init, for monitoring too.

      1. getPID.java
        1 kB
        Alexander Alten-Lorenz
      2. getThread.java
        1 kB
        Alexander Alten-Lorenz

        Activity

        Hide
        Alexander Alten-Lorenz added a comment -

        Submitted for review

        Show
        Alexander Alten-Lorenz added a comment - Submitted for review
        Hide
        Alexander Alten-Lorenz added a comment -

        get the PID from a java process via reflection, since flume starts as a thread safe process here we could printout the PID for managing / monitoring

        Show
        Alexander Alten-Lorenz added a comment - get the PID from a java process via reflection, since flume starts as a thread safe process here we could printout the PID for managing / monitoring
        Hide
        Brock Noland added a comment -

        Hi Alexander,

        Are you looking for the pid to be written out in the log or in a .pid file?

        Brock

        Show
        Brock Noland added a comment - Hi Alexander, Are you looking for the pid to be written out in the log or in a .pid file? Brock
        Hide
        Alexander Alten-Lorenz added a comment -

        Hi Brock,

        to manage flume later we need a working PID environment, at the moment I found nothing similar. First flume have to start from a valid init script, here we need the PID of the process himself (getPID). After, when flume runs, flume could have a lot of channels in a configuration. For all of that channels (single threads) we need the PID of the thread too, to manage the flows (kill, status etc). For that I thought two separate classes are will be fine, the first get the PID of the flume-init process back (as .pid via /var/run/ or /proc/), the second a PID of a running thread for easier management. Its a idea, not more

        thanks,
        Alex

        Show
        Alexander Alten-Lorenz added a comment - Hi Brock, to manage flume later we need a working PID environment, at the moment I found nothing similar. First flume have to start from a valid init script, here we need the PID of the process himself (getPID). After, when flume runs, flume could have a lot of channels in a configuration. For all of that channels (single threads) we need the PID of the thread too, to manage the flows (kill, status etc). For that I thought two separate classes are will be fine, the first get the PID of the flume-init process back (as .pid via /var/run/ or /proc/), the second a PID of a running thread for easier management. Its a idea, not more thanks, Alex
        Hide
        Bruno Mahé added a comment -

        Not sure if we all refer to the same thing when we talk about init scripts but from an Apache Bigtop (incubating) point of view, here is what we do:
        The init script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/flume-node.init
        The wrapper script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/install_flume.sh#L116

        It feels a little bit hackish right now due to the way flume was spawning itself in 1.0.0, but some patches were sent and the situation should be much better for the coming version of Apache Flume (incubating).
        But in any case, please keep things simple and stupid

        Show
        Bruno Mahé added a comment - Not sure if we all refer to the same thing when we talk about init scripts but from an Apache Bigtop (incubating) point of view, here is what we do: The init script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/flume-node.init The wrapper script: https://github.com/apache/bigtop/blob/trunk/bigtop-packages/src/common/flume/install_flume.sh#L116 It feels a little bit hackish right now due to the way flume was spawning itself in 1.0.0, but some patches were sent and the situation should be much better for the coming version of Apache Flume (incubating). But in any case, please keep things simple and stupid

          People

          • Assignee:
            Alexander Alten-Lorenz
            Reporter:
            Alexander Alten-Lorenz
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development