Chukwa
  1. Chukwa
  2. CHUKWA-589

Functionality allowing collectors to close .chukwa files at a fixed offset inside the rotateInterval

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Component/s: Data Collection
    • Labels:
      None
    • Release Note:
      Allow collectors to close .chukwa files at fixed offset.

      Description

      We have a system built on top of Chukwa that introduces the need to have all .chukwa files for a given rotateInterval closed and available at a predictable time. Currently, we are experience some drift in the times that the sequence files are closed due to the way the TimerTask is scheduled in the SeqFileWriter class. We would like to submit a solution that will allow people to configure the time all collectors should close their files for processing in a given interval, while still supporting the default functionality.

      1. CHUKWA-589_1.patch
        18 kB
        Himanshu Gahlot
      2. CHUKWA-589_2.patch
        19 kB
        Shweta Shah
      3. CHUKWA-589_3.patch
        20 kB
        Shweta Shah

        Activity

        Hide
        Shweta Shah added a comment -

        Himanshu and I are working on this feature and we would be submitting version 1 of the patch. Two new properties have been defined in chukwa-collectors-conf.xml.template as below:

        • To set the rotator scheme (boolean):
           
          <property>
              <name>chukwaCollector.isFixedTimeRotatorScheme</name>
              <value>false</value>
              <description>A flag to indicate that the collector should close at a fixed
              offset after every rotateInterval. If set to true then specify
              chukwaCollector.fixedTimeIntervalOffset value. A false value will use the
              default scheme where collectors close after regular rotateIntervals.
              e.g., if isFixedTimeRotatorScheme is true and fixedTimeIntervalOffset is
              set to 10000 and rotateInterval is set to 300000, then the collector will
              close its files at 10 seconds past the 5 minute mark, if
              isFixedTimeRotatorScheme is false, collectors will rotate approximately
              once every 5 minutes
              </description>
          </property>
          
        • To set the offset interval in milliseconds:
           
          <property>
              <name>chukwaCollector.fixedTimeIntervalOffset</name>
              <value>@TODO-COLLECTORS-FIXEDTIMEINTERVALOFFSET@</value>
              <description>Chukwa fixed time interval offset value (ms)</description>
          </property>
          

        In SeqFileWriter class, the scheduleNextRotation() method calculates the delay in scheduling the next rotate() TimerTask as per the chosen scheme and the interval offset (if isFixedTimeRotatorScheme is true).

        We have included two tests:

        • testSeqWriterFixedCloseInterval() - this test checks the timestamps between two successive sample .chukwa files and compares the time difference with the pre-calculated delay.
        • testFixedIntervalOffsetCalculation() - this test verifies the delay calculation using different combinations of rotateInterval, offsetInterval, and startTime (currentTime as used in the actual method).
        Show
        Shweta Shah added a comment - Himanshu and I are working on this feature and we would be submitting version 1 of the patch. Two new properties have been defined in chukwa-collectors-conf.xml.template as below: To set the rotator scheme (boolean): <property> <name>chukwaCollector.isFixedTimeRotatorScheme</name> <value>false</value> <description>A flag to indicate that the collector should close at a fixed offset after every rotateInterval. If set to true then specify chukwaCollector.fixedTimeIntervalOffset value. A false value will use the default scheme where collectors close after regular rotateIntervals. e.g., if isFixedTimeRotatorScheme is true and fixedTimeIntervalOffset is set to 10000 and rotateInterval is set to 300000, then the collector will close its files at 10 seconds past the 5 minute mark, if isFixedTimeRotatorScheme is false, collectors will rotate approximately once every 5 minutes </description> </property> To set the offset interval in milliseconds: <property> <name>chukwaCollector.fixedTimeIntervalOffset</name> <value>@TODO-COLLECTORS-FIXEDTIMEINTERVALOFFSET@</value> <description>Chukwa fixed time interval offset value (ms)</description> </property> In SeqFileWriter class, the scheduleNextRotation() method calculates the delay in scheduling the next rotate() TimerTask as per the chosen scheme and the interval offset (if isFixedTimeRotatorScheme is true). We have included two tests: testSeqWriterFixedCloseInterval() - this test checks the timestamps between two successive sample .chukwa files and compares the time difference with the pre-calculated delay. testFixedIntervalOffsetCalculation() - this test verifies the delay calculation using different combinations of rotateInterval, offsetInterval, and startTime (currentTime as used in the actual method).
        Hide
        Himanshu Gahlot added a comment -

        Submitting CHUKWA-589_1.patch

        Show
        Himanshu Gahlot added a comment - Submitting CHUKWA-589 _1.patch
        Hide
        Eric Yang added a comment -

        Overall, the patch looks good. One minor enhancement, can you add TODO-COLLECTORS-FIXEDTIMEINTERVALOFFSET to default.properties, and modify build.xml to replace the value for chukwa-config target?

        Show
        Eric Yang added a comment - Overall, the patch looks good. One minor enhancement, can you add TODO-COLLECTORS-FIXEDTIMEINTERVALOFFSET to default.properties, and modify build.xml to replace the value for chukwa-config target?
        Hide
        Shweta Shah added a comment -

        We have made the required changes to the default.properties and build.xml files. Additionally, the default value (false) for property chukwaCollector.isFixedTimeRotatorScheme is also included in default.properties now.

        <property>
            <name>chukwaCollector.isFixedTimeRotatorScheme</name>
            <value>@TODO-COLLECTORS-ISFIXEDTIMEROTATORSCHEME</value>
            <description>A flag to indicate that the collector should close at a fixed
            offset after every rotateInterval. The default value is false which uses
            the default scheme where collectors close after regular rotateIntervals.
            If set to true then specify chukwaCollector.fixedTimeIntervalOffset value.
            e.g., if isFixedTimeRotatorScheme is true and fixedTimeIntervalOffset is
            set to 10000 and rotateInterval is set to 300000, then the collector will
            close its files at 10 seconds past the 5 minute mark, if
            isFixedTimeRotatorScheme is false, collectors will rotate approximately
            once every 5 minutes
            </description>
        </property>
        

        Attached please find the version 2 of the patch - CHUKWA-589_2.patch with all changes.

        Show
        Shweta Shah added a comment - We have made the required changes to the default.properties and build.xml files. Additionally, the default value (false) for property chukwaCollector.isFixedTimeRotatorScheme is also included in default.properties now. <property> <name>chukwaCollector.isFixedTimeRotatorScheme</name> <value>@TODO-COLLECTORS-ISFIXEDTIMEROTATORSCHEME</value> <description>A flag to indicate that the collector should close at a fixed offset after every rotateInterval. The default value is false which uses the default scheme where collectors close after regular rotateIntervals. If set to true then specify chukwaCollector.fixedTimeIntervalOffset value. e.g., if isFixedTimeRotatorScheme is true and fixedTimeIntervalOffset is set to 10000 and rotateInterval is set to 300000, then the collector will close its files at 10 seconds past the 5 minute mark, if isFixedTimeRotatorScheme is false, collectors will rotate approximately once every 5 minutes </description> </property> Attached please find the version 2 of the patch - CHUKWA-589 _2.patch with all changes.
        Hide
        Bill Graham added a comment -

        Looks good to me too although I'd like to request one more small change. In TestChukwaWriters you re-use the "testChukwaWriters_JB_" directory prefix used in the test data. Can you change that to something that implies the test name, so its content can be mapped to the test. Something like "testChukwaWriters_testSeqWriterFixedCloseInterval_".

        I'm not sure how the "JB" convention was introduced in the pre-existing test in that class (maybe Jerome's initials?), but you might a well correct that in the other test as well (i.e. "testChukwaWriters_testWriter_").

        Show
        Bill Graham added a comment - Looks good to me too although I'd like to request one more small change. In TestChukwaWriters you re-use the "testChukwaWriters_JB_" directory prefix used in the test data. Can you change that to something that implies the test name, so its content can be mapped to the test. Something like "testChukwaWriters_testSeqWriterFixedCloseInterval_". I'm not sure how the " JB " convention was introduced in the pre-existing test in that class (maybe Jerome's initials?), but you might a well correct that in the other test as well (i.e. "testChukwaWriters_testWriter_").
        Hide
        Shweta Shah added a comment -

        In TestChukwaWriters, made changes to the outputDirectory paths in testWriters as "/testChukwaWriters_testWriters_" and in testSeqWriterFixedCloseInterval as "/testChukwaWriters_testSeqWriterFixedCloseInterval_"

        Attaching the patch version 3 with these changes.

        Show
        Shweta Shah added a comment - In TestChukwaWriters, made changes to the outputDirectory paths in testWriters as "/testChukwaWriters_testWriters_" and in testSeqWriterFixedCloseInterval as "/testChukwaWriters_testSeqWriterFixedCloseInterval_" Attaching the patch version 3 with these changes.
        Hide
        Bill Graham added a comment -

        +1. Reviewed code, ran unit tests. Only 1 failed (TestHBaseWriter). I'll commit unless anyone else has comments.

        Show
        Bill Graham added a comment - +1. Reviewed code, ran unit tests. Only 1 failed (TestHBaseWriter). I'll commit unless anyone else has comments.
        Hide
        Eric Yang added a comment -

        +1 looks good. I will look at TestHBaseWriter.

        Show
        Eric Yang added a comment - +1 looks good. I will look at TestHBaseWriter.
        Hide
        Bill Graham added a comment -

        Committed, thanks Himanshu and Shweta!

        Show
        Bill Graham added a comment - Committed, thanks Himanshu and Shweta!
        Hide
        Himanshu Gahlot added a comment -

        Thanks Bill !

        Show
        Himanshu Gahlot added a comment - Thanks Bill !

          People

          • Assignee:
            Himanshu Gahlot
            Reporter:
            Himanshu Gahlot
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development