Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: Data Collection
    • Labels:
      None
    • Tags:
      rotating and tailing file

      Description

      In LWFTAdaptor.java, lastSlurpTime is never updated, triggering an infinite loop if we use the
      adaptor FileTailingAdaptor and subclass adaptors.

      Loop occurs like this:
      1) lastSlurpTime is never changed and thus always 0.
      2) we append some data to the log file and we do a normal slurp
      3) after this slurp, we have: len == fileReadOffset && fixedNameLastModified > lastSlurpTime
      thus we think log file has rotated when it hasn't: fileReadOffset is reset to 0
      4) we get: len > fileReadOffset and so we do a slurp of len bytes.
      5) we are in the same situation as 3) -> infinite loop: the agent never stops sending chunks
      to the collector although log files does not change.

      It seems like this bug comes from CHUKWA-646. The first patch tailfile.patch seemed to do it
      right but in the second patch, the line "lastSlurpTime = System.currentTimeMillis();" just
      disapeared from LWFTAdaptor.java.

      1. chukwa-668.patch
        2 kB
        Sourygna Luangsay
      2. chukwa-668-2.patch
        4 kB
        Sourygna Luangsay

        Issue Links

          Activity

          Hide
          sourygna Sourygna Luangsay added a comment -

          The patch I submit fix the loop problem.

          It also covers 2 other small bugs:

          • In LWFTAdaptor#tailFile(), return value hasMoreData was never updated, causing some problem
            if the length of the data appended is greater than MAX_READ_SIZE_OPT.
          • In FileTailingAdaptor#tailFile(), I think we shouldn't call slurp if len == fileReadOffset and
            fixedNameLastModified <= lastSlurpTime (the file just stopped receiving more logs) so
            I moved the slurp call in the "if (fixedNameLastModified > lastSlurpTime)" condition.
          Show
          sourygna Sourygna Luangsay added a comment - The patch I submit fix the loop problem. It also covers 2 other small bugs: In LWFTAdaptor#tailFile(), return value hasMoreData was never updated, causing some problem if the length of the data appended is greater than MAX_READ_SIZE_OPT. In FileTailingAdaptor#tailFile(), I think we shouldn't call slurp if len == fileReadOffset and fixedNameLastModified <= lastSlurpTime (the file just stopped receiving more logs) so I moved the slurp call in the "if (fixedNameLastModified > lastSlurpTime)" condition.
          Hide
          eyang Eric Yang added a comment -

          Hi Sourygna, could you write a test case to demo the problem? This will prevent future regressions. Thanks

          Show
          eyang Eric Yang added a comment - Hi Sourygna, could you write a test case to demo the problem? This will prevent future regressions. Thanks
          Hide
          sourygna Sourygna Luangsay added a comment -

          Added junit test in TestFileTailingAdaptors.java.

          Show
          sourygna Sourygna Luangsay added a comment - Added junit test in TestFileTailingAdaptors.java.
          Hide
          eyang Eric Yang added a comment -

          +1 looks good.

          Show
          eyang Eric Yang added a comment - +1 looks good.
          Hide
          hudson Hudson added a comment -

          Integrated in Chukwa-trunk #460 (See https://builds.apache.org/job/Chukwa-trunk/460/)
          CHUKWA-668. Fix infinite loop in FileTailingAdaptor. (Sourygna Luangsay via Eric Yang) (Revision 1411818)

          Result = SUCCESS
          eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411818
          Files :

          • /incubator/chukwa/trunk/CHANGES.txt
          • /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/FileTailingAdaptor.java
          • /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/LWFTAdaptor.java
          • /incubator/chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/TestFileTailingAdaptors.java
          Show
          hudson Hudson added a comment - Integrated in Chukwa-trunk #460 (See https://builds.apache.org/job/Chukwa-trunk/460/ ) CHUKWA-668 . Fix infinite loop in FileTailingAdaptor. (Sourygna Luangsay via Eric Yang) (Revision 1411818) Result = SUCCESS eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1411818 Files : /incubator/chukwa/trunk/CHANGES.txt /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/FileTailingAdaptor.java /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/LWFTAdaptor.java /incubator/chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/adaptor/filetailer/TestFileTailingAdaptors.java
          Hide
          eyang Eric Yang added a comment -

          I committed this a while back, thanks Sourygna.

          Show
          eyang Eric Yang added a comment - I committed this a while back, thanks Sourygna.

            People

            • Assignee:
              sourygna Sourygna Luangsay
              Reporter:
              sourygna Sourygna Luangsay
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development