Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.2, 0.2.0
    • Component/s: Data Processors
    • Labels:
      None

      Description

      If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
      Symptom:
      a log file full of statements like:
      2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

        Activity

        Hide
        jboulon Jerome Boulon added a comment -

        Ari,
        We cannot do that since the code checks for daysInRawArchiveDir[0] and also we don't want to sleep for 30 minutes
        We need to add a specific block right after FileStatus[] daysInRawArchiveDir = fs.listStatus(pDailyRawArchivesInput);

        if (daysInRawArchiveDir.length == 0 )

        { log.info( pDailyRawArchivesInput + " is empty, going to sleep for 1 minute"); Thread.sleep(1 * 60 * 1000); continue; }
        Show
        jboulon Jerome Boulon added a comment - Ari, We cannot do that since the code checks for daysInRawArchiveDir [0] and also we don't want to sleep for 30 minutes We need to add a specific block right after FileStatus[] daysInRawArchiveDir = fs.listStatus(pDailyRawArchivesInput); if (daysInRawArchiveDir.length == 0 ) { log.info( pDailyRawArchivesInput + " is empty, going to sleep for 1 minute"); Thread.sleep(1 * 60 * 1000); continue; }
        Hide
        asrabkin Ari Rabkin added a comment -

        Revised as per Jerome's comment; except that the info() log has been changed to debug().

        Show
        asrabkin Ari Rabkin added a comment - Revised as per Jerome's comment; except that the info() log has been changed to debug().
        Hide
        jboulon Jerome Boulon added a comment -

        +1 looks good

        Show
        jboulon Jerome Boulon added a comment - +1 looks good
        Hide
        asrabkin Ari Rabkin added a comment -

        I just committed this.

        Show
        asrabkin Ari Rabkin added a comment - I just committed this.
        Hide
        hudson Hudson added a comment -

        Integrated in Chukwa-trunk #28 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/28/)
        . Resolve race condition in archiving.

        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #28 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/28/ ) . Resolve race condition in archiving.
        Hide
        hudson Hudson added a comment -
        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #45 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/ )

          People

          • Assignee:
            asrabkin Ari Rabkin
            Reporter:
            asrabkin Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development