Chukwa
  1. Chukwa
  2. CHUKWA-25

chukwa archiving should group by cluster name

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.3.0
    • Component/s: Data Processors
    • Labels:
      None
    • Release Note:
      Can now have archiver group by cluster name

      Description

      The chukwa archivers ought to have an option for grouping data by cluster name.

        Issue Links

          Activity

          Hide
          Ari Rabkin added a comment -

          I feel bad creating yet another OutputFormat, but it's not obvious what the right way to configure this is, since ArchiveBuilder doesn't read in the chukwa configuration files. Thoughts and suggestions?

          Show
          Ari Rabkin added a comment - I feel bad creating yet another OutputFormat, but it's not obvious what the right way to configure this is, since ArchiveBuilder doesn't read in the chukwa configuration files. Thoughts and suggestions?
          Hide
          Ari Rabkin added a comment -

          Add conf option to allow adding cluster name to archiver output. Comes with test case for archiver.

          Show
          Ari Rabkin added a comment - Add conf option to allow adding cluster name to archiver output. Comes with test case for archiver.
          Hide
          Ari Rabkin added a comment -

          Should probably pick better name for conf option, and document it, before committing.

          Show
          Ari Rabkin added a comment - Should probably pick better name for conf option, and document it, before committing.
          Hide
          Eric Yang added a comment -

          Is there a test case being planned at:

          +// fstat = fileSys.getFileStatus(OUTPUT_DIR);
          + // assertTrue(fstat.);

          If not, could we remove it.

          + static final Path OUTPUT_DIR = new Path("/chukwa/archives/");
          + public void testArchiving() throws Exception {
          +
          +
          + System.out.println("starting archive test");

          Probably line break this correctly if possible.

          The patch looks good, otherwise.

          Show
          Eric Yang added a comment - Is there a test case being planned at: +// fstat = fileSys.getFileStatus(OUTPUT_DIR); + // assertTrue(fstat.); If not, could we remove it. + static final Path OUTPUT_DIR = new Path("/chukwa/archives/"); + public void testArchiving() throws Exception { + + + System.out.println("starting archive test"); Probably line break this correctly if possible. The patch looks good, otherwise.
          Hide
          Ari Rabkin added a comment -

          revised as per Eric's comment. Conf option documented in conf/demux

          Show
          Ari Rabkin added a comment - revised as per Eric's comment. Conf option documented in conf/demux
          Hide
          Ari Rabkin added a comment -

          Want to make one more change.

          Show
          Ari Rabkin added a comment - Want to make one more change.
          Hide
          Ari Rabkin added a comment -

          Revised to allow user control of ArchiveManager grouper.

          Show
          Ari Rabkin added a comment - Revised to allow user control of ArchiveManager grouper.
          Hide
          Ari Rabkin added a comment -

          I just committed this, as per Eric's oral +1.

          Show
          Ari Rabkin added a comment - I just committed this, as per Eric's oral +1.

            People

            • Assignee:
              Ari Rabkin
              Reporter:
              Ari Rabkin
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development