Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Build and Test Code
    • Labels:
      None

      Description

      I'd like to build a stress-test for Chukwa, to verify that collectors don't lose or corrupt data under high load.

      1. CHUKWA-368.patch
        14 kB
        Ari Rabkin
      2. CHUKWA-368-c.patch
        6 kB
        Ari Rabkin

        Activity

        Hide
        Hudson added a comment -

        Integrated in Chukwa-trunk #128 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/128/)
        . Some improvements to display of results

        Show
        Hudson added a comment - Integrated in Chukwa-trunk #128 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/128/ ) . Some improvements to display of results
        Hide
        Ari Rabkin added a comment -

        I think this is now the last round of fixes.

        Show
        Ari Rabkin added a comment - I think this is now the last round of fixes.
        Hide
        Ari Rabkin added a comment -

        Tested at scale at Berkeley; will commit tomorrow afternoon barring objection. Code only in use at Berkeley so this shouldn't inconvenience anybody else.

        Show
        Ari Rabkin added a comment - Tested at scale at Berkeley; will commit tomorrow afternoon barring objection. Code only in use at Berkeley so this shouldn't inconvenience anybody else.
        Hide
        Ari Rabkin added a comment -
        • Cleaner separation of IO from state tracking.
        • Track which split data was in.
        • Make replay a bit more deterministic.
        Show
        Ari Rabkin added a comment - Cleaner separation of IO from state tracking. Track which split data was in. Make replay a bit more deterministic.
        Hide
        Ari Rabkin added a comment -

        Can improve on previous code.

        Show
        Ari Rabkin added a comment - Can improve on previous code.
        Hide
        Hudson added a comment -

        Integrated in Chukwa-trunk #121 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/121/)
        . Delete old file
        . New data integrity validation tool

        Show
        Hudson added a comment - Integrated in Chukwa-trunk #121 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/121/ ) . Delete old file . New data integrity validation tool
        Hide
        Ari Rabkin added a comment -

        Passes unit tests on my end, so I'm committing this. One non-trivial change. I'm moving src/test/org/apache/hadoop/chukwa/datacollection/TempFileUtil.java out of src/test and into the main src dir. The reason for this change is to make the temp file generation code available to contrib unit tests.

        Show
        Ari Rabkin added a comment - Passes unit tests on my end, so I'm committing this. One non-trivial change. I'm moving src/test/org/apache/hadoop/chukwa/datacollection/TempFileUtil.java out of src/test and into the main src dir. The reason for this change is to make the temp file generation code available to contrib unit tests.
        Hide
        Ari Rabkin added a comment -

        Will commit tomorrow afternoon, I meant to say.

        Show
        Ari Rabkin added a comment - Will commit tomorrow afternoon, I meant to say.
        Hide
        Ari Rabkin added a comment -

        I'm going to commit this, with local mods, barring objection.

        Show
        Ari Rabkin added a comment - I'm going to commit this, with local mods, barring objection.
        Hide
        Ari Rabkin added a comment -

        My proposal is as follows: modify ConstRateSender to emit, instead of random bytes, a byte pattern that depends on the sequence number of the Chunk being produced. (This is a small and painless change).

        Second, a MapReduce job that looks for that pattern and verifies that each chunk shows up at least once, that counts the number of duplicates, and that verifies that Chunks aren't corrupted.

        Really, I'm worried about lost or duplicate chunks; the data pattern thing is just easy to do at the same time.

        Show
        Ari Rabkin added a comment - My proposal is as follows: modify ConstRateSender to emit, instead of random bytes, a byte pattern that depends on the sequence number of the Chunk being produced. (This is a small and painless change). Second, a MapReduce job that looks for that pattern and verifies that each chunk shows up at least once, that counts the number of duplicates, and that verifies that Chunks aren't corrupted. Really, I'm worried about lost or duplicate chunks; the data pattern thing is just easy to do at the same time.

          People

          • Assignee:
            Ari Rabkin
            Reporter:
            Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development