Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Build and Test Code
    • Labels:
      None

      Description

      I'd like to build a stress-test for Chukwa, to verify that collectors don't lose or corrupt data under high load.

      1. CHUKWA-368-c.patch
        6 kB
        Ari Rabkin
      2. CHUKWA-368.patch
        14 kB
        Ari Rabkin

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        16d 14h 6m 1 Ari Rabkin 21/Aug/09 09:57
        In Progress In Progress Patch Available Patch Available
        2d 21h 1 Ari Rabkin 24/Aug/09 06:57
        Resolved Resolved Reopened Reopened
        5d 15h 44m 1 Ari Rabkin 03/Sep/09 08:57
        Reopened Reopened Patch Available Patch Available
        1m 58s 1 Ari Rabkin 03/Sep/09 08:59
        Patch Available Patch Available Resolved Resolved
        5d 21h 33m 2 Ari Rabkin 04/Sep/09 20:17
        Hide
        Hudson added a comment -

        Integrated in Chukwa-trunk #128 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/128/)
        . Some improvements to display of results

        Show
        Hudson added a comment - Integrated in Chukwa-trunk #128 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/128/ ) . Some improvements to display of results
        Ari Rabkin made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Ari Rabkin added a comment -

        I think this is now the last round of fixes.

        Show
        Ari Rabkin added a comment - I think this is now the last round of fixes.
        Ari Rabkin made changes -
        Status Reopened [ 4 ] Patch Available [ 10002 ]
        Hide
        Ari Rabkin added a comment -

        Tested at scale at Berkeley; will commit tomorrow afternoon barring objection. Code only in use at Berkeley so this shouldn't inconvenience anybody else.

        Show
        Ari Rabkin added a comment - Tested at scale at Berkeley; will commit tomorrow afternoon barring objection. Code only in use at Berkeley so this shouldn't inconvenience anybody else.
        Ari Rabkin made changes -
        Attachment CHUKWA-368-c.patch [ 12418480 ]
        Hide
        Ari Rabkin added a comment -
        • Cleaner separation of IO from state tracking.
        • Track which split data was in.
        • Make replay a bit more deterministic.
        Show
        Ari Rabkin added a comment - Cleaner separation of IO from state tracking. Track which split data was in. Make replay a bit more deterministic.
        Ari Rabkin made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        Ari Rabkin added a comment -

        Can improve on previous code.

        Show
        Ari Rabkin added a comment - Can improve on previous code.
        Hide
        Hudson added a comment -

        Integrated in Chukwa-trunk #121 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/121/)
        . Delete old file
        . New data integrity validation tool

        Show
        Hudson added a comment - Integrated in Chukwa-trunk #121 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/121/ ) . Delete old file . New data integrity validation tool
        Ari Rabkin made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Ari Rabkin added a comment -

        Passes unit tests on my end, so I'm committing this. One non-trivial change. I'm moving src/test/org/apache/hadoop/chukwa/datacollection/TempFileUtil.java out of src/test and into the main src dir. The reason for this change is to make the temp file generation code available to contrib unit tests.

        Show
        Ari Rabkin added a comment - Passes unit tests on my end, so I'm committing this. One non-trivial change. I'm moving src/test/org/apache/hadoop/chukwa/datacollection/TempFileUtil.java out of src/test and into the main src dir. The reason for this change is to make the temp file generation code available to contrib unit tests.
        Hide
        Ari Rabkin added a comment -

        Will commit tomorrow afternoon, I meant to say.

        Show
        Ari Rabkin added a comment - Will commit tomorrow afternoon, I meant to say.
        Hide
        Ari Rabkin added a comment -

        I'm going to commit this, with local mods, barring objection.

        Show
        Ari Rabkin added a comment - I'm going to commit this, with local mods, barring objection.
        Ari Rabkin made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Ari Rabkin made changes -
        Attachment CHUKWA-368.patch [ 12417442 ]
        Ari Rabkin made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Ari Rabkin made changes -
        Field Original Value New Value
        Assignee Ari Rabkin [ asrabkin ]
        Hide
        Ari Rabkin added a comment -

        My proposal is as follows: modify ConstRateSender to emit, instead of random bytes, a byte pattern that depends on the sequence number of the Chunk being produced. (This is a small and painless change).

        Second, a MapReduce job that looks for that pattern and verifies that each chunk shows up at least once, that counts the number of duplicates, and that verifies that Chunks aren't corrupted.

        Really, I'm worried about lost or duplicate chunks; the data pattern thing is just easy to do at the same time.

        Show
        Ari Rabkin added a comment - My proposal is as follows: modify ConstRateSender to emit, instead of random bytes, a byte pattern that depends on the sequence number of the Chunk being produced. (This is a small and painless change). Second, a MapReduce job that looks for that pattern and verifies that each chunk shows up at least once, that counts the number of duplicates, and that verifies that Chunks aren't corrupted. Really, I'm worried about lost or duplicate chunks; the data pattern thing is just easy to do at the same time.
        Ari Rabkin created issue -

          People

          • Assignee:
            Ari Rabkin
            Reporter:
            Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development