Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1840

[Gridmix] Exploit/Add security features in GridMix

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: contrib/gridmix
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Use security information while replaying jobs in Gridmix. This includes

      • Support for multiple users
      • Submitting jobs as different users
      • Allowing usage of secure cluster (hdfs + mapreduce)
      • Support for multiple queues

      Other features include :

      • Support for sleep job
      • Support for load job

      + testcases for verifying all of the above changes

        Issue Links

          Activity

          Amar Kamat created issue -
          Hide
          Amar Kamat added a comment -

          Attaching a patch that forwards ports the features from their respective jira's. test-patch and ant tests passed.

          Show
          Amar Kamat added a comment - Attaching a patch that forwards ports the features from their respective jira's. test-patch and ant tests passed.
          Amar Kamat made changes -
          Field Original Value New Value
          Attachment mapreduce-gridmix-fp-v1.3.3.patch [ 12446229 ]
          Amar Kamat made changes -
          Assignee Amar Kamat [ amar_kamat ]
          Hide
          Amar Kamat added a comment -

          Blocked on MAPREDUCE-1844 for testing.

          Show
          Amar Kamat added a comment - Blocked on MAPREDUCE-1844 for testing.
          Hide
          Amar Kamat added a comment -

          Attaching a new patch for review. Changes are as follows :

          1. Reused the existing CombineFileSplit
          2. Replaced string configuration keys with constants
          3. Corrected the indentation
          4. Removed deprecated api usage

          test-patch and ant-tests passed on my box.

          Show
          Amar Kamat added a comment - Attaching a new patch for review. Changes are as follows : Reused the existing CombineFileSplit Replaced string configuration keys with constants Corrected the indentation Removed deprecated api usage test-patch and ant-tests passed on my box.
          Amar Kamat made changes -
          Attachment mapreduce-gridmix-fp-v1.3.9.patch [ 12446657 ]
          Hide
          rahul k singh added a comment -

          I looked at the diff between 20.1xx and trunk gridmix directories.
          Changes look fine to me

          +1

          Show
          rahul k singh added a comment - I looked at the diff between 20.1xx and trunk gridmix directories. Changes look fine to me +1
          Hide
          Chris Douglas added a comment -

          +1 on these changes.

          Could this link to the other, open JIRAs incorporated in this patch?

          Show
          Chris Douglas added a comment - +1 on these changes. Could this link to the other, open JIRAs incorporated in this patch?
          Amar Kamat made changes -
          Link This issue incorporates MAPREDUCE-1408 [ MAPREDUCE-1408 ]
          Amar Kamat made changes -
          Link This issue incorporates MAPREDUCE-1594 [ MAPREDUCE-1594 ]
          Amar Kamat made changes -
          Link This issue incorporates MAPREDUCE-1376 [ MAPREDUCE-1376 ]
          Amar Kamat made changes -
          Link This issue incorporates MAPREDUCE-1711 [ MAPREDUCE-1711 ]
          Amar Kamat made changes -
          Link This issue incorporates MAPREDUCE-1526 [ MAPREDUCE-1526 ]
          Hide
          Amar Kamat added a comment -

          Tested the latest patch on a small cluster and it works fine. test-patch and ant-test passed.

          Show
          Amar Kamat added a comment - Tested the latest patch on a small cluster and it works fine. test-patch and ant-test passed.
          Chris Douglas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Chris Douglas added a comment -

          I committed this.

          Thanks to Amar, Rahul, and Hong

          Show
          Chris Douglas added a comment - I committed this. Thanks to Amar, Rahul, and Hong
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development