Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1066

multifilewc and possibly other mapreduce smoke tests rely on mapred: should they be separated or replaced?

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.7.0
    • Fix Version/s: backlog
    • Component/s: tests
    • Labels:
      None

      Description

      I've updated this ticket to be more focused on a solution rather than brainstorming the problem ...

      Problem: The use of old tests (i.e. those that use mapred) can cause issues. Bigtop gets its hadoop mapreduce tests from the underlying mapreduce library, which in some ways is good - it is expected to match the distro - but other ways , its bad, because if takes control away from the person running the bigtop smokes.

      Solution: Should bigtop have a specific version of mapreduce smokes which it runs by linking to the hadoop-examples as maven deps instead of by referencing the hadoop-examples jars in the distribution?

      Any thoughts? A patch that implemented this would simply add a pom file dependency snippet referencing hadoop-examples, or else, pull the jars down some other way, and run them directly rather then referencing HADOOP_HOME/examples*jar

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jayunit100 jay vyas
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: