Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12857

Rework hadoop-tools

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha1
    • 3.0.0-alpha1
    • build
    • None
    • Incompatible change
    • Hide
      <!-- markdown -->
      * Turning on optional things from the tools directory such as S3 support can now be done in hadoop-env.sh with the HADOOP\_OPTIONAL\_TOOLS environment variable without impacting the various user-facing CLASSPATH variables.
      * The tools directory is no longer pulled in blindly for any utilities that pull it in.
      * TOOL\_PATH / HADOOP\_TOOLS\_PATH has been broken apart and replaced with HADOOP\_TOOLS\_HOME, HADOOP\_TOOLS\_DIR and HADOOP\_TOOLS\_LIB\_JARS\_DIR to be consistent with the rest of Hadoop.
      Show
      <!-- markdown --> * Turning on optional things from the tools directory such as S3 support can now be done in hadoop-env.sh with the HADOOP\_OPTIONAL\_TOOLS environment variable without impacting the various user-facing CLASSPATH variables. * The tools directory is no longer pulled in blindly for any utilities that pull it in. * TOOL\_PATH / HADOOP\_TOOLS\_PATH has been broken apart and replaced with HADOOP\_TOOLS\_HOME, HADOOP\_TOOLS\_DIR and HADOOP\_TOOLS\_LIB\_JARS\_DIR to be consistent with the rest of Hadoop.

    Description

      As hadoop-tools grows bigger and bigger, it's becoming evident that having a single directory that gets sucked in is starting to become a big burden as the number of tools grows. Let's rework this to be smarter.

      Attachments

        1. HADOOP-12857.00.patch
          41 kB
          Allen Wittenauer
        2. HADOOP-12857.01.patch
          42 kB
          Allen Wittenauer
        3. HADOOP-12857.02.patch
          48 kB
          Allen Wittenauer

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-12930 [Umbrella] Dynamic subcommands for hadoop shell scripts

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-17042 Hadoop distcp throws "ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found"

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-13110 add a streaming subcommand to mapred

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-12864 Remove bin/rcc script

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-12865 hadoop-datajoin should be documented or dropped

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. BIGTOP-3495 Optional tools of hadoop-tools should be optional

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12866 add a subcommand for gridmix

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12867 clean up how rumen and sls are executed

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12868 Fix hadoop-openstack undeclared and unused dependencies

          • Major - Major loss of function.
          • Resolved
          supercedes

          Wish - General wishlist item. HADOOP-12721 Hadoop-tools jars should be included in the classpath of hadoop command

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              aw Allen Wittenauer
              aw Allen Wittenauer
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: