Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12857

Rework hadoop-tools

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha1
    • 3.0.0-alpha1
    • build
    • None
    • Incompatible change
    • Hide
      <!-- markdown -->
      * Turning on optional things from the tools directory such as S3 support can now be done in hadoop-env.sh with the HADOOP\_OPTIONAL\_TOOLS environment variable without impacting the various user-facing CLASSPATH variables.
      * The tools directory is no longer pulled in blindly for any utilities that pull it in.
      * TOOL\_PATH / HADOOP\_TOOLS\_PATH has been broken apart and replaced with HADOOP\_TOOLS\_HOME, HADOOP\_TOOLS\_DIR and HADOOP\_TOOLS\_LIB\_JARS\_DIR to be consistent with the rest of Hadoop.
      Show
      <!-- markdown --> * Turning on optional things from the tools directory such as S3 support can now be done in hadoop-env.sh with the HADOOP\_OPTIONAL\_TOOLS environment variable without impacting the various user-facing CLASSPATH variables. * The tools directory is no longer pulled in blindly for any utilities that pull it in. * TOOL\_PATH / HADOOP\_TOOLS\_PATH has been broken apart and replaced with HADOOP\_TOOLS\_HOME, HADOOP\_TOOLS\_DIR and HADOOP\_TOOLS\_LIB\_JARS\_DIR to be consistent with the rest of Hadoop.

    Description

      As hadoop-tools grows bigger and bigger, it's becoming evident that having a single directory that gets sucked in is starting to become a big burden as the number of tools grows. Let's rework this to be smarter.

      Attachments

        1. HADOOP-12857.00.patch
          41 kB
          Allen Wittenauer
        2. HADOOP-12857.01.patch
          42 kB
          Allen Wittenauer
        3. HADOOP-12857.02.patch
          48 kB
          Allen Wittenauer

        Issue Links

        blocks

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-12930 [Umbrella] Dynamic subcommands for hadoop shell scripts

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved
        is related to

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-17042 Hadoop distcp throws "ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found"

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        relates to

        Sub-task - The sub-task of the issue HADOOP-13110 add a streaming subcommand to mapred

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-12864 Remove bin/rcc script

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-12865 hadoop-datajoin should be documented or dropped

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. BIGTOP-3495 Optional tools of hadoop-tools should be optional

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12866 add a subcommand for gridmix

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12867 clean up how rumen and sls are executed

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12868 Fix hadoop-openstack undeclared and unused dependencies

        • Major - Major loss of function.
        • Resolved
        supercedes

        Wish - General wishlist item. HADOOP-12721 Hadoop-tools jars should be included in the classpath of hadoop command

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            aw Allen Wittenauer
            aw Allen Wittenauer
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment