HBase
  1. HBase
  2. HBASE-8438

Extend bin/hbase to print a "mapreduce classpath"

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.6.1, 0.95.0, 0.94.13
    • Fix Version/s: 0.98.0, 0.96.1, 0.94.14
    • Component/s: scripts
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      $ ./bin/hbase mapredcp --help
      Usage: hbase mapredcp [-Dtmpjars=...]
        Construct a CLASSPATH containing dependency jars required to run a mapreduce job. By default, includes any jars detected by TableMapReduceUtils. Provide additional entries by specifying a comma-separated list in tmpjars.
      Show
      $ ./bin/hbase mapredcp --help Usage: hbase mapredcp [-Dtmpjars=...]   Construct a CLASSPATH containing dependency jars required to run a mapreduce job. By default, includes any jars detected by TableMapReduceUtils. Provide additional entries by specifying a comma-separated list in tmpjars.

      Description

      For tools like pig and hive, blindly appending the full output of `bin/hbase classpath` to their own CLASSPATH is excessive. They already build CLASSPATH entries for hadoop. All they need from us is the delta entries, the dependencies we require w/o hadoop and all of it's transitive deps. This is also a kindness for Windows, where there's a shorter limit on the length of commandline arguments.

      See also HIVE-2055 for additional discussion.

        Issue Links

          Activity

            People

            • Assignee:
              Nick Dimiduk
              Reporter:
              Nick Dimiduk
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development