HBase
  1. HBase
  2. HBASE-3991

Add Util folder for Utility Scripts

    Details

    • Type: Brainstorming Brainstorming
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.92.0
    • Fix Version/s: None
    • Component/s: scripts, util
    • Labels:
      None

      Description

      This JIRA is to start discussion around adding some sort of 'util' folder to HBase for common operational scripts. We're starting to write a lot of HBase analysis utilities that we'd love to share with open source, but don't want to clutter the 'bin' folder, which seems like it should be reserved for start/stop tasks. If we add a 'util' folder, how do we keep it from becoming a cesspool of half-baked & duplicated operational hacks.

        Activity

        Hide
        Lars Hofhansl added a comment -

        @Nicolas: Are you still planning to do this?
        Personally I like jRuby, because it provides a simple scripting language and also access to the Java libraries.
        Maybe a subtask is to figure out how to integrate these into the test suite.

        Show
        Lars Hofhansl added a comment - @Nicolas: Are you still planning to do this? Personally I like jRuby, because it provides a simple scripting language and also access to the Java libraries. Maybe a subtask is to figure out how to integrate these into the test suite.
        Hide
        stack added a comment -

        @Nicolas What are the scripts written in? The util would be where? Under the bin or at top-level?

        Up to now I've seen 'bin' as place for 'scripts' that up to this have been non-java but scripts that are not actively in use have the tendency to rot especially if they do not need to be compiled. We have this prob. now w/ bin contents.

        Regards, "...how do we keep it from becoming a cesspool of half-baked & duplicated operational hacks", thats a tough. Tests for scripts would help with rot. Libs that scripts can reuse would help mitigate duplication (the ruby stuff has libs and primitive tests that has not been consistently exploited).

        If we think the cesspool inevitable, yeah, lets move the smell elsewhere as per Todd.

        Show
        stack added a comment - @Nicolas What are the scripts written in? The util would be where? Under the bin or at top-level? Up to now I've seen 'bin' as place for 'scripts' that up to this have been non-java but scripts that are not actively in use have the tendency to rot especially if they do not need to be compiled. We have this prob. now w/ bin contents. Regards, "...how do we keep it from becoming a cesspool of half-baked & duplicated operational hacks", thats a tough. Tests for scripts would help with rot. Libs that scripts can reuse would help mitigate duplication (the ruby stuff has libs and primitive tests that has not been consistently exploited). If we think the cesspool inevitable, yeah, lets move the smell elsewhere as per Todd.
        Hide
        Todd Lipcon added a comment -

        Here are my opinions:

        • writing scripts in ruby is convenient but not very maintainable since only a few of the committers actually know Ruby. they also don't really integrate with the test suite very well.
        • if you write the tools in Java, there should be at least minimal unit tests that make sure they work
        • if in Java, maybe we would want them to all go in from the same command line? eg "hbase tool hfile-stats <...>"?

        If it's truely a bunch of random one-off tools and scripts, something like a github project "hbase toolkit" would make more sense?

        Show
        Todd Lipcon added a comment - Here are my opinions: writing scripts in ruby is convenient but not very maintainable since only a few of the committers actually know Ruby. they also don't really integrate with the test suite very well. if you write the tools in Java, there should be at least minimal unit tests that make sure they work if in Java, maybe we would want them to all go in from the same command line? eg "hbase tool hfile-stats <...>"? If it's truely a bunch of random one-off tools and scripts, something like a github project "hbase toolkit" would make more sense?

          People

          • Assignee:
            Nicolas Spiegelberg
            Reporter:
            Nicolas Spiegelberg
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development