HBase
  1. HBase
  2. HBASE-4867

A tool to merge configuration files

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      With our cluster configuration setup it would be good to have a tool that would merge HBase configuration, so that files appearing later in the list would override properties specified in earlier files. This way we could merge application-specific configuration file with the cluster-specific configuration file (with the latter overriding the former) and produce a single HBase configuration file to install on the cluster.

        Activity

        Hide
        Phabricator added a comment -

        mbautin requested code review of "[jira] HBASE-4867 A tool to merge configuration files".
        Reviewers: todd, Karthik, tedyu, stack, JIRA

        With our cluster configuration setup it would be good to have a tool that would merge HBase configuration files so that files appearing later in the list would override properties specified in earlier files. This way we could merge an application-specific configuration file with a cluster-specific configuration file (with the latter overriding the former) and produce a single HBase configuration file to install on the cluster.

        TEST PLAN
        Run the tool on two configuration files (common and cluster-specific). Use the resulting configuration on a dev cluster.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        AFFECTED FILES
        src/main/python/hbase/merge_conf.py

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/1191/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - mbautin requested code review of " [jira] HBASE-4867 A tool to merge configuration files". Reviewers: todd, Karthik, tedyu, stack, JIRA With our cluster configuration setup it would be good to have a tool that would merge HBase configuration files so that files appearing later in the list would override properties specified in earlier files. This way we could merge an application-specific configuration file with a cluster-specific configuration file (with the latter overriding the former) and produce a single HBase configuration file to install on the cluster. TEST PLAN Run the tool on two configuration files (common and cluster-specific). Use the resulting configuration on a dev cluster. REVISION DETAIL https://reviews.facebook.net/D537 AFFECTED FILES src/main/python/hbase/merge_conf.py MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1191/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Phabricator added a comment -

        Karthik has accepted the revision "[jira] HBASE-4867 A tool to merge configuration files".

        Looks good!

        Are you planning on using this only for the opensource template on v2 install scripts or all of them?

        INLINE COMMENTS
        src/main/python/hbase/merge_conf.py:120 Nice!

        REVISION DETAIL
        https://reviews.facebook.net/D537

        Show
        Phabricator added a comment - Karthik has accepted the revision " [jira] HBASE-4867 A tool to merge configuration files". Looks good! Are you planning on using this only for the opensource template on v2 install scripts or all of them? INLINE COMMENTS src/main/python/hbase/merge_conf.py:120 Nice! REVISION DETAIL https://reviews.facebook.net/D537
        Hide
        Phabricator added a comment -

        mbautin has commented on the revision "[jira] HBASE-4867 A tool to merge configuration files".

        This is only necessary for the open-source HBase trunk (0.89-fb can read hbase-site-custom.xml). However, nothing prevents us from using this approach for all of our dev clusters.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        Show
        Phabricator added a comment - mbautin has commented on the revision " [jira] HBASE-4867 A tool to merge configuration files". This is only necessary for the open-source HBase trunk (0.89-fb can read hbase-site-custom.xml). However, nothing prevents us from using this approach for all of our dev clusters. REVISION DETAIL https://reviews.facebook.net/D537
        Hide
        Phabricator added a comment -

        todd has commented on the revision "[jira] HBASE-4867 A tool to merge configuration files".

        It seems it would be preferable to write this in Java rather than Python, since nothing else in HBase is written in python. But, failing a complete rewrite, I'd recommend you drop use of the with statement so it works in py2.4/RHEL5. Should be a trivial change to try..finally

        INLINE COMMENTS
        src/main/python/hbase/merge_conf.py:8 This isn't compatible with Python 2.4, which is the latest version on RHEL5. Most of our customers still run RHEL5, rather than RHEL6, so this script wouldn't be compatible.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        Show
        Phabricator added a comment - todd has commented on the revision " [jira] HBASE-4867 A tool to merge configuration files". It seems it would be preferable to write this in Java rather than Python, since nothing else in HBase is written in python. But, failing a complete rewrite, I'd recommend you drop use of the with statement so it works in py2.4/RHEL5. Should be a trivial change to try..finally INLINE COMMENTS src/main/python/hbase/merge_conf.py:8 This isn't compatible with Python 2.4, which is the latest version on RHEL5. Most of our customers still run RHEL5, rather than RHEL6, so this script wouldn't be compatible. REVISION DETAIL https://reviews.facebook.net/D537
        Hide
        Phabricator added a comment -

        mbautin updated the revision "[jira] HBASE-4867 A tool to merge configuration files".
        Reviewers: todd, Karthik, tedyu, stack, JIRA

        Not breaking long words (e.g. URLs) that might appear in the description.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        AFFECTED FILES
        src/main/python/hbase/merge_conf.py

        Show
        Phabricator added a comment - mbautin updated the revision " [jira] HBASE-4867 A tool to merge configuration files". Reviewers: todd, Karthik, tedyu, stack, JIRA Not breaking long words (e.g. URLs) that might appear in the description. REVISION DETAIL https://reviews.facebook.net/D537 AFFECTED FILES src/main/python/hbase/merge_conf.py
        Hide
        Phabricator added a comment -

        mbautin updated the revision "[jira] HBASE-4867 A tool to merge configuration files".
        Reviewers: todd, Karthik, tedyu, stack, JIRA

        Addressing Todd's comments. Tested with python 2.4.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        AFFECTED FILES
        src/main/python/hbase/merge_conf.py

        Show
        Phabricator added a comment - mbautin updated the revision " [jira] HBASE-4867 A tool to merge configuration files". Reviewers: todd, Karthik, tedyu, stack, JIRA Addressing Todd's comments. Tested with python 2.4. REVISION DETAIL https://reviews.facebook.net/D537 AFFECTED FILES src/main/python/hbase/merge_conf.py
        Hide
        Phabricator added a comment -

        todd has commented on the revision "[jira] HBASE-4867 A tool to merge configuration files".

        +1. (can't find the button to "accept" the diff)

        REVISION DETAIL
        https://reviews.facebook.net/D537

        Show
        Phabricator added a comment - todd has commented on the revision " [jira] HBASE-4867 A tool to merge configuration files". +1. (can't find the button to "accept" the diff) REVISION DETAIL https://reviews.facebook.net/D537
        Hide
        Phabricator added a comment -

        mbautin has commented on the revision "[jira] HBASE-4867 A tool to merge configuration files".

        @todd: the accept action should be in the Action dropdown above the comment box. It is OK to leave the comment box empty for that action and submit the form.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        Show
        Phabricator added a comment - mbautin has commented on the revision " [jira] HBASE-4867 A tool to merge configuration files". @todd: the accept action should be in the Action dropdown above the comment box. It is OK to leave the comment box empty for that action and submit the form. REVISION DETAIL https://reviews.facebook.net/D537
        Hide
        Phabricator added a comment -

        Karthik has commented on the revision "[jira] HBASE-4867 A tool to merge configuration files".

        I had already accepted it, so its not letting you accept Todd. Will commit.

        REVISION DETAIL
        https://reviews.facebook.net/D537

        Show
        Phabricator added a comment - Karthik has commented on the revision " [jira] HBASE-4867 A tool to merge configuration files". I had already accepted it, so its not letting you accept Todd. Will commit. REVISION DETAIL https://reviews.facebook.net/D537
        Hide
        Phabricator added a comment -

        mbautin has committed the revision "[jira] HBASE-4867 A tool to merge configuration files".

        REVISION DETAIL
        https://reviews.facebook.net/D537

        COMMIT
        https://reviews.facebook.net/rHBASE1208832

        Show
        Phabricator added a comment - mbautin has committed the revision " [jira] HBASE-4867 A tool to merge configuration files". REVISION DETAIL https://reviews.facebook.net/D537 COMMIT https://reviews.facebook.net/rHBASE1208832
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2503 (See https://builds.apache.org/job/HBase-TRUNK/2503/)
        [jira] HBASE-4867 A tool to merge configuration files

        Summary: With our cluster configuration setup it would be good to have a tool
        that would merge HBase configuration files so that files appearing later in the
        list would override properties specified in earlier files. This way we could
        merge an application-specific configuration file with a cluster-specific
        configuration file (with the latter overriding the former) and produce a single
        HBase configuration file to install on the cluster.

        Test Plan: Run the tool on two configuration files (common and
        cluster-specific). Use the resulting configuration on a dev cluster.

        Reviewers: todd, Karthik, tedyu, stack, JIRA

        Reviewed By: Karthik

        CC: Karthik, mbautin, todd

        Differential Revision: 537

        karthik :
        Files :

        • /hbase/trunk/src/main/python
        • /hbase/trunk/src/main/python/hbase
        • /hbase/trunk/src/main/python/hbase/merge_conf.py
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2503 (See https://builds.apache.org/job/HBase-TRUNK/2503/ ) [jira] HBASE-4867 A tool to merge configuration files Summary: With our cluster configuration setup it would be good to have a tool that would merge HBase configuration files so that files appearing later in the list would override properties specified in earlier files. This way we could merge an application-specific configuration file with a cluster-specific configuration file (with the latter overriding the former) and produce a single HBase configuration file to install on the cluster. Test Plan: Run the tool on two configuration files (common and cluster-specific). Use the resulting configuration on a dev cluster. Reviewers: todd, Karthik, tedyu, stack, JIRA Reviewed By: Karthik CC: Karthik, mbautin, todd Differential Revision: 537 karthik : Files : /hbase/trunk/src/main/python /hbase/trunk/src/main/python/hbase /hbase/trunk/src/main/python/hbase/merge_conf.py
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #17 (See https://builds.apache.org/job/HBase-TRUNK-security/17/)
        [jira] HBASE-4867 A tool to merge configuration files

        Summary: With our cluster configuration setup it would be good to have a tool
        that would merge HBase configuration files so that files appearing later in the
        list would override properties specified in earlier files. This way we could
        merge an application-specific configuration file with a cluster-specific
        configuration file (with the latter overriding the former) and produce a single
        HBase configuration file to install on the cluster.

        Test Plan: Run the tool on two configuration files (common and
        cluster-specific). Use the resulting configuration on a dev cluster.

        Reviewers: todd, Karthik, tedyu, stack, JIRA

        Reviewed By: Karthik

        CC: Karthik, mbautin, todd

        Differential Revision: 537

        karthik :
        Files :

        • /hbase/trunk/src/main/python
        • /hbase/trunk/src/main/python/hbase
        • /hbase/trunk/src/main/python/hbase/merge_conf.py
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #17 (See https://builds.apache.org/job/HBase-TRUNK-security/17/ ) [jira] HBASE-4867 A tool to merge configuration files Summary: With our cluster configuration setup it would be good to have a tool that would merge HBase configuration files so that files appearing later in the list would override properties specified in earlier files. This way we could merge an application-specific configuration file with a cluster-specific configuration file (with the latter overriding the former) and produce a single HBase configuration file to install on the cluster. Test Plan: Run the tool on two configuration files (common and cluster-specific). Use the resulting configuration on a dev cluster. Reviewers: todd, Karthik, tedyu, stack, JIRA Reviewed By: Karthik CC: Karthik, mbautin, todd Differential Revision: 537 karthik : Files : /hbase/trunk/src/main/python /hbase/trunk/src/main/python/hbase /hbase/trunk/src/main/python/hbase/merge_conf.py

          People

          • Assignee:
            Mikhail Bautin
            Reporter:
            Mikhail Bautin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development