Bigtop
  1. Bigtop
  2. BIGTOP-503

remove unsupported upstream launcher scripts from packaging

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.4.0
    • Component/s: General
    • Labels:
      None

      Description

      Currently the following scripts break if one tries to use them out of packages. This mostly happens because they expect daemons to be managed in a very particular way and are not init.d/upstart friendly. Removing them seems to be
      the best way to sort out those that are truly appreciated by users from those that are "just there". We can selectively fix/resurrect the ones that folks miss too much.

      For now, here's the list. Scream if you see your favorite kitten on there

      hadoop/sbin:
        start-all.sh
        stop-all.sh
      
      hadoop-yarn/sbin:
        slaves.sh
        start-yarn.sh
        stop-yarn.sh
      
      hadoop-hdfs/sbin:
        distribute-exclude.sh
        refresh-namenodes.sh
        start-balancer.sh
        stop-balancer.sh
        start-secure-dns.sh
        stop-secure-dns.sh
        start-dfs.sh
        stop-dfs.sh
      
      hbase/bin:
        rolling-restart.sh
        graceful_stop.sh
        local-regionservers.sh
        start-hbase.sh
        stop-hbase.sh
        local-master-backup.sh
      
      oozie/bin:
        addtowar.sh
        oozie-setup.sh
      
      1. BIGTOP-503.patch.txt
        2 kB
        Roman Shaposhnik

        Activity

        Hide
        Jonathan Hsieh added a comment -

        Is there a reason why we don't also get rid of:

        hbase/bin:
        start-hbase.sh
        stop-hbase.sh
        hbase-daemons.sh
        master-backup.sh
        regionservers.sh
        zookeepers.sh

        These all use ssh to "start" a cluster similar to how start/stop-all.sh, start/stop-dfs.sh and start/stop-mapred.sh scripts work for hadoop, and I believe these are all not to be used from package based installed.

        Show
        Jonathan Hsieh added a comment - Is there a reason why we don't also get rid of: hbase/bin: start-hbase.sh stop-hbase.sh hbase-daemons.sh master-backup.sh regionservers.sh zookeepers.sh These all use ssh to "start" a cluster similar to how start/stop-all.sh, start/stop-dfs.sh and start/stop-mapred.sh scripts work for hadoop, and I believe these are all not to be used from package based installed.
        Hide
        Alejandro Abdelnur added a comment -

        on the oozie side, is there a reason not to get rid of the oozie-start/oozie-stop/oozie-run? as all fallback to oozied

        Show
        Alejandro Abdelnur added a comment - on the oozie side, is there a reason not to get rid of the oozie-start/oozie-stop/oozie-run? as all fallback to oozied
        Hide
        Roman Shaposhnik added a comment -

        @Jon,

        I'm confused start-hbase.sh and stop-hbase.sh are on the list. We ARE planning to remove them. As for the rest of them – they actually seem to function. IOW, they let you run arbitrary shell code on all sort of nodes. Thought that could be useful. However, if you seem to think we should remove them – sure.

        Show
        Roman Shaposhnik added a comment - @Jon, I'm confused start-hbase.sh and stop-hbase.sh are on the list. We ARE planning to remove them. As for the rest of them – they actually seem to function. IOW, they let you run arbitrary shell code on all sort of nodes. Thought that could be useful. However, if you seem to think we should remove them – sure.
        Hide
        Roman Shaposhnik added a comment -

        @Alejandro,

        sure we can remove those. Will users get upset, though? They seem to work fine.

        Show
        Roman Shaposhnik added a comment - @Alejandro, sure we can remove those. Will users get upset, though? They seem to work fine.
        Hide
        Bruno Mahé added a comment -

        +1.

        Along with that patch, we should also probably be more explicit regarding our %file sections. So if a new script is removed/added, we would be made aware of it.

        Show
        Bruno Mahé added a comment - +1. Along with that patch, we should also probably be more explicit regarding our %file sections. So if a new script is removed/added, we would be made aware of it.
        Hide
        Jonathan Hsieh added a comment -

        @Roman – I didn't notice tha the code box has a scrollbar on the in my screen – the others were truncated. The scripts assume a populated regionservers and backup-masters file and I wasn't sure if that is supposed included in packaged versions of HBase

        Show
        Jonathan Hsieh added a comment - @Roman – I didn't notice tha the code box has a scrollbar on the in my screen – the others were truncated. The scripts assume a populated regionservers and backup-masters file and I wasn't sure if that is supposed included in packaged versions of HBase
        Hide
        Matt Foley added a comment -

        Wow, just found this bug as the result of a search for different issue. Quite a big change.

        For the scripts that start and stop a service on a single machine, there is a fairly clear substitute in the init.d start and stop scripts; that's fine. For those cases, please state, for the sake of documentation, the actual init.d/* command that substitutes for each of the above removed scripts.

        However, a number of those scripts deal with cluster startups, orchestration, or specialized activities. It seems to me those don't map well to init.d functionality. It does not seem right to remove those without providing comparable functionality. If you believe comparable functionality already exists, please state the mapping.

        Also, it seems you should get input from experts on the various services before deciding it is okay to whack these scripts that were provided by the developers of those services. Something more pro-active than a jira in the BIGTOP project, opened and committed in a 24-hour timeframe.

        Show
        Matt Foley added a comment - Wow, just found this bug as the result of a search for different issue. Quite a big change. For the scripts that start and stop a service on a single machine, there is a fairly clear substitute in the init.d start and stop scripts; that's fine. For those cases, please state, for the sake of documentation, the actual init.d/* command that substitutes for each of the above removed scripts. However, a number of those scripts deal with cluster startups, orchestration, or specialized activities. It seems to me those don't map well to init.d functionality. It does not seem right to remove those without providing comparable functionality. If you believe comparable functionality already exists, please state the mapping. Also, it seems you should get input from experts on the various services before deciding it is okay to whack these scripts that were provided by the developers of those services. Something more pro-active than a jira in the BIGTOP project, opened and committed in a 24-hour timeframe.
        Hide
        Roman Shaposhnik added a comment -

        Matt, I'm not quite sure who you consider "experts" but on this very JIRA you can see feedback given to us by an original architect of Oozie and a very significant member of the HBase project. They seem to be ok with the decision. As for Hadoop – who would you recommend as an "expert" in this field?

        Show
        Roman Shaposhnik added a comment - Matt, I'm not quite sure who you consider "experts" but on this very JIRA you can see feedback given to us by an original architect of Oozie and a very significant member of the HBase project. They seem to be ok with the decision. As for Hadoop – who would you recommend as an "expert" in this field?
        Hide
        Bruno Mahé added a comment -

        Matt>
        1/ A lot of these scripts use hard coded values or make wrong assumption. Given we cannot patch them, it is better to not ship them than ship broken scripts
        2/ We already have cluster startups and orchestration. See our puppet recipes. So there is indeed feature parity.
        3/ These are just dependencies, we are not redistributing them. So there wouldn't even need for feature parity.

        Show
        Bruno Mahé added a comment - Matt> 1/ A lot of these scripts use hard coded values or make wrong assumption. Given we cannot patch them, it is better to not ship them than ship broken scripts 2/ We already have cluster startups and orchestration. See our puppet recipes. So there is indeed feature parity. 3/ These are just dependencies, we are not redistributing them. So there wouldn't even need for feature parity.

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Roman Shaposhnik
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development