Bigtop
  1. Bigtop
  2. BIGTOP-568

Move the configuration directory for hadoop to /etc/hadoop

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      I find the repetition in /etc/hadoop/conf to be confusing for users and I propose that we remove the conf level.

        Activity

        Hide
        Roman Shaposhnik added a comment -

        Owen, the logic behind that extra level of indirection is that we want to support alternatives for multiple configs. The /etc/hadoop/conf is an alternative pointing to one of the actual configuration subdirectories.

        Now, we have this policy throughout Bigtop and it could be argued that for some components (like Pig, Sqoop) you don't really need this alternative complexity, Hadoop however is one of the things that seems to clearly benefit from it in my experience.

        Could you tell us a bit more about your use case?

        Show
        Roman Shaposhnik added a comment - Owen, the logic behind that extra level of indirection is that we want to support alternatives for multiple configs. The /etc/hadoop/conf is an alternative pointing to one of the actual configuration subdirectories. Now, we have this policy throughout Bigtop and it could be argued that for some components (like Pig, Sqoop) you don't really need this alternative complexity, Hadoop however is one of the things that seems to clearly benefit from it in my experience. Could you tell us a bit more about your use case?
        Hide
        Owen O'Malley added a comment -

        The use case is the obvious one, that users need to know where the HADOOP_CONF_DIR is and no one correctly guesses /etc/hadoop/conf. I've seen very few clusters set up with multiple client configurations. If you want to support different server configurations, I'd suggest something like:

        /etc/hadoop-server/{namenode,datanode,jobtracker,tasktracker}/
        

        What do you think?

        Show
        Owen O'Malley added a comment - The use case is the obvious one, that users need to know where the HADOOP_CONF_DIR is and no one correctly guesses /etc/hadoop/conf. I've seen very few clusters set up with multiple client configurations. If you want to support different server configurations, I'd suggest something like: /etc/hadoop-server/{namenode,datanode,jobtracker,tasktracker}/ What do you think?
        Hide
        Roman Shaposhnik added a comment -

        Well, alternatives are mostly useful on the client side when you have to deal with multiple hadoop clusters and have one as a default. I see your point now, but given that this is going to be a pretty big change I suggest that we discuss this on bigtop-dev in more details if you don't mind.

        Show
        Roman Shaposhnik added a comment - Well, alternatives are mostly useful on the client side when you have to deal with multiple hadoop clusters and have one as a default. I see your point now, but given that this is going to be a pretty big change I suggest that we discuss this on bigtop-dev in more details if you don't mind.
        Hide
        Roman Shaposhnik added a comment -

        Seems like we haven't really reached a consensus that this would be a good thing to do. Closing as won't fix.

        Show
        Roman Shaposhnik added a comment - Seems like we haven't really reached a consensus that this would be a good thing to do. Closing as won't fix.

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Owen O'Malley
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development