Bigtop
  1. Bigtop
  2. BIGTOP-838

Both conf.empty and conf.pseudo packages don't seem to include a sample hadoop-env.sh file

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.6.0
    • Component/s: general
    • Labels:

      Activity

      Show
      Pradeep Bhadani added a comment - - edited Linking Original Discussion https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!topic/cdh-user/1Uqyt-5k56M
      Hide
      Roman Shaposhnik added a comment -

      Is there any reason to actually include it? We're trying to de-emphasize the usage of hadoop-env.sh (and all the other -env.sh scripts) in favor of more reliable system integration (/etc/default/

      {component}

      ) files.

      I mean those who still need it can totally just add the file themselves. IOW, even if we add it – it'll be an empty file (or with all the content commented out).

      Show
      Roman Shaposhnik added a comment - Is there any reason to actually include it? We're trying to de-emphasize the usage of hadoop-env.sh (and all the other -env.sh scripts) in favor of more reliable system integration (/etc/default/ {component} ) files. I mean those who still need it can totally just add the file themselves. IOW, even if we add it – it'll be an empty file (or with all the content commented out).
      Hide
      Giridharan Kesavan added a comment -

      Having hadoop-env.sh would really help. Currently, log dir is set to /var/log/hadoop. Most of them would want their hadoop logs to be on a different disk other than root partition as this would fill the disk pretty quick. With hadoop-env.sh, one can set the log dir of their choice.

      Show
      Giridharan Kesavan added a comment - Having hadoop-env.sh would really help. Currently, log dir is set to /var/log/hadoop. Most of them would want their hadoop logs to be on a different disk other than root partition as this would fill the disk pretty quick. With hadoop-env.sh, one can set the log dir of their choice.
      Hide
      Roman Shaposhnik added a comment -

      Right, but that's what I'm saying – we prefer that users utilize well know locations for configuration files. In this particular case you can use /etc/default/hadoop-<daemon name> to tweak location of the log files as easily as you can with hadoop-env.sh.

      The trouble with hadoop-env.sh is that it is extremely hadoop specific and foreign to folks who don't have any Hadoop background.

      Show
      Roman Shaposhnik added a comment - Right, but that's what I'm saying – we prefer that users utilize well know locations for configuration files. In this particular case you can use /etc/default/hadoop-<daemon name> to tweak location of the log files as easily as you can with hadoop-env.sh. The trouble with hadoop-env.sh is that it is extremely hadoop specific and foreign to folks who don't have any Hadoop background.
      Hide
      Harsh J added a comment -

      Good points Roman; I hadn't considered the /etc/default/hadoop* when filing this.

      Indeed that acts in ways similar to hadoop-env.sh (does it also allow heap size control, etc., which is primarily what people modify it for?), but its not very discoverable.

      Having an empty file is worth it (and is what I filed for as well, sorry I did not elaborate early) - 'hadoop-env.sh' is written about in several guides online as a place to set stuff like specific daemon heap size, etc.; which a new Bigtop package user may easily miss?

      Show
      Harsh J added a comment - Good points Roman; I hadn't considered the /etc/default/hadoop* when filing this. Indeed that acts in ways similar to hadoop-env.sh (does it also allow heap size control, etc., which is primarily what people modify it for?), but its not very discoverable. Having an empty file is worth it (and is what I filed for as well, sorry I did not elaborate early) - 'hadoop-env.sh' is written about in several guides online as a place to set stuff like specific daemon heap size, etc.; which a new Bigtop package user may easily miss?
      Hide
      Pradeep Bhadani added a comment -

      if hadoop-env.sh file is not requires then why it is included in CDH4.1 Virtual image?
      It is not included in CDH4.1 (manual installation).

      If i have to provide some parameters like JAVA_HOME, then where i will provide?
      I always get JAVA_HOME not set during hadoop installation, if i dont have root permission.

      Show
      Pradeep Bhadani added a comment - if hadoop-env.sh file is not requires then why it is included in CDH4.1 Virtual image? It is not included in CDH4.1 (manual installation). If i have to provide some parameters like JAVA_HOME, then where i will provide? I always get JAVA_HOME not set during hadoop installation, if i dont have root permission.
      Hide
      Harsh J added a comment -

      Hi,

      The VM is not something Bigtop produces (AFAICT), so lets not discuss that as a comparative.

      Also, JAVA_HOME is detected by Bigtop's java-home detection scripts and has worked well for many so you shouldn't need to override it in normal cases (where the JDK is also package installed). But it can be defined in /etc/default/hadoop file as well.

      Show
      Harsh J added a comment - Hi, The VM is not something Bigtop produces (AFAICT), so lets not discuss that as a comparative. Also, JAVA_HOME is detected by Bigtop's java-home detection scripts and has worked well for many so you shouldn't need to override it in normal cases (where the JDK is also package installed). But it can be defined in /etc/default/hadoop file as well.
      Hide
      Pradeep Bhadani added a comment -

      ok..i will try to set JAVA_HOME in /etc/default/hadoop

      Show
      Pradeep Bhadani added a comment - ok..i will try to set JAVA_HOME in /etc/default/hadoop
      Hide
      Roman Shaposhnik added a comment -

      It seems that this is now fixed in trunk

      Show
      Roman Shaposhnik added a comment - It seems that this is now fixed in trunk

        People

        • Assignee:
          Roman Shaposhnik
          Reporter:
          Harsh J
        • Votes:
          0 Vote for this issue
          Watchers:
          4 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development