Hadoop Common
  1. Hadoop Common
  2. HADOOP-7710

create a script to setup application in order to create root directories for application such hbase, hcat, hive etc

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    1. HADOOP-7710-1.patch
      4 kB
      Arpit Gupta
    2. HADOOP-7710.20s-1.patch
      4 kB
      Arpit Gupta
    3. HADOOP-7710.patch
      4 kB
      Arpit Gupta
    4. HADOOP-7710.20s.patch
      4 kB
      Arpit Gupta

      Activity

      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-trunk #851 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/851/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #851 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/851/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-0.23-Build #37 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/37/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178874
      Files :

      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #37 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/37/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178874 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-0.23-Build #30 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/30/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178874
      Files :

      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #30 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/30/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178874 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk #821 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/821/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #821 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/821/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-trunk-Commit #1030 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1030/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1030 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1030/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Common-trunk-Commit #1012 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1012/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1012 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1012/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk-Commit #1090 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1090/)
      HADOOP-7710. Added hadoop-setup-application.sh for creating
      application directory (Arpit Gupta via Eric Yang)

      eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876
      Files :

      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
      • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1090 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1090/ ) HADOOP-7710 . Added hadoop-setup-application.sh for creating application directory (Arpit Gupta via Eric Yang) eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1178876 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-applications.sh
      Hide
      Eric Yang added a comment -

      Thanks Arpit.

      Show
      Eric Yang added a comment - Thanks Arpit.
      Hide
      Eric Yang added a comment -

      I just committed this to trunk, 0.23, 0.20-security, 0.20.205.

      Show
      Eric Yang added a comment - I just committed this to trunk, 0.23, 0.20-security, 0.20.205.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12497585/HADOOP-7710-1.patch
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      -1 tests included. The patch doesn't appear to include any new or modified tests.
      Please justify why no new tests are needed for this patch.
      Also please list what manual steps were performed to verify this patch.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 core tests. The patch passed unit tests in .

      +1 contrib tests. The patch passed contrib unit tests.

      Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/261//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/261//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497585/HADOOP-7710-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/261//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/261//console This message is automatically generated.
      Hide
      Arpit Gupta added a comment -

      Added the option to specify a user that is not the same as the app.

      You can specify apps as app1,app2:user,app3 etc. It will assume the user for app1 to be app1 and for app2 will use the user specified 'user'

      Show
      Arpit Gupta added a comment - Added the option to specify a user that is not the same as the app. You can specify apps as app1,app2:user,app3 etc. It will assume the user for app1 to be app1 and for app2 will use the user specified 'user'
      Hide
      Arpit Gupta added a comment -

      no tests included as this is a setup script. Manually verified this script buy running on an installed cluster.

      Show
      Arpit Gupta added a comment - no tests included as this is a setup script. Manually verified this script buy running on an installed cluster.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12497266/HADOOP-7710.patch
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      -1 tests included. The patch doesn't appear to include any new or modified tests.
      Please justify why no new tests are needed for this patch.
      Also please list what manual steps were performed to verify this patch.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 core tests. The patch passed unit tests in .

      +1 contrib tests. The patch passed contrib unit tests.

      Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/252//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/252//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497266/HADOOP-7710.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/252//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/252//console This message is automatically generated.
      Hide
      Arpit Gupta added a comment -

      branch for trunk

      Show
      Arpit Gupta added a comment - branch for trunk
      Hide
      Arpit Gupta added a comment -

      patch for branch 20 security

      Show
      Arpit Gupta added a comment - patch for branch 20 security
      Hide
      Arpit Gupta added a comment -

      the app dir name is the same name as the user that way we can make that user have write permission to the appropriate directory.

      Show
      Arpit Gupta added a comment - the app dir name is the same name as the user that way we can make that user have write permission to the appropriate directory.
      Hide
      Arpit Gupta added a comment -

      just like how we have a hadoop-setup-user script we should add a hadoop-setup-applications script to setup root directories that they can write to.

      Perhaps create directory structure like

      /apps/hbbase
      /apps/hcat
      /apps/hive

      etc. These dirs get created and appropriate users get permissions to write to these directories.

      Show
      Arpit Gupta added a comment - just like how we have a hadoop-setup-user script we should add a hadoop-setup-applications script to setup root directories that they can write to. Perhaps create directory structure like /apps/hbbase /apps/hcat /apps/hive etc. These dirs get created and appropriate users get permissions to write to these directories.

        People

        • Assignee:
          Arpit Gupta
          Reporter:
          Arpit Gupta
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development