• Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: deployment
    • Labels:


      There are a few things that can be improved regarding the HDFS helper script introduced by BIGTOP-547. Some of these issues have been discussed in BIGTOP-637.

      1. The script seems to create user directories for users "root" and "jenkins" but not the current user running the script. I think it will be a good idea to add in the script the commands to create the /user/$USER directory in HDFS. Of course, we should be careful in case the user running the command is root or jenkins, in that case a simple mkdir command would throw error given that those directories already exist.

      2. The script uses sudo which requires login shell. However, login shells are not available for certain use cases like init scripts. Consequently, we should consider using su instead.

      3. Look into how the helper script can be made faster

      Folks, please feel free to add/edit if there is something I missed.

      1. 0001-BIGTOP-852.-Improve-HDFS-helper-script.patch
        1 kB
        Bruno Mahé
      2. untarHdfs.groovy
        2 kB
        Roman Shaposhnik

        Issue Links


          Mark Grover created issue -
          Mark Grover made changes -
          Field Original Value New Value
          Link This issue is related to BIGTOP-547 [ BIGTOP-547 ]
          Bruno Mahé made changes -
          Parent BIGTOP-637 [ 12595331 ]
          Issue Type Bug [ 1 ] Sub-task [ 7 ]
          Mark Grover made changes -
          Assignee Mark Grover [ mgrover ]
          Roman Shaposhnik made changes -
          Attachment untarHdfs.groovy [ 12575644 ]
          Bruno Mahé made changes -
          Bruno Mahé made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Bruno Mahé made changes -
          Assignee Mark Grover [ mgrover ] Bruno Mahé [ bmahe ]
          Bruno Mahé made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Konstantin Boudnik made changes -
          Link This issue is related to BIGTOP-952 [ BIGTOP-952 ]


            • Assignee:
              Bruno Mahé
              Mark Grover
            • Votes:
              0 Vote for this issue
              5 Start watching this issue


              • Created: