Accumulo
  1. Accumulo
  2. ACCUMULO-2423

Converge Shell scripts on single implementation

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None

      Description

      Do we want to have a stated dependency on a particular shell? Most of our scripts explicitly use bash, but some use sh. Most scripts invoke #!/usr/bin/env bash but some do #!/bin/bash or other crazy things.

      I don't have a particular preference which way we go, but I'd like to see us standardize one way or the other.

      If we use /bin/sh, then we can run everything through the checkbashisms script (also available for other distros)

      If we go the other way, and switch everything to bash, then we can run them through ksh -n for deprecation warnings. I know that's a different shell, but they're still moderately useful.

        Issue Links

          Activity

          Hide
          Christopher Tubbs added a comment -

          I prefer

          #! /usr/bin/env bash

          because bash could be in /bin or /usr/bin, and I don't think we should care. I think bash is common enough, we don't need to care about using standard "bashisms", unless it's very new (bash >= 4.0), which may not exist on a Mac (OS X 10.8.5 has bash 3.2) or another likely supported platform.

          Example: bash 3.x doesn't support

          ${x^^}

          and doesn't properly do (at least on OS X 10.8.5)

          x=abc echo ${x}

          instead requiring something like

          export x=abc && echo ${x}
          Show
          Christopher Tubbs added a comment - I prefer #! /usr/bin/env bash because bash could be in /bin or /usr/bin, and I don't think we should care. I think bash is common enough, we don't need to care about using standard "bashisms", unless it's very new (bash >= 4.0), which may not exist on a Mac (OS X 10.8.5 has bash 3.2) or another likely supported platform. Example: bash 3.x doesn't support ${x^^} and doesn't properly do (at least on OS X 10.8.5) x=abc echo ${x} instead requiring something like export x=abc && echo ${x}
          Hide
          Bill Havanki added a comment -

          I much prefer bash over sh.

          bash 3 is certainly an acceptable minimum. Like Christopher said, bash 4 isn't shipped with Mac OS X ... even Mavericks is still on 3.2. However, it's easy to get it via Homebrew or MacPorts so it's not that unreasonable to require it there.

          Show
          Bill Havanki added a comment - I much prefer bash over sh. bash 3 is certainly an acceptable minimum. Like Christopher said, bash 4 isn't shipped with Mac OS X ... even Mavericks is still on 3.2 . However, it's easy to get it via Homebrew or MacPorts so it's not that unreasonable to require it there.
          Hide
          Mike Drob added a comment -

          Christopher Tubbs - I only meant to suggest checking for bashisms if we are using a non-bash shell. That is, make sure we aren't using any bash extensions when running through sh.

          Bash 3 is stable enough and featured enough that I'm fine with mandating it.

          Show
          Mike Drob added a comment - Christopher Tubbs - I only meant to suggest checking for bashisms if we are using a non-bash shell. That is, make sure we aren't using any bash extensions when running through sh . Bash 3 is stable enough and featured enough that I'm fine with mandating it.
          Hide
          Mike Drob added a comment -

          Attached is my first cut at implementing best practices for our shell scripts.

          This patch includes the following changes:

          • Standardize interpreter line on '#! /usr/bin/env foo'
          • Replace `...` with $(...) in bash scripts
          • Replace [...] with [[...]] in bash scripts
          • Replace = with == in [[...]] in bash scripts
          Show
          Mike Drob added a comment - Attached is my first cut at implementing best practices for our shell scripts. This patch includes the following changes: Standardize interpreter line on '#! /usr/bin/env foo' Replace `...` with $(...) in bash scripts Replace [...] with [[...]] in bash scripts Replace = with == in [[...]] in bash scripts
          Hide
          Mike Drob added a comment -

          Another good utility to run would be shellcheck, even though it looks to be awfully pedantic in some cases. And requires Haskell...

          Show
          Mike Drob added a comment - Another good utility to run would be shellcheck , even though it looks to be awfully pedantic in some cases. And requires Haskell...
          Hide
          Christopher Tubbs added a comment -

          I'm not sure I see the benefit of [[...]] over [...], as a standard, because I think that whatever works is fine. [[...]] is more flexible, so that's fine, but I'm not sure it warrants standardizing on it. This does presume bash, so just to be clear, bash is a dependency of this suggestion. I don't have a problem with it, per se, I just don't see much value in doing it.

          As for == over = as a standard, I actually dislike that. It looks incorrect, because of the precedent, and only works with [[...]]. If we were to standardize on one over the other, I say use the single =. Granted, I understand this looks like an assignment to those unfamiliar with shell scripting, but I think it's better to teach people to avoid ==, because of its limitations. It's just a better habit to use =, I think, when writing shell scripts.

          Show
          Christopher Tubbs added a comment - I'm not sure I see the benefit of [[...]] over [...], as a standard, because I think that whatever works is fine. [[...]] is more flexible, so that's fine, but I'm not sure it warrants standardizing on it. This does presume bash, so just to be clear, bash is a dependency of this suggestion. I don't have a problem with it, per se, I just don't see much value in doing it. As for == over = as a standard, I actually dislike that. It looks incorrect, because of the precedent, and only works with [[...]]. If we were to standardize on one over the other, I say use the single =. Granted, I understand this looks like an assignment to those unfamiliar with shell scripting, but I think it's better to teach people to avoid ==, because of its limitations. It's just a better habit to use =, I think, when writing shell scripts.
          Hide
          Mike Drob added a comment -

          If you're using bash, then there is no reason to not use [[. The main difference is that [[ handles word splitting and empty strings better than [. Example:

          mdrob@mdrob-W530:~$ echo $X
          
          mdrob@mdrob-W530:~$ [[ "" = $X ]] && echo true || echo false
          true
          mdrob@mdrob-W530:~$ [ "" = $X ] && echo true || echo false
          bash: [: : unary operator expected
          false
          

          Regarding = or ==, I personally prefer == because it matches with the numeric equality check in ((..)). Using = is fine though, I don't have a strong argument there.

          Show
          Mike Drob added a comment - If you're using bash, then there is no reason to not use [[. The main difference is that [[ handles word splitting and empty strings better than [. Example: mdrob@mdrob-W530:~$ echo $X mdrob@mdrob-W530:~$ [[ "" = $X ]] && echo true || echo false true mdrob@mdrob-W530:~$ [ "" = $X ] && echo true || echo false bash: [: : unary operator expected false Regarding = or ==, I personally prefer == because it matches with the numeric equality check in ((..)). Using = is fine though, I don't have a strong argument there.
          Hide
          Mike Drob added a comment -

          Note that I would be more than happy to converge on sh instead, but that might make other people's script writing experience much more painful. sh has a lot more rough edges than bash does.

          Show
          Mike Drob added a comment - Note that I would be more than happy to converge on sh instead, but that might make other people's script writing experience much more painful. sh has a lot more rough edges than bash does.
          Hide
          Christopher Tubbs added a comment -

          Yeah, I'm aware of the differences between [[...]] and [...] regarding unquoted variables, but I'm not sure that warrants establishing a standard.

          I can see the '==' consistency argument with numeric equality in ((..)), if we're requiring bash >= 3 to be used anyway.

          +0 on both counts.
          +1 on the suggested env/bash line and $(...) over backticks.

          Show
          Christopher Tubbs added a comment - Yeah, I'm aware of the differences between [[...]] and [...] regarding unquoted variables, but I'm not sure that warrants establishing a standard. I can see the '==' consistency argument with numeric equality in ((..)), if we're requiring bash >= 3 to be used anyway. +0 on both counts. +1 on the suggested env/bash line and $(...) over backticks.
          Hide
          William Slacum added a comment -

          How's about we ditch bash for something readable like python or ruby. It adds a system level dependency, but I think it'd make things more accessible.

          Show
          William Slacum added a comment - How's about we ditch bash for something readable like python or ruby. It adds a system level dependency, but I think it'd make things more accessible.
          Hide
          Josh Elser added a comment -

          How's about we ditch bash for something [...] like python or ruby

          I can't say I'm opposed to that. Granted, python and ruby would also have their own caveats about the version installed (python 2 to 3 being the biggest but still easy to work around).

          Show
          Josh Elser added a comment - How's about we ditch bash for something [...] like python or ruby I can't say I'm opposed to that. Granted, python and ruby would also have their own caveats about the version installed (python 2 to 3 being the biggest but still easy to work around).
          Hide
          Christopher Tubbs added a comment -

          Not ruby, please. Regarding python, I don't think it makes much sense to move everything to python... especially when it's just something simple, like launching randomwalk or some other environment set-up, just to run java, or something simple like that. For some things, it may make sense to use python... but I think that might be outside the scope of this ticket, and we should discuss those things in their proper context.

          Show
          Christopher Tubbs added a comment - Not ruby, please. Regarding python, I don't think it makes much sense to move everything to python... especially when it's just something simple, like launching randomwalk or some other environment set-up, just to run java, or something simple like that. For some things, it may make sense to use python... but I think that might be outside the scope of this ticket, and we should discuss those things in their proper context.
          Hide
          Mike Drob added a comment -
          Show
          Mike Drob added a comment - Review available at https://reviews.apache.org/r/23393/

            People

            • Assignee:
              Mike Drob
              Reporter:
              Mike Drob
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 10m
                4h 10m

                  Development