Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1235

Speed-up init-hdfs.sh for complete HCFS compliant provisioning

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment, general
    • Labels:
      None

      Description

      BIGTOP-1200 provides file schema in JSON format, for any HCFS related tool to read.

      BIGTOP-952 meanwhile give us fast provisioning by running in a single JVM (without multiple haoop fs -... commands_).

      Thus, when those two are finished, we can replace init-hdfs.sh (with the exception of a few extra shell commands in bottom of that script which still will need to be called).

      So this JIRA will:

      • create a new init-hcfs.groovy script which calls provision.groovy in BIGTOP-952, with init-hcfs.json as input.
      • add in the remaining straggler commands from init-hdfs.sh which dont fit into the generic json format as shell commands.
      • delete init-hdfs.sh entirely
      • update puppet scripts to use init-hcfs.groovy, calling them with the "hdfs" argument.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cos Konstantin Boudnik
                Reporter:
                jayunit100 jay vyas
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: