Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2238

Provisioner should propagate hdfs ssh keys for testing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: deployment, tests
    • Labels:
      None

      Description

      Our deployment has an option to setup ssh keys for hdfs passwordless ssh accross the cluster. Clearly, being quite a permissive feature, this shouldn't be automatically set for the production. However, in the test setting it is indeed useful, and some of the tests are relying on it. Ie see TestBlockRecovery.

      I'd like to propagate these keys in the docker and vm provisioner by default. For this we'll need to set
      hadoop::common_hdfs::testonly_hdfs_sshkeys: "yes"

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          An easy fix.

          Show
          cos Konstantin Boudnik added a comment - An easy fix.
          Hide
          cos Konstantin Boudnik added a comment -

          Which turn out not to have the effect. The variable is getting propagated, but for whatever reason the keys aren't getting installed. Will look into it tomorrow.

          Show
          cos Konstantin Boudnik added a comment - Which turn out not to have the effect. The variable is getting propagated, but for whatever reason the keys aren't getting installed. Will look into it tomorrow.
          Hide
          cos Konstantin Boudnik added a comment -

          Ah, scratch that - they keys are misplaced. However with BIGTOP-2252 everything works just fine!

          Show
          cos Konstantin Boudnik added a comment - Ah, scratch that - they keys are misplaced. However with BIGTOP-2252 everything works just fine!
          Hide
          cos Konstantin Boudnik added a comment -

          Tested with TestBlockRecovery that requires password-less ssh under hdfs user and it does work as expected.

          Show
          cos Konstantin Boudnik added a comment - Tested with TestBlockRecovery that requires password-less ssh under hdfs user and it does work as expected.
          Hide
          cos Konstantin Boudnik added a comment -

          Any input here on the general approach of enabling the key deployment by default? Or shell we use a env.var to trigger this?

          Show
          cos Konstantin Boudnik added a comment - Any input here on the general approach of enabling the key deployment by default? Or shell we use a env.var to trigger this?
          Hide
          cos Konstantin Boudnik added a comment -

          If I don't hear from anyone by the end of the I will commit this as is.

          Show
          cos Konstantin Boudnik added a comment - If I don't hear from anyone by the end of the I will commit this as is.
          Hide
          cos Konstantin Boudnik added a comment -

          Pushed to the master. By default the hdfs ssh keys will be deployed into provisioned docker cluster, so all HDFS test can run properly.

          Show
          cos Konstantin Boudnik added a comment - Pushed to the master. By default the hdfs ssh keys will be deployed into provisioned docker cluster, so all HDFS test can run properly.

            People

            • Assignee:
              cos Konstantin Boudnik
              Reporter:
              cos Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development