Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1325

Update TestHDFSCLI to include tests from upstream

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 1.0.0
    • Component/s: tests
    • Labels:
      None

      Description

      Add the following commands to TestCLI:

      appendToFile
      text
      rmdir
      rmdir with ignore-fail-on-non-empty
      df
      expunge
      getmerge

      The following tests have been added as TestDFSCLI in BIGTOP-1334:

      allowSnapshot
      disallowSnapshot
      createSnapshot
      renameSnapshot
      deleteSnapshot
      refreshUserToGroupsMappings
      refreshSuperUserGroupsConfiguration
      setQuota
      clrQuota
      setSpaceQuota
      setBalancerBandwidth
      finalizeUpgrade

      1. BIGTOP-1325.patch
        7 kB
        Dasha Boudnik
      2. BIGTOP-1325.patch
        7 kB
        Dasha Boudnik
      3. BIGTOP-1325.patch
        13 kB
        Dasha Boudnik

        Issue Links

          Activity

          Hide
          dasha.boudnik Dasha Boudnik added a comment -

          Patch attached.

          Show
          dasha.boudnik Dasha Boudnik added a comment - Patch attached.
          Hide
          dasha.boudnik Dasha Boudnik added a comment - - edited

          I just realized I automatically opened this as a BigTop issue, but it's actually Hadoop HDFS. Opening it there and keeping this one for now, since we'll ultimately need to add these tests in BigTop, too, as far as I understand...apologies for the confusion.

          Show
          dasha.boudnik Dasha Boudnik added a comment - - edited I just realized I automatically opened this as a BigTop issue, but it's actually Hadoop HDFS. Opening it there and keeping this one for now, since we'll ultimately need to add these tests in BigTop, too, as far as I understand...apologies for the confusion.
          Hide
          dasha.boudnik Dasha Boudnik added a comment -

          NOW the patch is attached.

          Show
          dasha.boudnik Dasha Boudnik added a comment - NOW the patch is attached.
          Hide
          mbukatov Martin Bukatovic added a comment - - edited

          Looking at the patch, don't we need to rename USERNAME to USER_NAME? See:

          $ grep USERNAME testHDFSConf.xml | wc -l
          0
          $ grep USER_NAME testHDFSConf.xml | wc -l
          81
          

          Also see recent JIRA: BIGTOP-1324

          Show
          mbukatov Martin Bukatovic added a comment - - edited Looking at the patch, don't we need to rename USERNAME to USER_NAME ? See: $ grep USERNAME testHDFSConf.xml | wc -l 0 $ grep USER_NAME testHDFSConf.xml | wc -l 81 Also see recent JIRA: BIGTOP-1324
          Hide
          dasha.boudnik Dasha Boudnik added a comment -

          Patch attached.

          Show
          dasha.boudnik Dasha Boudnik added a comment - Patch attached.
          Hide
          cos Konstantin Boudnik added a comment -

          Looks good. One comment though. When you do

          +        <command>-fs NAMENODE -mkdir -p /user/USER_NAME/dir1</command>
          +        <command>-fs NAMENODE -rmdir /user/USER_NAME/dir1</command>
          +        <command>-fs NAMENODE -ls /user/USER_NAME/</command>
          

          you are later checking that /user/USER_NAME/ is empty (ie the ls on the directory has no output). It might not be the case if cluster is active and other clients are putting files under /user/USER_NAME/. I'd recommend to
          do something like this

          +        <command>-fs NAMENODE -mkdir -p /user/USER_NAME/dir1/dir2</command>
          +        <command>-fs NAMENODE -rmdir /user/USER_NAME/dir1/dir2</command>
          +        <command>-fs NAMENODE -ls /user/USER_NAME/dir1</command>
          

          and then remove dir1 as before. Makes sense?

          Show
          cos Konstantin Boudnik added a comment - Looks good. One comment though. When you do + <command>-fs NAMENODE -mkdir -p /user/USER_NAME/dir1</command> + <command>-fs NAMENODE -rmdir /user/USER_NAME/dir1</command> + <command>-fs NAMENODE -ls /user/USER_NAME/</command> you are later checking that /user/USER_NAME/ is empty (ie the ls on the directory has no output). It might not be the case if cluster is active and other clients are putting files under /user/USER_NAME/ . I'd recommend to do something like this + <command>-fs NAMENODE -mkdir -p /user/USER_NAME/dir1/dir2</command> + <command>-fs NAMENODE -rmdir /user/USER_NAME/dir1/dir2</command> + <command>-fs NAMENODE -ls /user/USER_NAME/dir1</command> and then remove dir1 as before. Makes sense?
          Hide
          dasha.boudnik Dasha Boudnik added a comment -

          Yup, thanks. Changed rmdir and rmdir with ignore-fail-on-non-empty accordingly, tests are passing.

          Show
          dasha.boudnik Dasha Boudnik added a comment - Yup, thanks. Changed rmdir and rmdir with ignore-fail-on-non-empty accordingly, tests are passing.
          Hide
          cos Konstantin Boudnik added a comment -

          Thanks! Looks good, +1

          Show
          cos Konstantin Boudnik added a comment - Thanks! Looks good, +1
          Hide
          dasha.boudnik Dasha Boudnik added a comment -

          Committed and pushed. Thanks!

          Show
          dasha.boudnik Dasha Boudnik added a comment - Committed and pushed. Thanks!

            People

            • Assignee:
              dasha.boudnik Dasha Boudnik
              Reporter:
              dasha.boudnik Dasha Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development