Details

    • Type: Test Test
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Not a Problem
    • Affects Version/s: 0.22.0, 0.23.0
    • Fix Version/s: 0.22.0, 0.23.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There is no test coverage for quotas and append. Let's add a test to TestQuota that covers that quotas are updated correctly when appending to a file.

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          Looks good, apologies for missing this.

          Show
          Eli Collins added a comment - Looks good, apologies for missing this.
          Konstantin Shvachko made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Not A Problem [ 8 ]
          Hide
          Konstantin Boudnik added a comment -

          Looking good! Thank you

          Show
          Konstantin Boudnik added a comment - Looking good! Thank you
          Po Cheung made changes -
          Attachment DFSClient.append.png [ 12470298 ]
          Hide
          Po Cheung added a comment -

          Ran code coverage against TestQuota and verified that DFSClient.append is covered (see attachment).

          Show
          Po Cheung added a comment - Ran code coverage against TestQuota and verified that DFSClient.append is covered (see attachment).
          Hide
          Konstantin Boudnik added a comment -

          I'd suggest to run a coverage build and see if the 0-coverage has been addressed.

          Show
          Konstantin Boudnik added a comment - I'd suggest to run a coverage build and see if the 0-coverage has been addressed.
          Hide
          Konstantin Shvachko added a comment -

          It looks like we have both positive and negative tests. Cannot think of anything to add.
          Eli, do you?

          Show
          Konstantin Shvachko added a comment - It looks like we have both positive and negative tests. Cannot think of anything to add. Eli, do you?
          Hide
          Po Cheung added a comment -

          There exists a testcase for quotas and append in TestQuota.testSpaceCommands. Will that suffice?

                // Test Append :
                
                // verify space quota
                c = dfs.getContentSummary(quotaDir1);
                assertEquals(c.getSpaceQuota(), 4 * fileSpace);
                
                // verify space before append;
                c = dfs.getContentSummary(dstPath);
                assertEquals(c.getSpaceConsumed(), 3 * fileSpace);
                
                OutputStream out = dfs.append(file2);
                // appending 1 fileLen should succeed
                out.write(new byte[fileLen]);
                out.close();
                
                file2Len += fileLen; // after append
                
                // verify space after append;
                c = dfs.getContentSummary(dstPath);
                assertEquals(c.getSpaceConsumed(), 4 * fileSpace);
                
                // now increase the quota for quotaDir1
                dfs.setQuota(quotaDir1, FSConstants.QUOTA_DONT_SET, 5 * fileSpace);
                // Now, appending more than 1 fileLen should result in an error
                out = dfs.append(file2);
                hasException = false;
                try {
                  out.write(new byte[fileLen + 1024]);
                  out.flush();
                  out.close();
                } catch (DSQuotaExceededException e) {
                  hasException = true;
                  IOUtils.closeStream(out);
                }
                assertTrue(hasException);
                
                file2Len += fileLen; // after partial append
                
                // verify space after partial append
                c = dfs.getContentSummary(dstPath);
                assertEquals(c.getSpaceConsumed(), 5 * fileSpace);
          
          Show
          Po Cheung added a comment - There exists a testcase for quotas and append in TestQuota.testSpaceCommands. Will that suffice? // Test Append : // verify space quota c = dfs.getContentSummary(quotaDir1); assertEquals(c.getSpaceQuota(), 4 * fileSpace); // verify space before append; c = dfs.getContentSummary(dstPath); assertEquals(c.getSpaceConsumed(), 3 * fileSpace); OutputStream out = dfs.append(file2); // appending 1 fileLen should succeed out.write(new byte[fileLen]); out.close(); file2Len += fileLen; // after append // verify space after append; c = dfs.getContentSummary(dstPath); assertEquals(c.getSpaceConsumed(), 4 * fileSpace); // now increase the quota for quotaDir1 dfs.setQuota(quotaDir1, FSConstants.QUOTA_DONT_SET, 5 * fileSpace); // Now, appending more than 1 fileLen should result in an error out = dfs.append(file2); hasException = false; try { out.write(new byte[fileLen + 1024]); out.flush(); out.close(); } catch (DSQuotaExceededException e) { hasException = true; IOUtils.closeStream(out); } assertTrue(hasException); file2Len += fileLen; // after partial append // verify space after partial append c = dfs.getContentSummary(dstPath); assertEquals(c.getSpaceConsumed(), 5 * fileSpace);
          Hide
          Konstantin Boudnik added a comment -

          Also be aware that now test coverage can report which tests cover what parts of the code.

          Show
          Konstantin Boudnik added a comment - Also be aware that now test coverage can report which tests cover what parts of the code.
          Konstantin Shvachko made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          Hide
          Konstantin Shvachko added a comment -

          Blocker. Quotas with append have zero test coverage.

          Show
          Konstantin Shvachko added a comment - Blocker. Quotas with append have zero test coverage.
          Eli Collins made changes -
          Field Original Value New Value
          Link This issue is related to HDFS-1377 [ HDFS-1377 ]
          Eli Collins created issue -

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development