Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3393

Use Iterables.removeIf instead of Iterator.remove in HBase filters

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.9.0
    • Labels:
      None

      Description

      This is a performance improvement to use Iterables.removeIf in the filterRowCells method of ColumnProjectionFilter as described here:

      https://rayokota.wordpress.com/2016/10/20/tips-on-writing-custom-hbase-filters/

        Issue Links

          Activity

          Hide
          jamestaylor James Taylor added a comment -

          Thanks for the patch, Robert Yokota, and nice blog too. Looks good to me. What do you think, Ankit Singhal? If it looks good to you, would you mind committing?

          Have you done any performance comparisons, Robert?

          Show
          jamestaylor James Taylor added a comment - Thanks for the patch, Robert Yokota , and nice blog too. Looks good to me. What do you think, Ankit Singhal ? If it looks good to you, would you mind committing? Have you done any performance comparisons, Robert?
          Hide
          rayokota Robert Yokota added a comment -

          Thanks, I just did the performance comparison for one of our own custom filters as mentioned in the blog.

          Show
          rayokota Robert Yokota added a comment - Thanks, I just did the performance comparison for one of our own custom filters as mentioned in the blog.
          Hide
          rayokota Robert Yokota added a comment -

          Also, as Ted Yu mentioned, if Phoenix is using Java 8, you can use Collection.removeIf instead

          Show
          rayokota Robert Yokota added a comment - Also, as Ted Yu mentioned, if Phoenix is using Java 8, you can use Collection.removeIf instead
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12834487/PHOENIX-3393.master.001.patch
          against master branch at commit 1e78d3b368b7aa8397224074f691ea2fed340e34.
          ATTACHMENT ID: 12834487

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 43 warning messages.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//artifact/patchprocess/patchReleaseAuditWarnings.txt
          Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//artifact/patchprocess/patchJavadocWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12834487/PHOENIX-3393.master.001.patch against master branch at commit 1e78d3b368b7aa8397224074f691ea2fed340e34. ATTACHMENT ID: 12834487 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 43 warning messages. -1 release audit . The applied patch generated 1 release audit warnings (more than the master's current 0 warnings). +1 lineLengths . The patch does not introduce lines longer than 100 +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//artifact/patchprocess/patchReleaseAuditWarnings.txt Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/631//console This message is automatically generated.
          Hide
          ankit@apache.org Ankit Singhal added a comment -

          LGTM as well. Thanks Robert Yokota for the contribution. Checked code for other areas if above optimization can be applied, but didn't find any code with randomAccessList(like Arraylist) and having significant remove operations.

          Committed your change to 4.x branches and master.

          Show
          ankit@apache.org Ankit Singhal added a comment - LGTM as well. Thanks Robert Yokota for the contribution. Checked code for other areas if above optimization can be applied, but didn't find any code with randomAccessList(like Arraylist) and having significant remove operations. Committed your change to 4.x branches and master.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Phoenix-master #1444 (See https://builds.apache.org/job/Phoenix-master/1444/)
          PHOENIX-3393 Use Iterables.removeIf instead of Iterator.remove in HBase (ankitsinghal59: rev 5c9fb7b68ebeed20e3876579534df4adf21943a4)

          • (edit) phoenix-core/src/main/java/org/apache/phoenix/filter/ColumnProjectionFilter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Phoenix-master #1444 (See https://builds.apache.org/job/Phoenix-master/1444/ ) PHOENIX-3393 Use Iterables.removeIf instead of Iterator.remove in HBase (ankitsinghal59: rev 5c9fb7b68ebeed20e3876579534df4adf21943a4) (edit) phoenix-core/src/main/java/org/apache/phoenix/filter/ColumnProjectionFilter.java

            People

            • Assignee:
              rayokota Robert Yokota
              Reporter:
              rayokota Robert Yokota
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development