Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-4309

Potential null pointer dereference in DelegatingConfiguration#keySet()

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0, 1.1.2
    • Component/s: None
    • Labels:
      None

      Description

          final int prefixLen = this.prefix == null ? 0 : this.prefix.length();
      
          for (String key : this.backingConfig.keySet()) {
            if (key.startsWith(this.prefix)) {
      

      If this.prefix == null, we would get NPE in startsWith():

          public boolean startsWith(String prefix, int toffset) {
              char ta[] = value;
              int to = toffset;
              char pa[] = prefix.value;
      

        Issue Links

          Activity

          Hide
          uce Ufuk Celebi added a comment -

          Fixed in ad34540 (master), c2906c0 (release-1.1)

          Show
          uce Ufuk Celebi added a comment - Fixed in ad34540 (master), c2906c0 (release-1.1)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/2371

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2371
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tsunny commented on the issue:

          https://github.com/apache/flink/pull/2371

          No problem. Happy to contribute. Thanks for the review comments.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tsunny commented on the issue: https://github.com/apache/flink/pull/2371 No problem. Happy to contribute. Thanks for the review comments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/2371

          Thanks for addressing our comments! I'm going to merge this later today.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/2371 Thanks for addressing our comments! I'm going to merge this later today.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74901158

          — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java —
          @@ -56,6 +57,7 @@ public DelegatingConfiguration() {
          */
          public DelegatingConfiguration(Configuration backingConfig, String prefix)
          {
          + Preconditions.checkNotNull(backingConfig);
          this.backingConfig = backingConfig;
          — End diff –

          Trivial note: We can also write `this.backingConfig = checkNotNull(backingConfig)` here.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74901158 — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java — @@ -56,6 +57,7 @@ public DelegatingConfiguration() { */ public DelegatingConfiguration(Configuration backingConfig, String prefix) { + Preconditions.checkNotNull(backingConfig); this.backingConfig = backingConfig; — End diff – Trivial note: We can also write `this.backingConfig = checkNotNull(backingConfig)` here.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74894098

          — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java —
          @@ -178,14 +178,19 @@ public String toString() {

          @Override
          public Set<String> keySet() {
          +
          final HashSet<String> set = new HashSet<String>();

          • final int prefixLen = this.prefix == null ? 0 : this.prefix.length();

          for (String key : this.backingConfig.keySet()) {

          • if (key.startsWith(this.prefix)) {
            +
            + if (this.prefix == null) {
              • End diff –

          Actually, it's probably better to just return the keySet of the backing config.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74894098 — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java — @@ -178,14 +178,19 @@ public String toString() { @Override public Set<String> keySet() { + final HashSet<String> set = new HashSet<String>(); final int prefixLen = this.prefix == null ? 0 : this.prefix.length(); for (String key : this.backingConfig.keySet()) { if (key.startsWith(this.prefix)) { + + if (this.prefix == null) { End diff – Actually, it's probably better to just return the keySet of the backing config.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/2371

          Changes look good all in all. Could you also add a `org.apache.flink.util.Preconditions.checkNotNull` check for the backing config in the constructor?

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/2371 Changes look good all in all. Could you also add a `org.apache.flink.util.Preconditions.checkNotNull` check for the backing config in the constructor?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74893687

          — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java —
          @@ -178,14 +178,19 @@ public String toString() {

          @Override
          public Set<String> keySet() {
          +
          final HashSet<String> set = new HashSet<String>();

          • final int prefixLen = this.prefix == null ? 0 : this.prefix.length();

          for (String key : this.backingConfig.keySet()) {

          • if (key.startsWith(this.prefix)) {
            +
            + if (this.prefix == null) {
              • End diff –

          Yes, we can use can use `addAll`

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74893687 — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java — @@ -178,14 +178,19 @@ public String toString() { @Override public Set<String> keySet() { + final HashSet<String> set = new HashSet<String>(); final int prefixLen = this.prefix == null ? 0 : this.prefix.length(); for (String key : this.backingConfig.keySet()) { if (key.startsWith(this.prefix)) { + + if (this.prefix == null) { End diff – Yes, we can use can use `addAll`
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tillrohrmann commented on the issue:

          https://github.com/apache/flink/pull/2371

          Thanks for your contribution @tsunny. Your changes look good. I only had a minor comment.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/2371 Thanks for your contribution @tsunny. Your changes look good. I only had a minor comment.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74893581

          — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java —
          @@ -88,4 +90,49 @@ private String typeParamToString(Class<?>[] classes)

          { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); }

          }
          +
          + @Test
          + public void testDelegationConfigurationWithNullPrefix() {
          +
          + Configuration backingConf = new Configuration();
          + backingConf.setValueInternal("test-key", "value");
          +
          + DelegatingConfiguration configuration = new DelegatingConfiguration(
          + backingConf, null);
          + Set<String> keySet = configuration.keySet();
          +
          + assertEquals(keySet, backingConf.keySet());
          +
          — End diff –

          empty line

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74893581 — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java — @@ -88,4 +90,49 @@ private String typeParamToString(Class<?>[] classes) { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); } } + + @Test + public void testDelegationConfigurationWithNullPrefix() { + + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("test-key", "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration( + backingConf, null); + Set<String> keySet = configuration.keySet(); + + assertEquals(keySet, backingConf.keySet()); + — End diff – empty line
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74893573

          — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java —
          @@ -88,4 +90,49 @@ private String typeParamToString(Class<?>[] classes)

          { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); }

          }
          +
          + @Test
          + public void testDelegationConfigurationWithNullPrefix()

          { + + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("test-key", "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration( + backingConf, null); + Set<String> keySet = configuration.keySet(); + + assertEquals(keySet, backingConf.keySet()); + + }

          +
          + @Test
          + public void testDelegationConfigurationWithPrefix() {
          +
          + String prefix = "pref-";
          + String expectedKey = "key";
          +
          + /*
          + * Key matches the prefix
          + */
          + Configuration backingConf = new Configuration();
          + backingConf.setValueInternal(prefix + expectedKey, "value");
          +
          + DelegatingConfiguration configuration = new DelegatingConfiguration(backingConf, prefix);
          + Set<String> keySet = configuration.keySet();
          +
          — End diff –

          empty line

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74893573 — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java — @@ -88,4 +90,49 @@ private String typeParamToString(Class<?>[] classes) { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); } } + + @Test + public void testDelegationConfigurationWithNullPrefix() { + + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("test-key", "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration( + backingConf, null); + Set<String> keySet = configuration.keySet(); + + assertEquals(keySet, backingConf.keySet()); + + } + + @Test + public void testDelegationConfigurationWithPrefix() { + + String prefix = "pref-"; + String expectedKey = "key"; + + /* + * Key matches the prefix + */ + Configuration backingConf = new Configuration(); + backingConf.setValueInternal(prefix + expectedKey, "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration(backingConf, prefix); + Set<String> keySet = configuration.keySet(); + — End diff – empty line
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74893448

          — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java —
          @@ -178,14 +178,19 @@ public String toString() {

          @Override
          public Set<String> keySet() {
          +
          — End diff –

          We usually don't have an empty line when a method block starts (also applies to lines 185 and in `DelegatingConfigurationTest` lines 96 and 110).

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74893448 — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java — @@ -178,14 +178,19 @@ public String toString() { @Override public Set<String> keySet() { + — End diff – We usually don't have an empty line when a method block starts (also applies to lines 185 and in `DelegatingConfigurationTest` lines 96 and 110).
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tillrohrmann commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74893388

          — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java —
          @@ -178,14 +178,19 @@ public String toString() {

          @Override
          public Set<String> keySet() {
          +
          final HashSet<String> set = new HashSet<String>();

          • final int prefixLen = this.prefix == null ? 0 : this.prefix.length();

          for (String key : this.backingConfig.keySet()) {

          • if (key.startsWith(this.prefix)) {
            +
            + if (this.prefix == null) {
              • End diff –

          Can we move the condition out of the loop? The condition does not change over different iteration so we don't have to perform it every time.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74893388 — Diff: flink-core/src/main/java/org/apache/flink/configuration/DelegatingConfiguration.java — @@ -178,14 +178,19 @@ public String toString() { @Override public Set<String> keySet() { + final HashSet<String> set = new HashSet<String>(); final int prefixLen = this.prefix == null ? 0 : this.prefix.length(); for (String key : this.backingConfig.keySet()) { if (key.startsWith(this.prefix)) { + + if (this.prefix == null) { End diff – Can we move the condition out of the loop? The condition does not change over different iteration so we don't have to perform it every time.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74724570

          — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java —
          @@ -88,4 +90,49 @@ private String typeParamToString(Class<?>[] classes)

          { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); }

          }
          +
          + @Test
          + public void testDelegationConfigurationWithNullPrefix()

          { + + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("test-key", "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration( + backingConf, null); + Set<String> keySet = configuration.keySet(); + + assertEquals(keySet, backingConf.keySet()); + + }

          +
          + @Test
          + public void testDelegationConfigurationWithPrefix() {
          +
          + String prefix = "pref-";
          +
          + /*
          + * Key matches the prefix
          + */
          + Configuration backingConf = new Configuration();
          + backingConf.setValueInternal("pref-key", "value");
          — End diff –

          I think `"pref-"` can be replaced by the `prefix` variable, just to make it consistent.

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74724570 — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java — @@ -88,4 +90,49 @@ private String typeParamToString(Class<?>[] classes) { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); } } + + @Test + public void testDelegationConfigurationWithNullPrefix() { + + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("test-key", "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration( + backingConf, null); + Set<String> keySet = configuration.keySet(); + + assertEquals(keySet, backingConf.keySet()); + + } + + @Test + public void testDelegationConfigurationWithPrefix() { + + String prefix = "pref-"; + + /* + * Key matches the prefix + */ + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("pref-key", "value"); — End diff – I think `"pref-"` can be replaced by the `prefix` variable, just to make it consistent.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user wuchong commented on the issue:

          https://github.com/apache/flink/pull/2371

          LGTM +1

          Show
          githubbot ASF GitHub Bot added a comment - Github user wuchong commented on the issue: https://github.com/apache/flink/pull/2371 LGTM +1
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user wuchong commented on a diff in the pull request:

          https://github.com/apache/flink/pull/2371#discussion_r74720168

          — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java —
          @@ -88,4 +89,51 @@ private String typeParamToString(Class<?>[] classes)

          { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); }

          }
          +
          + @Test
          + public void testDelegationConfigurationWithNullPrefix() {
          +
          + Configuration backingConf = new Configuration();
          + backingConf.setValueInternal("test-key", "value");
          +
          + DelegatingConfiguration configuration = new DelegatingConfiguration(
          + backingConf, null);
          + Set<String> keySet = configuration.keySet();
          +
          + assertTrue(keySet.equals(backingConf.keySet()));
          — End diff –

          minor, it's better to use `assertEquals(keySet, backingConf.keySet())` here I think. Below is the same

          Show
          githubbot ASF GitHub Bot added a comment - Github user wuchong commented on a diff in the pull request: https://github.com/apache/flink/pull/2371#discussion_r74720168 — Diff: flink-core/src/test/java/org/apache/flink/configuration/DelegatingConfigurationTest.java — @@ -88,4 +89,51 @@ private String typeParamToString(Class<?>[] classes) { assertTrue("Foo method '" + configurationMethod.getName() + "' has not been wrapped correctly in DelegatingConfiguration wrapper", hasMethod); } } + + @Test + public void testDelegationConfigurationWithNullPrefix() { + + Configuration backingConf = new Configuration(); + backingConf.setValueInternal("test-key", "value"); + + DelegatingConfiguration configuration = new DelegatingConfiguration( + backingConf, null); + Set<String> keySet = configuration.keySet(); + + assertTrue(keySet.equals(backingConf.keySet())); — End diff – minor, it's better to use `assertEquals(keySet, backingConf.keySet())` here I think. Below is the same
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user tsunny opened a pull request:

          https://github.com/apache/flink/pull/2371

          FLINK-4309 Potential null pointer dereference in DelegatingCo…

          Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
          If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
          In addition to going through the list, please provide a meaningful description of your changes.

          • [ ] General
          • The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
          • The pull request addresses only one issue
          • Each commit in the PR has a meaningful commit message (including the JIRA id)
          • [ ] Documentation
          • Documentation has been added for new functionality
          • Old documentation affected by the pull request has been updated
          • JavaDoc for public methods has been added
          • [ ] Tests & Build
          • Functionality added by the pull request is covered by tests
          • `mvn clean verify` has been executed successfully locally or a Travis build has passed

          …nfiguration#keySet()

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/tsunny/flink FLINK-4309

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/2371.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2371


          commit 4de73e7c6a27fb854e43ea8a9e19b1b8a24b07bd
          Author: Sunny T <sunny@sunnys-macbook-pro-3.local>
          Date: 2016-08-14T22:50:08Z

          FLINK-4309 Fixed Potential null pointer dereference in DelegatingConfiguration#keySet()


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user tsunny opened a pull request: https://github.com/apache/flink/pull/2371 FLINK-4309 Potential null pointer dereference in DelegatingCo… Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration. If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide] ( http://flink.apache.org/how-to-contribute.html ). In addition to going through the list, please provide a meaningful description of your changes. [ ] General The pull request references the related JIRA issue (" [FLINK-XXX] Jira title text") The pull request addresses only one issue Each commit in the PR has a meaningful commit message (including the JIRA id) [ ] Documentation Documentation has been added for new functionality Old documentation affected by the pull request has been updated JavaDoc for public methods has been added [ ] Tests & Build Functionality added by the pull request is covered by tests `mvn clean verify` has been executed successfully locally or a Travis build has passed …nfiguration#keySet() You can merge this pull request into a Git repository by running: $ git pull https://github.com/tsunny/flink FLINK-4309 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2371.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2371 commit 4de73e7c6a27fb854e43ea8a9e19b1b8a24b07bd Author: Sunny T <sunny@sunnys-macbook-pro-3.local> Date: 2016-08-14T22:50:08Z FLINK-4309 Fixed Potential null pointer dereference in DelegatingConfiguration#keySet()
          Hide
          tsunny Sunny T added a comment -

          Hi,

          I would like to work on this issue. However, I am not able to assign the issue to myself. How can I assign this issue to myself?

          Thanks,
          Sunny

          Show
          tsunny Sunny T added a comment - Hi, I would like to work on this issue. However, I am not able to assign the issue to myself. How can I assign this issue to myself? Thanks, Sunny

            People

            • Assignee:
              tsunny Sunny T
              Reporter:
              yuzhihong@gmail.com Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development