Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6439

Unclosed InputStream in OperatorSnapshotUtil#readStateHandle()

    Details

      Description

          FileInputStream in = new FileInputStream(path);
          DataInputStream dis = new DataInputStream(in);
      

      None of the in / dis is closed upon return from the method.

      In writeStateHandle(), OutputStream should be closed in finally block.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user fanyon opened a pull request:

          https://github.com/apache/flink/pull/3819

          FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil

          …tUtil

          Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
          If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
          In addition to going through the list, please provide a meaningful description of your changes.

          • [ ] General
          • The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
          • The pull request addresses only one issue
          • Each commit in the PR has a meaningful commit message (including the JIRA id)
          • [ ] Documentation
          • Documentation has been added for new functionality
          • Old documentation affected by the pull request has been updated
          • JavaDoc for public methods has been added
          • [ ] Tests & Build
          • Functionality added by the pull request is covered by tests
          • `mvn clean verify` has been executed successfully locally or a Travis build has passed

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/fanyon/flink FLINK-6439

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3819.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3819


          commit de723c7eb9ad9b8c8c27d6b884d47faa7d324dfd
          Author: mengji.fy <mengji.fy@taobao.com>
          Date: 2017-05-04T05:37:01Z

          FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user fanyon opened a pull request: https://github.com/apache/flink/pull/3819 FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil …tUtil Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration. If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide] ( http://flink.apache.org/how-to-contribute.html ). In addition to going through the list, please provide a meaningful description of your changes. [ ] General The pull request references the related JIRA issue (" [FLINK-XXX] Jira title text") The pull request addresses only one issue Each commit in the PR has a meaningful commit message (including the JIRA id) [ ] Documentation Documentation has been added for new functionality Old documentation affected by the pull request has been updated JavaDoc for public methods has been added [ ] Tests & Build Functionality added by the pull request is covered by tests `mvn clean verify` has been executed successfully locally or a Travis build has passed You can merge this pull request into a Git repository by running: $ git pull https://github.com/fanyon/flink FLINK-6439 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3819.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3819 commit de723c7eb9ad9b8c8c27d6b884d47faa7d324dfd Author: mengji.fy <mengji.fy@taobao.com> Date: 2017-05-04T05:37:01Z FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3819

          You git settings appear to not be in sync with your github account credentials; contributions will not be attributed correctly to the account.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3819 You git settings appear to not be in sync with your github account credentials; contributions will not be attributed correctly to the account.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zjureel opened a pull request:

          https://github.com/apache/flink/pull/3904

          FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil

          …tUtil

          Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
          If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
          In addition to going through the list, please provide a meaningful description of your changes.

          • [ ] General
          • The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
          • The pull request addresses only one issue
          • Each commit in the PR has a meaningful commit message (including the JIRA id)
          • [ ] Documentation
          • Documentation has been added for new functionality
          • Old documentation affected by the pull request has been updated
          • JavaDoc for public methods has been added
          • [ ] Tests & Build
          • Functionality added by the pull request is covered by tests
          • `mvn clean verify` has been executed successfully locally or a Travis build has passed

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zjureel/flink FLINK-6439

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3904.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3904


          commit 85fc9997d18bd6693b468a4cca9e495da6303d63
          Author: zjureel <zjureel@gmail.com>
          Date: 2017-05-15T10:14:11Z

          FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zjureel opened a pull request: https://github.com/apache/flink/pull/3904 FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil …tUtil Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration. If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide] ( http://flink.apache.org/how-to-contribute.html ). In addition to going through the list, please provide a meaningful description of your changes. [ ] General The pull request references the related JIRA issue (" [FLINK-XXX] Jira title text") The pull request addresses only one issue Each commit in the PR has a meaningful commit message (including the JIRA id) [ ] Documentation Documentation has been added for new functionality Old documentation affected by the pull request has been updated JavaDoc for public methods has been added [ ] Tests & Build Functionality added by the pull request is covered by tests `mvn clean verify` has been executed successfully locally or a Travis build has passed You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjureel/flink FLINK-6439 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3904.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3904 commit 85fc9997d18bd6693b468a4cca9e495da6303d63 Author: zjureel <zjureel@gmail.com> Date: 2017-05-15T10:14:11Z FLINK-6439 Fix close OutputStream && InputStream in OperatorSnapshotUtil
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjureel commented on the issue:

          https://github.com/apache/flink/pull/3904

          @zentol I have fix the issue 6439 in this PR, could you please review the code here when you are free. Thaks

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjureel commented on the issue: https://github.com/apache/flink/pull/3904 @zentol I have fix the issue 6439 in this PR, could you please review the code here when you are free. Thaks
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fanyon commented on the issue:

          https://github.com/apache/flink/pull/3819

          Use https://github.com/apache/flink/pull/3904(url) instead

          Show
          githubbot ASF GitHub Bot added a comment - Github user fanyon commented on the issue: https://github.com/apache/flink/pull/3819 Use https://github.com/apache/flink/pull/3904 (url) instead
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fanyon closed the pull request at:

          https://github.com/apache/flink/pull/3819

          Show
          githubbot ASF GitHub Bot added a comment - Github user fanyon closed the pull request at: https://github.com/apache/flink/pull/3819
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3904#discussion_r116483406

          — Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/util/OperatorSnapshotUtil.java —
          @@ -46,111 +46,113 @@ public static String getResourceFilename(String filename) {

          public static void writeStateHandle(OperatorStateHandles state, String path) throws IOException {
          FileOutputStream out = new FileOutputStream(path);

          • DataOutputStream dos = new DataOutputStream(out);
            -
          • dos.writeInt(state.getOperatorChainIndex());
            -
          • SavepointV1Serializer.serializeStreamStateHandle(state.getLegacyOperatorState(), dos);
            -
          • Collection<OperatorStateHandle> rawOperatorState = state.getRawOperatorState();
          • if (rawOperatorState != null) {
          • dos.writeInt(rawOperatorState.size());
          • for (OperatorStateHandle operatorStateHandle : rawOperatorState) {
          • SavepointV1Serializer.serializeOperatorStateHandle(operatorStateHandle, dos);
            +
            + try(DataOutputStream dos = new DataOutputStream(out)) {
              • End diff –

          missing space after try

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/3904#discussion_r116483406 — Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/util/OperatorSnapshotUtil.java — @@ -46,111 +46,113 @@ public static String getResourceFilename(String filename) { public static void writeStateHandle(OperatorStateHandles state, String path) throws IOException { FileOutputStream out = new FileOutputStream(path); DataOutputStream dos = new DataOutputStream(out); - dos.writeInt(state.getOperatorChainIndex()); - SavepointV1Serializer.serializeStreamStateHandle(state.getLegacyOperatorState(), dos); - Collection<OperatorStateHandle> rawOperatorState = state.getRawOperatorState(); if (rawOperatorState != null) { dos.writeInt(rawOperatorState.size()); for (OperatorStateHandle operatorStateHandle : rawOperatorState) { SavepointV1Serializer.serializeOperatorStateHandle(operatorStateHandle, dos); + + try(DataOutputStream dos = new DataOutputStream(out)) { End diff – missing space after try
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3904#discussion_r116483708

          — Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/util/OperatorSnapshotUtil.java —
          @@ -46,111 +46,113 @@ public static String getResourceFilename(String filename) {

          public static void writeStateHandle(OperatorStateHandles state, String path) throws IOException {
          FileOutputStream out = new FileOutputStream(path);

          • DataOutputStream dos = new DataOutputStream(out);
            -
          • dos.writeInt(state.getOperatorChainIndex());
            -
          • SavepointV1Serializer.serializeStreamStateHandle(state.getLegacyOperatorState(), dos);
            -
          • Collection<OperatorStateHandle> rawOperatorState = state.getRawOperatorState();
          • if (rawOperatorState != null) {
          • dos.writeInt(rawOperatorState.size());
          • for (OperatorStateHandle operatorStateHandle : rawOperatorState) {
          • SavepointV1Serializer.serializeOperatorStateHandle(operatorStateHandle, dos);
            +
            + try(DataOutputStream dos = new DataOutputStream(out)) {
            +
            + dos.writeInt(state.getOperatorChainIndex());
            +
            + SavepointV1Serializer.serializeStreamStateHandle(state.getLegacyOperatorState(), dos);
            +
            + Collection<OperatorStateHandle> rawOperatorState = state.getRawOperatorState();
            + if (rawOperatorState != null)
            Unknown macro: { + dos.writeInt(rawOperatorState.size()); + for (OperatorStateHandle operatorStateHandle }

            else

            { + // this means no states, not even an empty list + dos.writeInt(-1); }
          • } else { - // this means no states, not even an empty list - dos.writeInt(-1); - }
          • Collection<KeyedStateHandle> rawKeyedState = state.getRawKeyedState();
          • if (rawKeyedState != null) {
          • dos.writeInt(rawKeyedState.size());
          • for (KeyedStateHandle keyedStateHandle : rawKeyedState) {
          • SavepointV1Serializer.serializeKeyedStateHandle(keyedStateHandle, dos);
            + Collection<KeyedStateHandle> rawKeyedState = state.getRawKeyedState();
            + if (rawKeyedState != null)
            Unknown macro: { + dos.writeInt(rawKeyedState.size()); + for (KeyedStateHandle keyedStateHandle }

            else

            { + // this means no operator states, not even an empty list + dos.writeInt(-1); }
          • } else { - // this means no operator states, not even an empty list - dos.writeInt(-1); - }
          • dos.flush();
          • out.close();
            + dos.flush();
            + }
            }

          public static OperatorStateHandles readStateHandle(String path) throws IOException, ClassNotFoundException {
          FileInputStream in = new FileInputStream(path);

          • DataInputStream dis = new DataInputStream(in);
          • int index = dis.readInt();
            + try(DataInputStream dis = new DataInputStream(in)) {
              • End diff –

          missing space after try.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/3904#discussion_r116483708 — Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/util/OperatorSnapshotUtil.java — @@ -46,111 +46,113 @@ public static String getResourceFilename(String filename) { public static void writeStateHandle(OperatorStateHandles state, String path) throws IOException { FileOutputStream out = new FileOutputStream(path); DataOutputStream dos = new DataOutputStream(out); - dos.writeInt(state.getOperatorChainIndex()); - SavepointV1Serializer.serializeStreamStateHandle(state.getLegacyOperatorState(), dos); - Collection<OperatorStateHandle> rawOperatorState = state.getRawOperatorState(); if (rawOperatorState != null) { dos.writeInt(rawOperatorState.size()); for (OperatorStateHandle operatorStateHandle : rawOperatorState) { SavepointV1Serializer.serializeOperatorStateHandle(operatorStateHandle, dos); + + try(DataOutputStream dos = new DataOutputStream(out)) { + + dos.writeInt(state.getOperatorChainIndex()); + + SavepointV1Serializer.serializeStreamStateHandle(state.getLegacyOperatorState(), dos); + + Collection<OperatorStateHandle> rawOperatorState = state.getRawOperatorState(); + if (rawOperatorState != null) Unknown macro: { + dos.writeInt(rawOperatorState.size()); + for (OperatorStateHandle operatorStateHandle } else { + // this means no states, not even an empty list + dos.writeInt(-1); } } else { - // this means no states, not even an empty list - dos.writeInt(-1); - } Collection<KeyedStateHandle> rawKeyedState = state.getRawKeyedState(); if (rawKeyedState != null) { dos.writeInt(rawKeyedState.size()); for (KeyedStateHandle keyedStateHandle : rawKeyedState) { SavepointV1Serializer.serializeKeyedStateHandle(keyedStateHandle, dos); + Collection<KeyedStateHandle> rawKeyedState = state.getRawKeyedState(); + if (rawKeyedState != null) Unknown macro: { + dos.writeInt(rawKeyedState.size()); + for (KeyedStateHandle keyedStateHandle } else { + // this means no operator states, not even an empty list + dos.writeInt(-1); } } else { - // this means no operator states, not even an empty list - dos.writeInt(-1); - } dos.flush(); out.close(); + dos.flush(); + } } public static OperatorStateHandles readStateHandle(String path) throws IOException, ClassNotFoundException { FileInputStream in = new FileInputStream(path); DataInputStream dis = new DataInputStream(in); int index = dis.readInt(); + try(DataInputStream dis = new DataInputStream(in)) { End diff – missing space after try.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjureel commented on the issue:

          https://github.com/apache/flink/pull/3904

          @zentol Thank you for your reply, I have update the code and add space after `try`

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjureel commented on the issue: https://github.com/apache/flink/pull/3904 @zentol Thank you for your reply, I have update the code and add space after `try`
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3904

          merging.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3904 merging.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3904

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3904
          Hide
          Zentol Chesnay Schepler added a comment -

          1.3: 5fde739fd2b040a90d42a6a73f1d119648e863a7
          1.4: 65fdadac805cb1efe30ff9a57605676b1b8e45b9

          Show
          Zentol Chesnay Schepler added a comment - 1.3: 5fde739fd2b040a90d42a6a73f1d119648e863a7 1.4: 65fdadac805cb1efe30ff9a57605676b1b8e45b9

            People

            • Assignee:
              zjureel Fang Yong
              Reporter:
              yuzhihong@gmail.com Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development