Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6131

Add side inputs for DataStream API

Details

    Description

      This is an umbrella issue for tracking the implementation of FLIP-17: https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API.

      Attachments

        Issue Links

          Activity

            liurenjie1024 Renjie Liu added a comment -

            Hi, we are facing the same issues and want to solve it with community together. Could anyone assign this task to me?

            liurenjie1024 Renjie Liu added a comment - Hi, we are facing the same issues and want to solve it with community together. Could anyone assign this task to me?

            Hi team, has there been any progress on this?

            ashishpok@yahoo.com Ashish Pokharel added a comment - Hi team, has there been any progress on this?

            Unfortunately, there has not been any progress. It turns out that doing this properly might require more thinking. For example, the current design would use buffering of elements until side input is ready. This can lead to problems and it might be wise to instead block an input until data is ready. This does not work with the current network stack, however, since we cannot block on one input and not on the other without possibly creating a deadlock somewhere upstream.

            Working on the first two subtasks would still make sense, IMHO, but right now people are working on different things.

            aljoscha Aljoscha Krettek added a comment - Unfortunately, there has not been any progress. It turns out that doing this properly might require more thinking. For example, the current design would use buffering of elements until side input is ready. This can lead to problems and it might be wise to instead block an input until data is ready. This does not work with the current network stack, however, since we cannot block on one input and not on the other without possibly creating a deadlock somewhere upstream. Working on the first two subtasks would still make sense, IMHO, but right now people are working on different things.

            Thanks for the update. I agree that first two still makes sense.

            ashishpok@yahoo.com Ashish Pokharel added a comment - Thanks for the update. I agree that first two still makes sense.

            Do we have any progress on that one?

            igroeg Georgi Stoyanov added a comment - Do we have any progress on that one?
            trohrmann Till Rohrmann added a comment -

            There hasn't been much progress on this issue recently within the community igroeg.

            trohrmann Till Rohrmann added a comment - There hasn't been much progress on this issue recently within the community igroeg .

            All, looping back again on this. This feature seems very powerful to me but has lost traction. Wondering if this will ever make it back into roadmap.

            ashishpok@yahoo.com Ashish Pokharel added a comment - All, looping back again on this. This feature seems very powerful to me but has lost traction. Wondering if this will ever make it back into roadmap.
            fhueske Fabian Hueske added a comment -

            Hi ashishpok@yahoo.com, side inputs are actually mentioned in the roadmap.
            See last paragraph of the "Batch and Streaming Unification" section in the roadmap.

            fhueske Fabian Hueske added a comment - Hi ashishpok@yahoo.com , side inputs are actually mentioned in the roadmap. See last paragraph of the "Batch and Streaming Unification" section in the roadmap .

            Thanks Fabian! Looking forward to try out Blink integration as well ...

            ashishpok@yahoo.com Ashish Pokharel added a comment - Thanks Fabian! Looking forward to try out Blink integration as well ...
            amangill Amaninder Gill added a comment -

            Hi fhueske, I am also faced with the challenge of solving for slowly changing side inputs. After exhausted my options trying with KTables, GlobalKTables, Broadcast state and Table API, I am reaching out for help here. You mentioned that the feature is on the roadmap and I sincerely understand that it is not a small change but it would be really helpful if you can share any kind of timeline/targeted release. Thank you.

            amangill Amaninder Gill added a comment - Hi fhueske , I am also faced with the challenge of solving for slowly changing side inputs. After exhausted my options trying with KTables, GlobalKTables, Broadcast state and Table API, I am reaching out for help here. You mentioned that the feature is on the roadmap and I sincerely understand that it is not a small change but it would be really helpful if you can share any kind of timeline/targeted release. Thank you.
            flink-jira-bot Flink Jira Bot added a comment -

            This issue is assigned but has not received an update in 7 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.

            flink-jira-bot Flink Jira Bot added a comment - This issue is assigned but has not received an update in 7 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.
            flink-jira-bot Flink Jira Bot added a comment -

            This issue was marked "stale-assigned" and has not received an update in 7 days. It is now automatically unassigned. If you are still working on it, you can assign it to yourself again. Please also give an update about the status of the work.

            flink-jira-bot Flink Jira Bot added a comment - This issue was marked "stale-assigned" and has not received an update in 7 days. It is now automatically unassigned. If you are still working on it, you can assign it to yourself again. Please also give an update about the status of the work.
            flink-jira-bot Flink Jira Bot added a comment -

            I am the Flink Jira Bot and I help the community manage its development. I see this issues has been marked as Major but is unassigned and neither itself nor its Sub-Tasks have been updated for 30 days. I have gone ahead and added a "stale-major" to the issue". If this ticket is a Major, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized.

            flink-jira-bot Flink Jira Bot added a comment - I am the Flink Jira Bot and I help the community manage its development. I see this issues has been marked as Major but is unassigned and neither itself nor its Sub-Tasks have been updated for 30 days. I have gone ahead and added a "stale-major" to the issue". If this ticket is a Major, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized.
            flink-jira-bot Flink Jira Bot added a comment -

            This issue was labeled "stale-major" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Major, please raise the priority and ask a committer to assign you the issue or revive the public discussion.

            flink-jira-bot Flink Jira Bot added a comment - This issue was labeled "stale-major" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Major, please raise the priority and ask a committer to assign you the issue or revive the public discussion.
            flink-jira-bot Flink Jira Bot added a comment -

            I am the Flink Jira Bot and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized.

            flink-jira-bot Flink Jira Bot added a comment - I am the Flink Jira Bot and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized.
            flink-jira-bot Flink Jira Bot added a comment -

            This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion.

            flink-jira-bot Flink Jira Bot added a comment - This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion.
            Alcántara Salva added a comment -

            If nobody is working on this I could give it a try, assuming it's easy enough!

            Alcántara Salva added a comment - If nobody is working on this I could give it a try, assuming it's easy enough!

            People

              Unassigned Unassigned
              aljoscha Aljoscha Krettek
              Votes:
              20 Vote for this issue
              Watchers:
              44 Start watching this issue

              Dates

                Created:
                Updated: