Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.9
    • Fix Version/s: None
    • Component/s: Table API & SQL
    • Labels:

      Description

      Add a ParquetTableSource to read data from a Apache Parquet file. The ParquetTableSource should implement the ProjectableTableSource (FLINK-3848) and FilterableTableSource (FLINK-3849) interfaces.

        Issue Links

          Activity

          Hide
          fhueske Fabian Hueske added a comment -

          Sure, let's continue at FLINK-3848.

          Show
          fhueske Fabian Hueske added a comment - Sure, let's continue at FLINK-3848 .
          Hide
          dawidwys Dawid Wysakowicz added a comment -

          Thanks for you answer Fabian Hueske. I would be eager to have a look at your branch and problems for FLINK-3848 if you don't mind. If I am unable to help, I will start working on FLINK-3872/FLINK-3871. Is that ok with you?

          Show
          dawidwys Dawid Wysakowicz added a comment - Thanks for you answer Fabian Hueske . I would be eager to have a look at your branch and problems for FLINK-3848 if you don't mind. If I am unable to help, I will start working on FLINK-3872 / FLINK-3871 . Is that ok with you?
          Hide
          fhueske Fabian Hueske added a comment -

          Hi Dawid Wysakowicz, I started with FLINK-3848 but it turned out to be more complex than I anticipated. I haven't found a good way to integrate the ProjectableTableSource with Calcite's planner. I expect FLINK-3849 to be similar.

          If you'd like I can share my branch for FLINK-3848 and explain the problems. Alternatively, you could also try to start with FLINK-3871 or FLINK-3872 which do not depend on FLINK-3848 and FLINK-3849.

          Show
          fhueske Fabian Hueske added a comment - Hi Dawid Wysakowicz , I started with FLINK-3848 but it turned out to be more complex than I anticipated. I haven't found a good way to integrate the ProjectableTableSource with Calcite's planner. I expect FLINK-3849 to be similar. If you'd like I can share my branch for FLINK-3848 and explain the problems. Alternatively, you could also try to start with FLINK-3871 or FLINK-3872 which do not depend on FLINK-3848 and FLINK-3849 .
          Hide
          dawidwys Dawid Wysakowicz added a comment -

          I don't think anybody is working on this one right now. If I am correct I would gladly take over or if possible I could help with FLINK-3848 or FLINK-3849. Fabian Hueske what do you think?

          Show
          dawidwys Dawid Wysakowicz added a comment - I don't think anybody is working on this one right now. If I am correct I would gladly take over or if possible I could help with FLINK-3848 or FLINK-3849 . Fabian Hueske what do you think?
          Hide
          fhueske Fabian Hueske added a comment -

          Updated the issue title and description to reflect the new TableSource interface.

          Show
          fhueske Fabian Hueske added a comment - Updated the issue title and description to reflect the new TableSource interface.
          Hide
          rerngvit rerngvit yanggratoke added a comment -

          Aljoscha Krettek Thanks for letting me know. I will look for something else then.

          Show
          rerngvit rerngvit yanggratoke added a comment - Aljoscha Krettek Thanks for letting me know. I will look for something else then.
          Hide
          aljoscha Aljoscha Krettek added a comment -

          There is currently an open PR (https://github.com/apache/flink/pull/1127) by Timo Walther that adds fromHCat to TableEnvironment along with all the required plumbing. So maybe you should wait for him, or at least coordinate with him.

          Show
          aljoscha Aljoscha Krettek added a comment - There is currently an open PR ( https://github.com/apache/flink/pull/1127 ) by Timo Walther that adds fromHCat to TableEnvironment along with all the required plumbing. So maybe you should wait for him, or at least coordinate with him.
          Hide
          rerngvit rerngvit yanggratoke added a comment -

          Fabian Hueske I have a question. The method that should be added is "fromCsvFile()" as in the description or "fromParquet()" as in the title?

          Show
          rerngvit rerngvit yanggratoke added a comment - Fabian Hueske I have a question. The method that should be added is "fromCsvFile()" as in the description or "fromParquet()" as in the title?

            People

            • Assignee:
              godfreyhe godfrey he
              Reporter:
              fhueske Fabian Hueske
            • Votes:
              3 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:

                Development