Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: Storage - Other
    • Labels:
      None

      Description

      Merge the work done here into Drill master so others can utilize the plugin: https://github.com/dremio/drill-storage-kudu

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jacques-n commented on the pull request:

        https://github.com/apache/drill/pull/314#issuecomment-185738887

        The Kudu plugin was contributed by six different developers, three of which are Drill PMC members and two more which are PMC members of other Apache projects. The code remained available for five days for review and received two plus ones and no negative feedback. It is modeled after the HBase plugin and works the same. It is unfortunate that there aren't integrated tests (due to the fact that there wasn't an easy way to provide integrated tests such as mini hbase cluster) but it was and is regularly manually tested. Due to the light testing, we are communicating it as experimental to users.

        Suggesting it didn't go through review when you have that large a group of developers involved is weird. Assuming that the user api isn't in dispute (which no one here disputed most likely because Kudu looks exactly like an Oracle table), providing experimental plugins increases the breadth of Drill's appeal and thus broadens and strengthens the community.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jacques-n commented on the pull request: https://github.com/apache/drill/pull/314#issuecomment-185738887 The Kudu plugin was contributed by six different developers, three of which are Drill PMC members and two more which are PMC members of other Apache projects. The code remained available for five days for review and received two plus ones and no negative feedback. It is modeled after the HBase plugin and works the same. It is unfortunate that there aren't integrated tests (due to the fact that there wasn't an easy way to provide integrated tests such as mini hbase cluster) but it was and is regularly manually tested. Due to the light testing, we are communicating it as experimental to users. Suggesting it didn't go through review when you have that large a group of developers involved is weird. Assuming that the user api isn't in dispute (which no one here disputed most likely because Kudu looks exactly like an Oracle table), providing experimental plugins increases the breadth of Drill's appeal and thus broadens and strengthens the community.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user tdunning commented on the pull request:

        https://github.com/apache/drill/pull/314#issuecomment-185636190

        How could this have been merged? There is a huge double standard going on here.

        This code has NO comments. No tests. No documentation. No design. It isn't nearly good enough to pass the reviews that are required for others to contribute code.

        How can it be merged without any kind of significant review?

        Show
        githubbot ASF GitHub Bot added a comment - Github user tdunning commented on the pull request: https://github.com/apache/drill/pull/314#issuecomment-185636190 How could this have been merged? There is a huge double standard going on here. This code has NO comments. No tests. No documentation. No design. It isn't nearly good enough to pass the reviews that are required for others to contribute code. How can it be merged without any kind of significant review?
        Hide
        jnadeau Jacques Nadeau added a comment -

        Merged in 392d1f7e9398fb1bee8f67b0d49a82436c3145fb

        Show
        jnadeau Jacques Nadeau added a comment - Merged in 392d1f7e9398fb1bee8f67b0d49a82436c3145fb
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/drill/pull/314

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/314
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jaltekruse commented on the pull request:

        https://github.com/apache/drill/pull/314#issuecomment-168730376

        I agree with Steven, at the hackathon we already did some basic performance and correctness verification at reasonable scale. +1

        Show
        githubbot ASF GitHub Bot added a comment - Github user jaltekruse commented on the pull request: https://github.com/apache/drill/pull/314#issuecomment-168730376 I agree with Steven, at the hackathon we already did some basic performance and correctness verification at reasonable scale. +1
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user StevenMPhillips commented on the pull request:

        https://github.com/apache/drill/pull/314#issuecomment-168588400

        +1

        As long as this doesn't break the build, I say we go ahead and merge it. We can add tests and fixes later.

        Show
        githubbot ASF GitHub Bot added a comment - Github user StevenMPhillips commented on the pull request: https://github.com/apache/drill/pull/314#issuecomment-168588400 +1 As long as this doesn't break the build, I say we go ahead and merge it. We can add tests and fixes later.
        Hide
        jnadeau Jacques Nadeau added a comment -

        Jason Altekruse, can you please review?

        Show
        jnadeau Jacques Nadeau added a comment - Jason Altekruse , can you please review?
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jacques-n opened a pull request:

        https://github.com/apache/drill/pull/314

        DRILL-4241: Add Kudu reader

        Implements an experimental Kudu reader & writer.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jacques-n/drill DRILL-4241

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/drill/pull/314.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #314


        commit bc9da49f93b90507c94a6eab3f83187b44d1d213
        Author: Jacques Nadeau <jacques.drill@gmail.com>
        Date: 2015-11-19T02:45:56Z

        DRILL-4241: initial commit

        commit dc9c0d097ab7cfd1b722a9065509a5167155a3b1
        Author: Todd Lipcon <todd@cloudera.com>
        Date: 2015-11-19T22:15:58Z

        DRILL-4241: Add pushdown of column projections

        commit 6fe5b352de522670b77985334d7b1af0f15c7467
        Author: Jacques Nadeau <jacques@apache.org>
        Date: 2015-11-19T22:21:14Z

        DRILL-4241: Add table metadata and DROP table support

        commit 756b5dfb5e2de3b5c45b5943451d7e73ab0b65e6
        Author: Todd Lipcon <todd@cloudera.com>
        Date: 2015-11-19T23:00:00Z

        DRILL-4241: Improve record reader and type mappings

        commit 4765e2d499510a844e24b33782b0402d6ea640d7
        Author: Steven Phillips <smp@apache.org>
        Date: 2015-11-20T00:00:40Z

        DRILL-4241: Add parallelization and assignment

        commit 4e8517898ed966f08ac8c0b1b20f7a9b08d2aae3
        Author: Todd Lipcon <todd@cloudera.com>
        Date: 2015-11-20T02:56:52Z

        DRILL-4241: Various Fixes

        • ReaderFix scanning of tables with >4096 rows
        • Fixing the build, make RAT and checkstyle happy.
        • Fix last batch of query not to NPE

        commit a7ff45079379dda2fb49bff71e1b12e4e37c7dda
        Author: Todd Lipcon <todd@cloudera.com>
        Date: 2015-11-20T00:15:36Z

        DRILL-4241: Rewrite RecordReader to support NULLs and be less Java-like

        commit 5f31e7a0a6e57db4d71214b52413b2e5937f545a
        Author: Amit Hadke <amit.hadke@gmail.com>
        Date: 2015-11-20T00:16:17Z

        DRILL-4241: Create table with multiple tablets in kudu test.

        commit 48a988a4ef7cd85dfd8dbf8f348b8fc9c9185ce7
        Author: Todd Lipcon <todd@cloudera.com>
        Date: 2015-11-20T00:19:45Z

        DRILL-4241: Timestamps should divide by 1000 in reader

        commit 77921147bcc2d9ebd6dac3241109c90337440868
        Author: Steven Phillips <smp@apache.org>
        Date: 2015-11-20T00:54:41Z

        DRILL-4241: Add wait stats and estimated row count in RecordReader

        commit 64c8e99a05f69af6d6118531d2d9081e843d1b0f
        Author: Jacques Nadeau <jacques@apache.org>
        Date: 2015-11-20T02:20:03Z

        DRILL-4241: Add Single Tablet Writer

        • Also move to a test bootstrap
        • Update to the latest kudu and Drill
        • Add plugin to Drill distribution
        • Checkstyle and directory cleanup

        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jacques-n opened a pull request: https://github.com/apache/drill/pull/314 DRILL-4241 : Add Kudu reader Implements an experimental Kudu reader & writer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacques-n/drill DRILL-4241 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/314.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #314 commit bc9da49f93b90507c94a6eab3f83187b44d1d213 Author: Jacques Nadeau <jacques.drill@gmail.com> Date: 2015-11-19T02:45:56Z DRILL-4241 : initial commit commit dc9c0d097ab7cfd1b722a9065509a5167155a3b1 Author: Todd Lipcon <todd@cloudera.com> Date: 2015-11-19T22:15:58Z DRILL-4241 : Add pushdown of column projections commit 6fe5b352de522670b77985334d7b1af0f15c7467 Author: Jacques Nadeau <jacques@apache.org> Date: 2015-11-19T22:21:14Z DRILL-4241 : Add table metadata and DROP table support commit 756b5dfb5e2de3b5c45b5943451d7e73ab0b65e6 Author: Todd Lipcon <todd@cloudera.com> Date: 2015-11-19T23:00:00Z DRILL-4241 : Improve record reader and type mappings commit 4765e2d499510a844e24b33782b0402d6ea640d7 Author: Steven Phillips <smp@apache.org> Date: 2015-11-20T00:00:40Z DRILL-4241 : Add parallelization and assignment commit 4e8517898ed966f08ac8c0b1b20f7a9b08d2aae3 Author: Todd Lipcon <todd@cloudera.com> Date: 2015-11-20T02:56:52Z DRILL-4241 : Various Fixes ReaderFix scanning of tables with >4096 rows Fixing the build, make RAT and checkstyle happy. Fix last batch of query not to NPE commit a7ff45079379dda2fb49bff71e1b12e4e37c7dda Author: Todd Lipcon <todd@cloudera.com> Date: 2015-11-20T00:15:36Z DRILL-4241 : Rewrite RecordReader to support NULLs and be less Java-like commit 5f31e7a0a6e57db4d71214b52413b2e5937f545a Author: Amit Hadke <amit.hadke@gmail.com> Date: 2015-11-20T00:16:17Z DRILL-4241 : Create table with multiple tablets in kudu test. commit 48a988a4ef7cd85dfd8dbf8f348b8fc9c9185ce7 Author: Todd Lipcon <todd@cloudera.com> Date: 2015-11-20T00:19:45Z DRILL-4241 : Timestamps should divide by 1000 in reader commit 77921147bcc2d9ebd6dac3241109c90337440868 Author: Steven Phillips <smp@apache.org> Date: 2015-11-20T00:54:41Z DRILL-4241 : Add wait stats and estimated row count in RecordReader commit 64c8e99a05f69af6d6118531d2d9081e843d1b0f Author: Jacques Nadeau <jacques@apache.org> Date: 2015-11-20T02:20:03Z DRILL-4241 : Add Single Tablet Writer Also move to a test bootstrap Update to the latest kudu and Drill Add plugin to Drill distribution Checkstyle and directory cleanup

          People

          • Assignee:
            jnadeau Jacques Nadeau
            Reporter:
            jnadeau Jacques Nadeau
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development