Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-40

C++ Reader does not support predicate pushdown.

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: C++
    • Labels:
      None

      Description

      The Java reader can push down predicates to filter the data being read. The C++ reader doesn't do that yet.

        Activity

        Hide
        owen.omalley Owen O'Malley added a comment -

        The list of supported OSes for the C++ reader are:

        • Centos 6, and 7
        • Debian 7, and 8
        • Ubuntu 12, 14, and 16
        • MacOS 10.10, 10.11, 10.12

        There are docker scripts in the docker directory that let you build in each of the Linux variants, so that if you are running on Linux,

        % cd docker
        % ./run-all.sh omalley orc-104

        will build the orc-104 branch from my (omalley) github orc repo on all 7 Linux variants. If you replace omalley with your github id you can build from one of your branches. The oldest version of g++ that we are currently supporting is in centos 6 with gcc 4.4.7, which has shared_ptr support.

        Show
        owen.omalley Owen O'Malley added a comment - The list of supported OSes for the C++ reader are: Centos 6, and 7 Debian 7, and 8 Ubuntu 12, 14, and 16 MacOS 10.10, 10.11, 10.12 There are docker scripts in the docker directory that let you build in each of the Linux variants, so that if you are running on Linux, % cd docker % ./run-all.sh omalley orc-104 will build the orc-104 branch from my (omalley) github orc repo on all 7 Linux variants. If you replace omalley with your github id you can build from one of your branches. The oldest version of g++ that we are currently supporting is in centos 6 with gcc 4.4.7, which has shared_ptr support.
        Hide
        chunyang-wen Chunyang Wen added a comment - - edited

        Most of the APIs have been implemented using C++. But in order to reduce memory leak problem, I use std::shared_ptr which is supported in C++11.

        Whether C++ sources will only be compiled in compilers which support at C++11 (with option -std=c++11)?

        If so, I will use std::shared_ptr, otherwise I am considering boost::shared_ptr from boost (Actually I think shared_ptr comes from tr1 which originates from boost)

        Show
        chunyang-wen Chunyang Wen added a comment - - edited Most of the APIs have been implemented using C++. But in order to reduce memory leak problem, I use std::shared_ptr which is supported in C++11. Whether C++ sources will only be compiled in compilers which support at C++11 (with option -std=c++11)? If so, I will use std::shared_ptr, otherwise I am considering boost::shared_ptr from boost (Actually I think shared_ptr comes from tr1 which originates from boost)
        Hide
        chunyang-wen Chunyang Wen added a comment -

        It seems that we should first provide a similar C++ API. Then users can use that API to build search argument. Process of building search argument is application related. Good point.

        Show
        chunyang-wen Chunyang Wen added a comment - It seems that we should first provide a similar C++ API. Then users can use that API to build search argument. Process of building search argument is application related. Good point.
        Hide
        owen.omalley Owen O'Malley added a comment -

        Actually, in Java, you can build up the SearchArgument via the API. Look at the SearchArgumentFactory. Hive's code uses that API to build it up the SearchArgument from its AST, but that is layered on top.

        No one is currently working on it and it would be great to get it implemented using an equivalent API to the Java one.

        Show
        owen.omalley Owen O'Malley added a comment - Actually, in Java, you can build up the SearchArgument via the API. Look at the SearchArgumentFactory. Hive's code uses that API to build it up the SearchArgument from its AST, but that is layered on top. No one is currently working on it and it would be great to get it implemented using an equivalent API to the Java one.
        Hide
        chunyang-wen Chunyang Wen added a comment -

        Is there any plan or any thought about this issue?

        In java, search argument is built by traversing the abstract syntax tree which I think is generated by sql parser. But in C++, we use available APIs to read orc files, so there is no such thing (AST). Of course, we can use any method to describe filter conditions that users want. E.g., implementing a simple parser that can parse simple search arguments, >, <, + etc. But it seems that it is not elegant.

        Show
        chunyang-wen Chunyang Wen added a comment - Is there any plan or any thought about this issue? In java, search argument is built by traversing the abstract syntax tree which I think is generated by sql parser. But in C++, we use available APIs to read orc files, so there is no such thing (AST). Of course, we can use any method to describe filter conditions that users want. E.g., implementing a simple parser that can parse simple search arguments, >, <, + etc. But it seems that it is not elegant.

          People

          • Assignee:
            Unassigned
            Reporter:
            owen.omalley Owen O'Malley
          • Votes:
            4 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development