Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Storage
    • Labels:

      Description

      ORCFile was merged to Hive's trunk. We can also consider the integration of ORCFile for Tajo.

      1. orc_file.png
        49 kB
        Hyoungjun Kim

        Activity

        Hide
        jhkim Jinho Kim added a comment -

        The ORC file looks like a good design.
        I'll start this feature.
        https://issues.apache.org/jira/secure/attachment/12564124/OrcFileIntro.pptx

        Show
        jhkim Jinho Kim added a comment - The ORC file looks like a good design. I'll start this feature. https://issues.apache.org/jira/secure/attachment/12564124/OrcFileIntro.pptx
        Hide
        hyunsik Hyunsik Choi added a comment -

        I'm looking forward to this feature!

        Show
        hyunsik Hyunsik Choi added a comment - I'm looking forward to this feature!
        Hide
        jihoonson Jihoon Son added a comment -

        Hi, Jinho
        I'm investigating documents of ORCFile and source codes of Hive.
        If you don't start this issue yet, I'd like to do.

        Show
        jihoonson Jihoon Son added a comment - Hi, Jinho I'm investigating documents of ORCFile and source codes of Hive. If you don't start this issue yet, I'd like to do.
        Hide
        jhkim Jinho Kim added a comment -

        Jihoon,
        I've assigned this issue to you.

        Thanks.

        Show
        jhkim Jinho Kim added a comment - Jihoon, I've assigned this issue to you. Thanks.
        Hide
        jihoonson Jihoon Son added a comment -

        Thanks a lot!

        Show
        jihoonson Jihoon Son added a comment - Thanks a lot!
        Hide
        hjkim Hyoungjun Kim added a comment -

        I've attached that explains orc file detail.

        Show
        hjkim Hyoungjun Kim added a comment - I've attached that explains orc file detail.
        Hide
        jihoonson Jihoon Son added a comment -

        This figure looks great!
        It will be very helpful.
        Thanks, Hyoungjun.

        Show
        jihoonson Jihoon Son added a comment - This figure looks great! It will be very helpful. Thanks, Hyoungjun.
        Hide
        hyunsik Hyunsik Choi added a comment -

        Great work! Thank you for your sharing

        Show
        hyunsik Hyunsik Choi added a comment - Great work! Thank you for your sharing
        Hide
        dongjoon Dongjoon Hyun added a comment -

        Hi, Jihoon Son.

        This issue looks great, but it seems to be open for a long time.
        I think it's a tough issue.
        If you don't mind, may I assign me this issue and start processing?
        I think ORC is essential for TPC-DS.

        Warmly,
        Dongjoon

        Show
        dongjoon Dongjoon Hyun added a comment - Hi, Jihoon Son . This issue looks great, but it seems to be open for a long time. I think it's a tough issue. If you don't mind, may I assign me this issue and start processing? I think ORC is essential for TPC-DS. Warmly, Dongjoon
        Hide
        jihoonson Jihoon Son added a comment -

        I've assigned this issue to you.
        Thanks for your volunteer work!

        Show
        jihoonson Jihoon Son added a comment - I've assigned this issue to you. Thanks for your volunteer work!
        Hide
        dongjoon Dongjoon Hyun added a comment -

        Thank you, Jihoon! I see your re-assignment a few seconds ago. I will start in this afternoon.

        Show
        dongjoon Dongjoon Hyun added a comment - Thank you, Jihoon! I see your re-assignment a few seconds ago. I will start in this afternoon.
        Hide
        jihoonson Jihoon Son added a comment -

        Hi Dongjoon Hyun. Thank you for starting this issue.
        I'd like to advise on progressing an umbrella issue. As you already do, umbrella issues usually divided into several detailed sub-tasks each of which is related to only a part of the whole implementation. So, newly implemented functions in sub-tasks may incur misfunctioning until the whole tasks is finished. This can cause unexpected errors and might make users confused.
        To avoid this situation, you have three choices as follows.

        • Progressing sub-tasks in a separate branch. You can freely create a new branch, and put your implementation on that branch. When you finish all sub-tasks, your implementation will be merged into the master branch. Please refer to TAJO-1118.
        • Progressing sub-tasks in the master branch. If you choose this option, you must handle expected errors. For example, after TAJO-1463, the create table statement with ORC is ok, but the select statement should be prevented from execution.
        • Progressing an unified task rather than dividing into sub-takss. This option is not recommended when the patch size is large.
        Show
        jihoonson Jihoon Son added a comment - Hi Dongjoon Hyun . Thank you for starting this issue. I'd like to advise on progressing an umbrella issue. As you already do, umbrella issues usually divided into several detailed sub-tasks each of which is related to only a part of the whole implementation. So, newly implemented functions in sub-tasks may incur misfunctioning until the whole tasks is finished. This can cause unexpected errors and might make users confused. To avoid this situation, you have three choices as follows. Progressing sub-tasks in a separate branch. You can freely create a new branch, and put your implementation on that branch. When you finish all sub-tasks, your implementation will be merged into the master branch. Please refer to TAJO-1118 . Progressing sub-tasks in the master branch. If you choose this option, you must handle expected errors. For example, after TAJO-1463 , the create table statement with ORC is ok, but the select statement should be prevented from execution. Progressing an unified task rather than dividing into sub-takss. This option is not recommended when the patch size is large.
        Hide
        dongjoon Dongjoon Hyun added a comment -

        Thank you for your advice, Jihoon Son! The second choice you've given looks good to me.

        Up to now, I tested creating a new ORC table and selecting that empty table. Both cases are okay, so I made a pull request.
        However, your advice is right for TAJO-1463, I had better add error handling codes like throwing 'Not Implemented' exception for non-empty table. In addition, for TAJO-1464, I will also add error handling codes for insert statements into ORC tables.

        Thank you again!

        Show
        dongjoon Dongjoon Hyun added a comment - Thank you for your advice, Jihoon Son ! The second choice you've given looks good to me. Up to now, I tested creating a new ORC table and selecting that empty table. Both cases are okay, so I made a pull request. However, your advice is right for TAJO-1463 , I had better add error handling codes like throwing 'Not Implemented ' exception for non-empty table. In addition, for TAJO-1464 , I will also add error handling codes for insert statements into ORC tables. Thank you again!
        Hide
        eminency Jongyoung Park added a comment -

        Hi, Dongjoon Hyun.

        May I ask how much it is going?

        Show
        eminency Jongyoung Park added a comment - Hi, Dongjoon Hyun . May I ask how much it is going?
        Hide
        eminency Jongyoung Park added a comment -

        It looks no one is touching it, so it is assigned to me.

        Show
        eminency Jongyoung Park added a comment - It looks no one is touching it, so it is assigned to me.

          People

          • Assignee:
            eminency Jongyoung Park
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development