Tajo
  1. Tajo
  2. TAJO-36

Improve ExternalSortExec with N-merge sort and final pass omission

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Physical Operator
    • Labels:
      None

      Description

      Background:

      The current ExternalSortExec just uses the binary external merge sort algorithm (http://en.wikipedia.org/wiki/External_sorting#External_merge_sort). In other words, for each pass, ExternalSortExec just merges two files into one sorted file.

      Proposal:
      The goal of this proposal is to improve ExternalSortExec with the following improvements:

      • N-merge sort - we can merge N files though more memory at each pass. It will reduce the number of passes. Consequently, it will reduces considerable I/O overheads.
      • the final pass omission - a physical operator is pipelined by the parent operator. The final pass of the merge sort must also be invoked by the parent physical operator. So, we can omit the final pass of the merge sort.
      1. TAJO-36_final.patch
        102 kB
        Hyunsik Choi
      2. TAJO-36_20140205_00:21:44.patch
        103 kB
        Hyunsik Choi
      3. TAJO-36_140204_172847.patch
        102 kB
        Hyunsik Choi
      4. TAJO-36_140204_163419.patch
        100 kB
        Hyunsik Choi
      5. TAJO-36.patch
        100 kB
        Hyunsik Choi

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          291d 21h 14m 1 Hyunsik Choi 04/Feb/14 05:27
          In Progress In Progress Patch Available Patch Available
          1h 40m 1 Hyunsik Choi 04/Feb/14 07:08
          Patch Available Patch Available Resolved Resolved
          14h 2m 1 Hyunsik Choi 04/Feb/14 21:10
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #55 (See https://builds.apache.org/job/Tajo-master-build/55/)
          TAJO-36: Improve ExternalSortExec with N-merge sort and final pass omission. (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=5177dcfa4b44e953919f47b94d39f9c5f7afb38b)

          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashSemiJoinExec.java
          • CHANGES.txt
          • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
          • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/UnaryPhysicalExec.java
          • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestPhysicalPlanner.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/eval/ExprTestBase.java
          • tajo-common/src/main/java/org/apache/tajo/datum/Float8Datum.java
          • tajo-storage/src/test/java/org/apache/tajo/storage/v2/TestStorages.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/RowFile.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/worker/TestRangeRetrieverHandler.java
          • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanningException.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/RawFile.java
          • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalExec.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashJoinExec.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/LazyTuple.java
          • tajo-common/src/main/java/org/apache/tajo/util/ClassSize.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/MemoryUtil.java
          • tajo-common/src/main/java/org/apache/tajo/util/CommonTestingUtil.java
          • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/MetaDataTuple.java
          • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestExternalSortExec.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/Tuple.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/VTuple.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestBNLJoinExec.java
          • tajo-storage/src/main/java/org/apache/tajo/storage/FrameTuple.java
          • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java
          • tajo-storage/src/test/java/org/apache/tajo/storage/TestVTuple.java
          • tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/storage/Tuple.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashAntiJoinExec.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestNLJoinExec.java
          • tajo-storage/src/test/java/org/apache/tajo/storage/TestLazyTuple.java
          • tajo-storage/src/test/java/org/apache/tajo/storage/TestStorages.java
          • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestMergeJoinExec.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #55 (See https://builds.apache.org/job/Tajo-master-build/55/ ) TAJO-36 : Improve ExternalSortExec with N-merge sort and final pass omission. (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=5177dcfa4b44e953919f47b94d39f9c5f7afb38b ) tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashSemiJoinExec.java CHANGES.txt tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/UnaryPhysicalExec.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestPhysicalPlanner.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/eval/ExprTestBase.java tajo-common/src/main/java/org/apache/tajo/datum/Float8Datum.java tajo-storage/src/test/java/org/apache/tajo/storage/v2/TestStorages.java tajo-storage/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-storage/src/main/java/org/apache/tajo/storage/RowFile.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/worker/TestRangeRetrieverHandler.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanningException.java tajo-storage/src/main/java/org/apache/tajo/storage/RawFile.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalExec.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashJoinExec.java tajo-storage/src/main/java/org/apache/tajo/storage/LazyTuple.java tajo-common/src/main/java/org/apache/tajo/util/ClassSize.java tajo-storage/src/main/java/org/apache/tajo/storage/MemoryUtil.java tajo-common/src/main/java/org/apache/tajo/util/CommonTestingUtil.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/MetaDataTuple.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestExternalSortExec.java tajo-storage/src/main/java/org/apache/tajo/storage/Tuple.java tajo-storage/src/main/java/org/apache/tajo/storage/VTuple.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestBNLJoinExec.java tajo-storage/src/main/java/org/apache/tajo/storage/FrameTuple.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java tajo-storage/src/test/java/org/apache/tajo/storage/TestVTuple.java tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/storage/Tuple.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashAntiJoinExec.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestNLJoinExec.java tajo-storage/src/test/java/org/apache/tajo/storage/TestLazyTuple.java tajo-storage/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestMergeJoinExec.java
          Hyunsik Choi made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hyunsik Choi added a comment -

          This issue got +1 on RB. committed.

          Show
          Hyunsik Choi added a comment - This issue got +1 on RB. committed.
          Hyunsik Choi made changes -
          Comment [ Updated the review request against branch master in reviewboard
          ]
          Hyunsik Choi made changes -
          Attachment TAJO-36_final.patch [ 12626885 ]
          Hide
          Hyunsik Choi added a comment -

          I fixed the latest comment on RB.

          Show
          Hyunsik Choi added a comment - I fixed the latest comment on RB.
          Hyunsik Choi made changes -
          Attachment TAJO-36_20140205_00:21:44.patch [ 12626884 ]
          Hyunsik Choi made changes -
          Link This issue is depended upon by TAJO-584 [ TAJO-584 ]
          Hide
          Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12626845/TAJO-36_140204_172847.patch
          against master revision 51919cf.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 14 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 212 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-common tajo-core/tajo-core-backend tajo-core/tajo-core-pullserver tajo-jdbc tajo-storage.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/102//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-pullserver.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/102//console

          This message is automatically generated.

          Show
          Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12626845/TAJO-36_140204_172847.patch against master revision 51919cf. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 212 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-common tajo-core/tajo-core-backend tajo-core/tajo-core-pullserver tajo-jdbc tajo-storage. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/102//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-pullserver.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/102//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/102//console This message is automatically generated.
          Hide
          Hyunsik Choi added a comment -

          Updated Tuple.java in pullserver.

          Updated the review request against branch master in reviewboard
          https://reviews.apache.org/r/17693/

          Show
          Hyunsik Choi added a comment - Updated Tuple.java in pullserver. Updated the review request against branch master in reviewboard https://reviews.apache.org/r/17693/
          Hyunsik Choi made changes -
          Attachment TAJO-36_140204_172847.patch [ 12626845 ]
          Hide
          Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12626839/TAJO-36_140204_163419.patch
          against master revision 0c0111c.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 14 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 222 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-common tajo-core/tajo-core-backend tajo-jdbc tajo-storage.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/101//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/101//console

          This message is automatically generated.

          Show
          Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12626839/TAJO-36_140204_163419.patch against master revision 0c0111c. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 222 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-common tajo-core/tajo-core-backend tajo-jdbc tajo-storage. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/101//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/101//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/101//console This message is automatically generated.
          Hide
          Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12626835/TAJO-36.patch
          against master revision 0c0111c.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 14 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 222 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-common tajo-core/tajo-core-backend tajo-jdbc tajo-storage.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/100//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/100//console

          This message is automatically generated.

          Show
          Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12626835/TAJO-36.patch against master revision 0c0111c. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 14 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 222 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-common tajo-core/tajo-core-backend tajo-jdbc tajo-storage. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/100//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/100//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/100//console This message is automatically generated.
          Hide
          Hyunsik Choi added a comment -

          removed an unused dependency in pom.xml

          Updated the review request against branch master in reviewboard
          https://reviews.apache.org/r/17693/

          Show
          Hyunsik Choi added a comment - removed an unused dependency in pom.xml Updated the review request against branch master in reviewboard https://reviews.apache.org/r/17693/
          Hyunsik Choi made changes -
          Attachment TAJO-36_140204_163419.patch [ 12626839 ]
          Hide
          Hyunsik Choi added a comment -

          This patch completely rewrites ExternalSortExec which is a physical operator for sort. It has the following improvements:

          • in-memory sort if input data fits main-memory (given sort buffer)
          • it works as n-way external merge sort operator if input data exceeds once a sort buffer size.
          • It performs k-way merge and it can be parallelized depending on the config (tajo.executor.external-sort.thread-num).
          • It performs unbalanced merge if possible. (if you need more information, please refer a paper 'Query evaluation techniques for large databases' or a polyphase sort.
          • It uses worker temporal directories in a round-robin manner.
          • It omits the final merge step because the final result is used as an input data of the subsequent physical operators. So, we don't need to merge the remain input files into only one file.

          You can adjust the following configs, and later ExternalSortExec should be able to change the following configs according tothe allocated container's resource.

          TajoConf.java
           EXECUTOR_EXTERNAL_SORT_THREAD_NUM("tajo.executor.external-sort.thread-num", 1),
           EXECUTOR_EXTERNAL_SORT_BUFFER_SIZE("tajo.executor.external-sort.buffer-mb", 200),
           EXECUTOR_EXTERNAL_SORT_FANOUT("tajo.executor.external-sort.fanout-num", 8),
          
          Show
          Hyunsik Choi added a comment - This patch completely rewrites ExternalSortExec which is a physical operator for sort. It has the following improvements: in-memory sort if input data fits main-memory (given sort buffer) it works as n-way external merge sort operator if input data exceeds once a sort buffer size. It performs k-way merge and it can be parallelized depending on the config (tajo.executor.external-sort.thread-num). It performs unbalanced merge if possible. (if you need more information, please refer a paper 'Query evaluation techniques for large databases' or a polyphase sort. It uses worker temporal directories in a round-robin manner. It omits the final merge step because the final result is used as an input data of the subsequent physical operators. So, we don't need to merge the remain input files into only one file. You can adjust the following configs, and later ExternalSortExec should be able to change the following configs according tothe allocated container's resource. TajoConf.java EXECUTOR_EXTERNAL_SORT_THREAD_NUM( "tajo.executor.external-sort.thread-num" , 1), EXECUTOR_EXTERNAL_SORT_BUFFER_SIZE( "tajo.executor.external-sort.buffer-mb" , 200), EXECUTOR_EXTERNAL_SORT_FANOUT( "tajo.executor.external-sort.fanout-num" , 8),
          Hyunsik Choi made changes -
          Labels gsoc gsoc2013 mentor
          Hyunsik Choi made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Fix Version/s 0.8-incubating [ 12324253 ]
          Hide
          Hyunsik Choi added a comment -

          Created a review request against branch master in reviewboard
          https://reviews.apache.org/r/17693/

          Show
          Hyunsik Choi added a comment - Created a review request against branch master in reviewboard https://reviews.apache.org/r/17693/
          Hyunsik Choi made changes -
          Attachment TAJO-36.patch [ 12626835 ]
          Hyunsik Choi made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hyunsik Choi made changes -
          Assignee Hyunsik Choi [ hyunsik ]
          Hyunsik Choi made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          Gavin made changes -
          Workflow jira [ 12777572 ] patch-available, re-open possible [ 12778010 ]
          Hide
          Hyunsik Choi added a comment -

          Hi Alexander Sibetheros,

          Thank you for your interest! You seem to have great background related to this project.

          You should submit your proposal to GSoC. Then, we will select candidate(s) based on the strength of their proposal. If you want proposal examples, you can find them in Jira.

          You can find more information about Tajo from http://tajo.incubator.apache.org and http://wiki.apache.org/tajo. Especially, "GettingStarted" (http://wiki.apache.org/tajo/GettingStarted) is a good point to start, and "HowToContribute" would be helpful for you.

          Thank you!

          Show
          Hyunsik Choi added a comment - Hi Alexander Sibetheros, Thank you for your interest! You seem to have great background related to this project. You should submit your proposal to GSoC. Then, we will select candidate(s) based on the strength of their proposal. If you want proposal examples, you can find them in Jira. You can find more information about Tajo from http://tajo.incubator.apache.org and http://wiki.apache.org/tajo . Especially, "GettingStarted" ( http://wiki.apache.org/tajo/GettingStarted ) is a good point to start, and "HowToContribute" would be helpful for you. Thank you!
          Hide
          Alexander Sibetheros added a comment - - edited

          Hello,
          I read the above summary of the project requested and it seems rather interesting.
          Although I don't have any coding history with the apache project, I just finished reading the ExternalSortExec.java and the implementation now seems rather straight forward, so I believe with some guidance I should be able to handle the task.

          Currently I am a 4th year student in Informatics(University of Athens), with great background in algorithms(especially sorting), strong c,c++,java,python skills and recently took part in the Sigmod 2013 programming contest(results pending), which required lots of research into algorithms and fast indexing and sorting mechanisms. This summer I have no remaining classes and my graduation thesis will begin in October, so I will have plenty of time to write code, test thoroughly and document.

          Show
          Alexander Sibetheros added a comment - - edited Hello, I read the above summary of the project requested and it seems rather interesting. Although I don't have any coding history with the apache project, I just finished reading the ExternalSortExec.java and the implementation now seems rather straight forward, so I believe with some guidance I should be able to handle the task. Currently I am a 4th year student in Informatics(University of Athens), with great background in algorithms(especially sorting), strong c,c++,java,python skills and recently took part in the Sigmod 2013 programming contest(results pending), which required lots of research into algorithms and fast indexing and sorting mechanisms. This summer I have no remaining classes and my graduation thesis will begin in October, so I will have plenty of time to write code, test thoroughly and document.
          Hyunsik Choi made changes -
          Field Original Value New Value
          Labels gsoc2013 mentor gsoc gsoc2013 mentor
          Hyunsik Choi created issue -

            People

            • Assignee:
              Hyunsik Choi
              Reporter:
              Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development