Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1026

Implement Query history persistency manager.

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: QueryMaster, TajoMaster
    • Labels:
      None

      Description

      Currently query history stored in the memory of TajoMaster or QueryMaster. Query detail information is disappeared when the cluster is restarted. And because of memory limitations, only a few hours or a few days histories are kept.
      I suggest the following features for the query history.

      • Worker saves a query task history to the HDFS after finishing a query.
      • QueryMaster saves a query history to the HDFS after finishing a query.
      • If query is finished, Information Web server in TajoMaster shows query history in the web page, If query is running TajoMaster forward to QueryMaster.
      • History data will be saved to HDFS with JSON format.
      • JSON format has history file version for backward compatibility.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user babokim opened a pull request:

          https://github.com/apache/tajo/pull/179

          TAJO-1026: Implement Query history persistency manager.

          Query history has the three types of files.

          • Query list file: saving the list of query(default store is hdfs), hourly rolling, stored by TajoMaster.

          `<tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist`

          • Query detail file: saving the query plan and execution block summary(default store is hdfs), single file per query or execution block, stored by QueryMaster.

          `<tajo.query-history.path>/<yyyyMMdd>/query-detail/<QUERY_ID>/query.hist`

          `<tajo.query-history.path>/<yyyyMMdd>/query-detail/<QUERY_ID>/<EB_ID>.hist`

          • Task detail file: saving the task history(default is local), stored by Worker.

          `<tajo.task-history.path>/<yyyyMMdd>/tasks/<WORKER_HOST><WORKER_PORT>/<WORKER_HOST><WORKER_PORT><HH><seq>.hist`

          And I added page navigation in query.jsp and querytasks.jsp

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/babokim/tajo TAJO-1026

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/179.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #179


          commit f9cd6c0129805f274cba63c4ff12e569bd0fd060
          Author: HyoungJun Kim <babokim@babokim-macbook-pro.local>
          Date: 2014-09-26T14:50:33Z

          TAJO-1026: Implement Query history persistency manager.
          Completed Writer

          commit c6d3ba3952d0e0f4d9355e68d9eb040e5d334dd2
          Author: HyoungJun Kim <babokim@babokim-macbook-pro.local>
          Date: 2014-10-04T09:31:43Z

          TAJO-1026: Implement Query history persistency manager.

          commit f3eed7f74bb12480b66980e4a098d5780b726abe
          Author: HyoungJun Kim <babokim@babokim-macbook-pro.local>
          Date: 2014-10-05T05:33:17Z

          TAJO-1026: Implement Query history persistency manager.

          commit bad8e84c62629685b46c8f46316750509952bebf
          Author: HyoungJun Kim <babokim@babokim-macbook-pro.local>
          Date: 2014-10-05T05:34:05Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1026


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user babokim opened a pull request: https://github.com/apache/tajo/pull/179 TAJO-1026 : Implement Query history persistency manager. Query history has the three types of files. Query list file: saving the list of query(default store is hdfs), hourly rolling, stored by TajoMaster. `<tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist` Query detail file: saving the query plan and execution block summary(default store is hdfs), single file per query or execution block, stored by QueryMaster. `<tajo.query-history.path>/<yyyyMMdd>/query-detail/<QUERY_ID>/query.hist` `<tajo.query-history.path>/<yyyyMMdd>/query-detail/<QUERY_ID>/<EB_ID>.hist` Task detail file: saving the task history(default is local), stored by Worker. `<tajo.task-history.path>/<yyyyMMdd>/tasks/<WORKER_HOST> <WORKER_PORT>/<WORKER_HOST> <WORKER_PORT> <HH> <seq>.hist` And I added page navigation in query.jsp and querytasks.jsp You can merge this pull request into a Git repository by running: $ git pull https://github.com/babokim/tajo TAJO-1026 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/179.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #179 commit f9cd6c0129805f274cba63c4ff12e569bd0fd060 Author: HyoungJun Kim <babokim@babokim-macbook-pro.local> Date: 2014-09-26T14:50:33Z TAJO-1026 : Implement Query history persistency manager. Completed Writer commit c6d3ba3952d0e0f4d9355e68d9eb040e5d334dd2 Author: HyoungJun Kim <babokim@babokim-macbook-pro.local> Date: 2014-10-04T09:31:43Z TAJO-1026 : Implement Query history persistency manager. commit f3eed7f74bb12480b66980e4a098d5780b726abe Author: HyoungJun Kim <babokim@babokim-macbook-pro.local> Date: 2014-10-05T05:33:17Z TAJO-1026 : Implement Query history persistency manager. commit bad8e84c62629685b46c8f46316750509952bebf Author: HyoungJun Kim <babokim@babokim-macbook-pro.local> Date: 2014-10-05T05:34:05Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1026
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/179#issuecomment-60498664

          I'm sorry for late review. Could you rebase it against the latest revision?

          Actually, I'm already reviewing it after trivial rebase. But it is better for you to push the rebased version because github pull request (PR) does not allow some user to push to another user's PR.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/179#issuecomment-60498664 I'm sorry for late review. Could you rebase it against the latest revision? Actually, I'm already reviewing it after trivial rebase. But it is better for you to push the rebased version because github pull request (PR) does not allow some user to push to another user's PR.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on the pull request:

          https://github.com/apache/tajo/pull/179#issuecomment-61930943

          I'm sorry for late rebase. I just rebase.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on the pull request: https://github.com/apache/tajo/pull/179#issuecomment-61930943 I'm sorry for late rebase. I just rebase.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067385

          — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java —
          @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) {
          METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"),

          // Misc -------------------------------------------------------------------
          + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"),
          — End diff –

          I'd like to suggest using ```tajo.history``` prefix for query and task histories. It seems to be more consistency. Also, we already use ```expiry-time[unit]``` suffix for expiry time. So, I'd like to suggest the followings:

          • ```tajo.history.query.dir```
          • ```tajo.history.task.dir```
          • ```tajo.history.expiry-time-day```
          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067385 — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java — @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) { METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"), // Misc ------------------------------------------------------------------- + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"), — End diff – I'd like to suggest using ```tajo.history``` prefix for query and task histories. It seems to be more consistency. Also, we already use ``` expiry-time [unit] ``` suffix for expiry time. So, I'd like to suggest the followings: ```tajo.history.query.dir``` ```tajo.history.task.dir``` ```tajo.history.expiry-time-day```
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067404

          — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java —
          @@ -19,21 +19,42 @@
          package org.apache.tajo.master.querymaster;

          +import com.google.gson.annotations.Expose;
          import org.apache.tajo.QueryId;
          import org.apache.tajo.TajoProtos;
          -
          -public class QueryInfo {
          +import org.apache.tajo.engine.json.CoreGsonHelper;
          +import org.apache.tajo.ipc.ClientProtos.QueryInfoProto;
          +import org.apache.tajo.ipc.ClientProtos.QueryInfoProtoOrBuilder;
          — End diff –

          It is an unused import.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067404 — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java — @@ -19,21 +19,42 @@ package org.apache.tajo.master.querymaster; +import com.google.gson.annotations.Expose; import org.apache.tajo.QueryId; import org.apache.tajo.TajoProtos; - -public class QueryInfo { +import org.apache.tajo.engine.json.CoreGsonHelper; +import org.apache.tajo.ipc.ClientProtos.QueryInfoProto; +import org.apache.tajo.ipc.ClientProtos.QueryInfoProtoOrBuilder; — End diff – It is an unused import.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067427

          — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java —
          @@ -394,6 +397,84 @@ public void addTask(QueryUnit task)

          { tasks.put(task.getId(), task); }

          + public SubQueryHistory getSubQueryHistory() {
          + if (finalSubQueryHistory != null) {
          + if (finalSubQueryHistory.getFinishTime() == 0)

          { + finalSubQueryHistory = makeSubQueryHistory(); + finalSubQueryHistory.setQueryUnits(makeQueryUnitHistories()); + }

          + return finalSubQueryHistory;
          + } else

          { + return makeSubQueryHistory(); + }

          + }
          +
          + public List<QueryUnitHistory> getQueryUnitHistories() {
          — End diff –

          It seems to be unused. Is it for the future work?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067427 — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java — @@ -394,6 +397,84 @@ public void addTask(QueryUnit task) { tasks.put(task.getId(), task); } + public SubQueryHistory getSubQueryHistory() { + if (finalSubQueryHistory != null) { + if (finalSubQueryHistory.getFinishTime() == 0) { + finalSubQueryHistory = makeSubQueryHistory(); + finalSubQueryHistory.setQueryUnits(makeQueryUnitHistories()); + } + return finalSubQueryHistory; + } else { + return makeSubQueryHistory(); + } + } + + public List<QueryUnitHistory> getQueryUnitHistories() { — End diff – It seems to be unused. Is it for the future work?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067542

          — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java —
          @@ -0,0 +1,470 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.util.history;
          +
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.hadoop.conf.Configuration;
          +import org.apache.hadoop.fs.*;
          +import org.apache.hadoop.service.AbstractService;
          +import org.apache.tajo.conf.TajoConf;
          +import org.apache.tajo.master.querymaster.QueryInfo;
          +import org.apache.tajo.worker.TaskHistory;
          +
          +import java.io.IOException;
          +import java.text.SimpleDateFormat;
          +import java.util.*;
          +import java.util.concurrent.LinkedBlockingQueue;
          +import java.util.concurrent.atomic.AtomicBoolean;
          +
          +/**
          + * History directory structure
          + * tajo.query-history.path: hdfs
          + * tajo.task-history.path: local or hdfs
          + *
          + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling)
          — End diff –

          Is ```<sequence>``` just a number? It would be better if the sequence is the written time. How do you think about that?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067542 — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java — @@ -0,0 +1,470 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.util.history; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.*; +import org.apache.hadoop.service.AbstractService; +import org.apache.tajo.conf.TajoConf; +import org.apache.tajo.master.querymaster.QueryInfo; +import org.apache.tajo.worker.TaskHistory; + +import java.io.IOException; +import java.text.SimpleDateFormat; +import java.util.*; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * History directory structure + * tajo.query-history.path: hdfs + * tajo.task-history.path: local or hdfs + * + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling) — End diff – Is ```<sequence>``` just a number? It would be better if the sequence is the written time. How do you think about that?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067563

          — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java —
          @@ -0,0 +1,470 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.util.history;
          +
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.hadoop.conf.Configuration;
          +import org.apache.hadoop.fs.*;
          +import org.apache.hadoop.service.AbstractService;
          +import org.apache.tajo.conf.TajoConf;
          +import org.apache.tajo.master.querymaster.QueryInfo;
          +import org.apache.tajo.worker.TaskHistory;
          +
          +import java.io.IOException;
          +import java.text.SimpleDateFormat;
          +import java.util.*;
          +import java.util.concurrent.LinkedBlockingQueue;
          +import java.util.concurrent.atomic.AtomicBoolean;
          +
          +/**
          + * History directory structure
          + * tajo.query-history.path: hdfs
          + * tajo.task-history.path: local or hdfs
          + *
          + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling)
          + * /query-detail/<QUERY_ID>/query.hist (QueryMaster's query detail)
          + * /<EB_ID>.hist (QueryMaster's subquery detail)
          + * <tajo.task-history.path>/<yyyyMMdd>/tasks/<WORKER_HOST><WORKER_PORT>/<WORKER_HOST><WORKER_PORT><HH><seq>.hist
          + * History files are kept for "tajo.query-history.preserve.hours" (default value is 68 hours-1 week.)
          — End diff –

          It seems to intend ```tajo.history.file.liveness.day```. I suggested ```tajo.history.expiry-time-day``` for it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067563 — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java — @@ -0,0 +1,470 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.util.history; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.*; +import org.apache.hadoop.service.AbstractService; +import org.apache.tajo.conf.TajoConf; +import org.apache.tajo.master.querymaster.QueryInfo; +import org.apache.tajo.worker.TaskHistory; + +import java.io.IOException; +import java.text.SimpleDateFormat; +import java.util.*; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * History directory structure + * tajo.query-history.path: hdfs + * tajo.task-history.path: local or hdfs + * + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling) + * /query-detail/<QUERY_ID>/query.hist (QueryMaster's query detail) + * /<EB_ID>.hist (QueryMaster's subquery detail) + * <tajo.task-history.path>/<yyyyMMdd>/tasks/<WORKER_HOST> <WORKER_PORT>/<WORKER_HOST> <WORKER_PORT> <HH> <seq>.hist + * History files are kept for "tajo.query-history.preserve.hours" (default value is 68 hours-1 week.) — End diff – It seems to intend ```tajo.history.file.liveness.day```. I suggested ```tajo.history.expiry-time-day``` for it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067593

          — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java —
          @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) {
          METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"),

          // Misc -------------------------------------------------------------------
          + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"),
          + QUERY_TASK_HISTORY_DIR("tajo.task-history.dir", "file:///tmp/tajo-$

          {user.name}

          /history"),
          + HISTORY_FILE_LIVENESS_DAY("tajo.history.file.liveness.day", 7),
          — End diff –

          Day seems to be too big unit. How about using hours?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067593 — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java — @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) { METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"), // Misc ------------------------------------------------------------------- + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"), + QUERY_TASK_HISTORY_DIR("tajo.task-history.dir", "file:///tmp/tajo-$ {user.name} /history"), + HISTORY_FILE_LIVENESS_DAY("tajo.history.file.liveness.day", 7), — End diff – Day seems to be too big unit. How about using hours?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20067635

          — Diff: tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java —
          @@ -246,4 +350,76 @@ public static String tableStatToString(CatalogProtos.TableStatsProto tableStats)

          return result;
          }
          +
          + public static String getPageSizeSelection(String id, int pageSize) {
          — End diff –

          It seems to be unused. Is it for future work?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20067635 — Diff: tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java — @@ -246,4 +350,76 @@ public static String tableStatToString(CatalogProtos.TableStatsProto tableStats) return result; } + + public static String getPageSizeSelection(String id, int pageSize) { — End diff – It seems to be unused. Is it for future work?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/179#issuecomment-62344523

          Hi @babokim,

          It's really great work. This work will reduce our memory consumption for keeping history in memory, and this work is able to keep history longer than now.

          I leave some comments and some questions. After your feedback, I'll finish the review.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/179#issuecomment-62344523 Hi @babokim, It's really great work. This work will reduce our memory consumption for keeping history in memory, and this work is able to keep history longer than now. I leave some comments and some questions. After your feedback, I'll finish the review.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20068990

          — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java —
          @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) {
          METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"),

          // Misc -------------------------------------------------------------------
          + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"),
          — End diff –

          It looks better. I will modify it as your comment.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20068990 — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java — @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) { METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"), // Misc ------------------------------------------------------------------- + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"), — End diff – It looks better. I will modify it as your comment.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20068998

          — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java —
          @@ -19,21 +19,42 @@
          package org.apache.tajo.master.querymaster;

          +import com.google.gson.annotations.Expose;
          import org.apache.tajo.QueryId;
          import org.apache.tajo.TajoProtos;
          -
          -public class QueryInfo {
          +import org.apache.tajo.engine.json.CoreGsonHelper;
          +import org.apache.tajo.ipc.ClientProtos.QueryInfoProto;
          +import org.apache.tajo.ipc.ClientProtos.QueryInfoProtoOrBuilder;
          — End diff –

          I will remove it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20068998 — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java — @@ -19,21 +19,42 @@ package org.apache.tajo.master.querymaster; +import com.google.gson.annotations.Expose; import org.apache.tajo.QueryId; import org.apache.tajo.TajoProtos; - -public class QueryInfo { +import org.apache.tajo.engine.json.CoreGsonHelper; +import org.apache.tajo.ipc.ClientProtos.QueryInfoProto; +import org.apache.tajo.ipc.ClientProtos.QueryInfoProtoOrBuilder; — End diff – I will remove it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20069150

          — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java —
          @@ -394,6 +397,84 @@ public void addTask(QueryUnit task)

          { tasks.put(task.getId(), task); }

          + public SubQueryHistory getSubQueryHistory() {
          + if (finalSubQueryHistory != null) {
          + if (finalSubQueryHistory.getFinishTime() == 0)

          { + finalSubQueryHistory = makeSubQueryHistory(); + finalSubQueryHistory.setQueryUnits(makeQueryUnitHistories()); + }

          + return finalSubQueryHistory;
          + } else

          { + return makeSubQueryHistory(); + }

          + }
          +
          + public List<QueryUnitHistory> getQueryUnitHistories() {
          — End diff –

          You are right. getQueryUnitHistories() method is unused. I will remove it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20069150 — Diff: tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java — @@ -394,6 +397,84 @@ public void addTask(QueryUnit task) { tasks.put(task.getId(), task); } + public SubQueryHistory getSubQueryHistory() { + if (finalSubQueryHistory != null) { + if (finalSubQueryHistory.getFinishTime() == 0) { + finalSubQueryHistory = makeSubQueryHistory(); + finalSubQueryHistory.setQueryUnits(makeQueryUnitHistories()); + } + return finalSubQueryHistory; + } else { + return makeSubQueryHistory(); + } + } + + public List<QueryUnitHistory> getQueryUnitHistories() { — End diff – You are right. getQueryUnitHistories() method is unused. I will remove it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20069326

          — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java —
          @@ -0,0 +1,470 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.util.history;
          +
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.hadoop.conf.Configuration;
          +import org.apache.hadoop.fs.*;
          +import org.apache.hadoop.service.AbstractService;
          +import org.apache.tajo.conf.TajoConf;
          +import org.apache.tajo.master.querymaster.QueryInfo;
          +import org.apache.tajo.worker.TaskHistory;
          +
          +import java.io.IOException;
          +import java.text.SimpleDateFormat;
          +import java.util.*;
          +import java.util.concurrent.LinkedBlockingQueue;
          +import java.util.concurrent.atomic.AtomicBoolean;
          +
          +/**
          + * History directory structure
          + * tajo.query-history.path: hdfs
          + * tajo.task-history.path: local or hdfs
          + *
          + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling)
          — End diff –

          I will change to 'HHmmss'.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20069326 — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java — @@ -0,0 +1,470 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.util.history; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.*; +import org.apache.hadoop.service.AbstractService; +import org.apache.tajo.conf.TajoConf; +import org.apache.tajo.master.querymaster.QueryInfo; +import org.apache.tajo.worker.TaskHistory; + +import java.io.IOException; +import java.text.SimpleDateFormat; +import java.util.*; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * History directory structure + * tajo.query-history.path: hdfs + * tajo.task-history.path: local or hdfs + * + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling) — End diff – I will change to 'HHmmss'.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20069424

          — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java —
          @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) {
          METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"),

          // Misc -------------------------------------------------------------------
          + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"),
          + QUERY_TASK_HISTORY_DIR("tajo.task-history.dir", "file:///tmp/tajo-$

          {user.name}

          /history"),
          + HISTORY_FILE_LIVENESS_DAY("tajo.history.file.liveness.day", 7),
          — End diff –

          The reason is that history files are saved in the directory by date.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20069424 — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java — @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) { METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"), // Misc ------------------------------------------------------------------- + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"), + QUERY_TASK_HISTORY_DIR("tajo.task-history.dir", "file:///tmp/tajo-$ {user.name} /history"), + HISTORY_FILE_LIVENESS_DAY("tajo.history.file.liveness.day", 7), — End diff – The reason is that history files are saved in the directory by date.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20069513

          — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java —
          @@ -0,0 +1,470 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.util.history;
          +
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.hadoop.conf.Configuration;
          +import org.apache.hadoop.fs.*;
          +import org.apache.hadoop.service.AbstractService;
          +import org.apache.tajo.conf.TajoConf;
          +import org.apache.tajo.master.querymaster.QueryInfo;
          +import org.apache.tajo.worker.TaskHistory;
          +
          +import java.io.IOException;
          +import java.text.SimpleDateFormat;
          +import java.util.*;
          +import java.util.concurrent.LinkedBlockingQueue;
          +import java.util.concurrent.atomic.AtomicBoolean;
          +
          +/**
          + * History directory structure
          + * tajo.query-history.path: hdfs
          + * tajo.task-history.path: local or hdfs
          + *
          + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling)
          + * /query-detail/<QUERY_ID>/query.hist (QueryMaster's query detail)
          + * /<EB_ID>.hist (QueryMaster's subquery detail)
          + * <tajo.task-history.path>/<yyyyMMdd>/tasks/<WORKER_HOST><WORKER_PORT>/<WORKER_HOST><WORKER_PORT><HH><seq>.hist
          + * History files are kept for "tajo.query-history.preserve.hours" (default value is 68 hours-1 week.)
          — End diff –

          I will modify as you mentioned.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20069513 — Diff: tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java — @@ -0,0 +1,470 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.util.history; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.*; +import org.apache.hadoop.service.AbstractService; +import org.apache.tajo.conf.TajoConf; +import org.apache.tajo.master.querymaster.QueryInfo; +import org.apache.tajo.worker.TaskHistory; + +import java.io.IOException; +import java.text.SimpleDateFormat; +import java.util.*; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * History directory structure + * tajo.query-history.path: hdfs + * tajo.task-history.path: local or hdfs + * + * <tajo.query-history.path>/<yyyyMMdd>/query-list/query-list-<sequence>.hist (TajoMaster's query list, hourly rolling) + * /query-detail/<QUERY_ID>/query.hist (QueryMaster's query detail) + * /<EB_ID>.hist (QueryMaster's subquery detail) + * <tajo.task-history.path>/<yyyyMMdd>/tasks/<WORKER_HOST> <WORKER_PORT>/<WORKER_HOST> <WORKER_PORT> <HH> <seq>.hist + * History files are kept for "tajo.query-history.preserve.hours" (default value is 68 hours-1 week.) — End diff – I will modify as you mentioned.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20069524

          — Diff: tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java —
          @@ -246,4 +350,76 @@ public static String tableStatToString(CatalogProtos.TableStatsProto tableStats)

          return result;
          }
          +
          + public static String getPageSizeSelection(String id, int pageSize) {
          — End diff –

          You are right. I will remove it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20069524 — Diff: tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java — @@ -246,4 +350,76 @@ public static String tableStatToString(CatalogProtos.TableStatsProto tableStats) return result; } + + public static String getPageSizeSelection(String id, int pageSize) { — End diff – You are right. I will remove it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/179#discussion_r20069533

          — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java —
          @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) {
          METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"),

          // Misc -------------------------------------------------------------------
          + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"),
          + QUERY_TASK_HISTORY_DIR("tajo.task-history.dir", "file:///tmp/tajo-$

          {user.name}

          /history"),
          + HISTORY_FILE_LIVENESS_DAY("tajo.history.file.liveness.day", 7),
          — End diff –

          The reason looks reasonable. Please go ahead!

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/179#discussion_r20069533 — Diff: tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java — @@ -297,6 +297,9 @@ public static int setDateOrder(int dateOrder) { METRICS_PROPERTY_FILENAME("tajo.metrics.property.file", "tajo-metrics.properties"), // Misc ------------------------------------------------------------------- + QUERY_HISTORY_DIR("tajo.query-history.dir", STAGING_ROOT_DIR.defaultVal + "/history"), + QUERY_TASK_HISTORY_DIR("tajo.task-history.dir", "file:///tmp/tajo-$ {user.name} /history"), + HISTORY_FILE_LIVENESS_DAY("tajo.history.file.liveness.day", 7), — End diff – The reason looks reasonable. Please go ahead!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/179#issuecomment-62427235

          +1

          The patch looks nice to me. I checked the history list and task details in web UI. They work well.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/179#issuecomment-62427235 +1 The patch looks nice to me. I checked the history list and task details in web UI. They work well.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user babokim commented on the pull request:

          https://github.com/apache/tajo/pull/179#issuecomment-62487872

          @hyunsik Thanks. I had just committed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user babokim commented on the pull request: https://github.com/apache/tajo/pull/179#issuecomment-62487872 @hyunsik Thanks. I had just committed.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #442 (See https://builds.apache.org/job/Tajo-master-build/442/)
          TAJO-1026: Implement Query history persistency manager. (babokim: rev e01b00a7b05b98cc407ced04dad1c2f2144d0f88)

          • tajo-core/src/main/java/org/apache/tajo/worker/Task.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TaskHistory.java
          • tajo-client/src/main/proto/QueryMasterClientProtocol.proto
          • tajo-core/src/main/resources/webapps/worker/querytasks.jsp
          • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
          • tajo-core/src/main/resources/webapps/admin/querytasks.jsp
          • tajo-core/src/main/resources/webapps/admin/query.jsp
          • tajo-core/src/main/java/org/apache/tajo/util/history/History.java
          • tajo-client/src/main/java/org/apache/tajo/client/QueryClient.java
          • tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/QueryHistory.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/HistoryCleaner.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/HistoryReader.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/QueryUnitHistory.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java
          • tajo-core/src/main/resources/webapps/worker/querydetail.jsp
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
          • tajo-core/src/main/resources/webapps/worker/taskhistory.jsp
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/SubQueryHistory.java
          • tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java
          • tajo-client/src/main/proto/TajoMasterClientProtocol.proto
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          • CHANGES
          • tajo-core/src/test/java/org/apache/tajo/util/history/TestHistoryWriterReader.java
          • tajo-core/src/main/resources/webapps/admin/queryunit.jsp
          • tajo-core/src/test/java/org/apache/tajo/util/TestJSPUtil.java
          • tajo-core/src/test/java/org/apache/tajo/client/TestTajoClient.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-core/src/main/resources/webapps/admin/querydetail.jsp
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #442 (See https://builds.apache.org/job/Tajo-master-build/442/ ) TAJO-1026 : Implement Query history persistency manager. (babokim: rev e01b00a7b05b98cc407ced04dad1c2f2144d0f88) tajo-core/src/main/java/org/apache/tajo/worker/Task.java tajo-core/src/main/java/org/apache/tajo/worker/TaskHistory.java tajo-client/src/main/proto/QueryMasterClientProtocol.proto tajo-core/src/main/resources/webapps/worker/querytasks.jsp tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-core/src/main/resources/webapps/admin/querytasks.jsp tajo-core/src/main/resources/webapps/admin/query.jsp tajo-core/src/main/java/org/apache/tajo/util/history/History.java tajo-client/src/main/java/org/apache/tajo/client/QueryClient.java tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java tajo-client/src/main/proto/ClientProtos.proto tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java tajo-core/src/main/java/org/apache/tajo/util/history/QueryHistory.java tajo-core/src/main/java/org/apache/tajo/util/history/HistoryCleaner.java tajo-core/src/main/java/org/apache/tajo/util/history/HistoryReader.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java tajo-core/src/main/java/org/apache/tajo/util/history/QueryUnitHistory.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java tajo-core/src/main/resources/webapps/worker/querydetail.jsp tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java tajo-core/src/main/resources/webapps/worker/taskhistory.jsp tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java tajo-core/src/main/java/org/apache/tajo/util/history/SubQueryHistory.java tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java tajo-client/src/main/proto/TajoMasterClientProtocol.proto tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java CHANGES tajo-core/src/test/java/org/apache/tajo/util/history/TestHistoryWriterReader.java tajo-core/src/main/resources/webapps/admin/queryunit.jsp tajo-core/src/test/java/org/apache/tajo/util/TestJSPUtil.java tajo-core/src/test/java/org/apache/tajo/client/TestTajoClient.java tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/src/main/resources/webapps/admin/querydetail.jsp tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #84 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/84/)
          TAJO-1026: Implement Query history persistency manager. (babokim: rev e01b00a7b05b98cc407ced04dad1c2f2144d0f88)

          • tajo-core/src/main/java/org/apache/tajo/worker/TaskHistory.java
          • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-core/src/main/resources/webapps/worker/querydetail.jsp
          • tajo-core/src/main/resources/webapps/admin/query.jsp
          • tajo-core/src/main/java/org/apache/tajo/util/history/HistoryReader.java
          • tajo-core/src/main/resources/webapps/admin/querydetail.jsp
          • tajo-core/src/main/java/org/apache/tajo/util/history/HistoryCleaner.java
          • tajo-core/src/main/resources/webapps/worker/taskhistory.jsp
          • tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java
          • tajo-core/src/main/resources/webapps/admin/queryunit.jsp
          • tajo-core/src/main/resources/webapps/worker/querytasks.jsp
          • tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/QueryUnitHistory.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          • tajo-core/src/test/java/org/apache/tajo/util/history/TestHistoryWriterReader.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-core/src/main/java/org/apache/tajo/util/history/SubQueryHistory.java
          • tajo-core/src/main/resources/webapps/admin/querytasks.jsp
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java
          • tajo-client/src/main/proto/TajoMasterClientProtocol.proto
          • tajo-core/src/main/java/org/apache/tajo/util/history/History.java
          • tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java
          • tajo-client/src/main/java/org/apache/tajo/client/QueryClient.java
          • tajo-core/src/main/java/org/apache/tajo/worker/Task.java
          • tajo-core/src/test/java/org/apache/tajo/util/TestJSPUtil.java
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java
          • tajo-core/src/test/java/org/apache/tajo/client/TestTajoClient.java
          • tajo-client/src/main/proto/QueryMasterClientProtocol.proto
          • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
          • tajo-core/src/main/java/org/apache/tajo/util/history/QueryHistory.java
          • CHANGES
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #84 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/84/ ) TAJO-1026 : Implement Query history persistency manager. (babokim: rev e01b00a7b05b98cc407ced04dad1c2f2144d0f88) tajo-core/src/main/java/org/apache/tajo/worker/TaskHistory.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/src/main/resources/webapps/worker/querydetail.jsp tajo-core/src/main/resources/webapps/admin/query.jsp tajo-core/src/main/java/org/apache/tajo/util/history/HistoryReader.java tajo-core/src/main/resources/webapps/admin/querydetail.jsp tajo-core/src/main/java/org/apache/tajo/util/history/HistoryCleaner.java tajo-core/src/main/resources/webapps/worker/taskhistory.jsp tajo-client/src/main/java/org/apache/tajo/client/QueryClientImpl.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/Query.java tajo-core/src/main/resources/webapps/admin/queryunit.jsp tajo-core/src/main/resources/webapps/worker/querytasks.jsp tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java tajo-core/src/main/java/org/apache/tajo/util/history/QueryUnitHistory.java tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java tajo-core/src/test/java/org/apache/tajo/util/history/TestHistoryWriterReader.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-client/src/main/proto/ClientProtos.proto tajo-core/src/main/java/org/apache/tajo/util/history/SubQueryHistory.java tajo-core/src/main/resources/webapps/admin/querytasks.jsp tajo-core/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java tajo-client/src/main/proto/TajoMasterClientProtocol.proto tajo-core/src/main/java/org/apache/tajo/util/history/History.java tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java tajo-client/src/main/java/org/apache/tajo/client/QueryClient.java tajo-core/src/main/java/org/apache/tajo/worker/Task.java tajo-core/src/test/java/org/apache/tajo/util/TestJSPUtil.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java tajo-core/src/test/java/org/apache/tajo/client/TestTajoClient.java tajo-client/src/main/proto/QueryMasterClientProtocol.proto tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java tajo-core/src/main/java/org/apache/tajo/util/history/QueryHistory.java CHANGES
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/179

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/179

            People

            • Assignee:
              hjkim Hyoungjun Kim
              Reporter:
              hjkim Hyoungjun Kim
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development