Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1306

HAServiceUtil should not directly use HDFS.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: Java Client, TajoMaster
    • Labels:
      None

      Description

      HAServiceUtil is tightly coupled with HDFS Filesystem. We have a plan to implement multiple HA implementations like HDFS and Zookeeper. We need to abstract it to support multiple implementations.

      Also, it forces TajoClient to have Hadoop HDFS dependency. If we decouple using HDFS from HAServiceUtil, we can eliminate HDFS dependency from TajoClient.

      1. TAJO-1306.patch
        105 kB
        Hyunsik Choi

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user hyunsik opened a pull request:

          https://github.com/apache/tajo/pull/358

          TAJO-1306: HAServiceUtil should not directly use HDFS.

          In this patch, I added an abstract class ServiceTracker which registers Tajo rpc services and provides discovery services for them. I changed almost all parts using HAServiceUtil to use ServiceTracker.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/hyunsik/tajo TAJO-1306

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/358.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #358


          commit a3c9f839aaffff72280bdd89cc20181d58403e9b
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-01-22T00:03:36Z

          Merged initial work.

          commit 9551c9825b651e3edbbb88cc35ac71eda95cae3c
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-01-22T16:30:34Z

          Refactored client side of HA.

          commit 388186e6267599542b741bb0b4f56c5a3d2278de
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-01-22T17:18:16Z

          Recover changed conf name.

          commit ff21df376835df448e2847b832ec1fe298a6c36a
          Author: Hyunsik Choi <hyunsik@apache.org>
          Date: 2015-01-22T17:18:30Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1306_CC


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/358 TAJO-1306 : HAServiceUtil should not directly use HDFS. In this patch, I added an abstract class ServiceTracker which registers Tajo rpc services and provides discovery services for them. I changed almost all parts using HAServiceUtil to use ServiceTracker. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1306 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/358.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #358 commit a3c9f839aaffff72280bdd89cc20181d58403e9b Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-01-22T00:03:36Z Merged initial work. commit 9551c9825b651e3edbbb88cc35ac71eda95cae3c Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-01-22T16:30:34Z Refactored client side of HA. commit 388186e6267599542b741bb0b4f56c5a3d2278de Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-01-22T17:18:16Z Recover changed conf name. commit ff21df376835df448e2847b832ec1fe298a6c36a Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-01-22T17:18:30Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1306 _CC
          Hide
          tajoqa Tajo QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12693937/TAJO-1306.patch
          against master revision release-0.9.0-rc0-149-gf80ec42.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          -1 findbugs. The patch appears to introduce 200 new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 303 release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-client tajo-client tajo-common tajo-core tajo-jdbc tajo-storage/tajo-storage-common.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/574//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/574//console

          This message is automatically generated.

          Show
          tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12693937/TAJO-1306.patch against master revision release-0.9.0-rc0-149-gf80ec42. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 200 new Findbugs (version 2.0.3) warnings. -1 release audit. The applied patch generated 303 release audit warnings. +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-client tajo-client tajo-common tajo-core tajo-jdbc tajo-storage/tajo-storage-common. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/574//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/574//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/574//console This message is automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jinossy commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/358#discussion_r23427872

          — Diff: tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java —
          @@ -0,0 +1,576 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.ha;
          +
          +import com.google.common.base.Preconditions;
          +import org.apache.commons.logging.Log;
          +import org.apache.commons.logging.LogFactory;
          +import org.apache.hadoop.fs.*;
          +import org.apache.hadoop.fs.permission.FsPermission;
          +import org.apache.hadoop.net.NetUtils;
          +import org.apache.tajo.TajoConstants;
          +import org.apache.tajo.conf.TajoConf;
          +import org.apache.tajo.conf.TajoConf.ConfVars;
          +import org.apache.tajo.master.TajoMaster;
          +import org.apache.tajo.service.HAServiceTracker;
          +import org.apache.tajo.service.ServiceTrackerException;
          +import org.apache.tajo.service.TajoMasterInfo;
          +import org.apache.tajo.util.TUtil;
          +
          +import javax.net.SocketFactory;
          +import java.io.IOException;
          +import java.net.InetSocketAddress;
          +import java.net.Socket;
          +import java.util.ArrayList;
          +import java.util.List;
          +
          +/**
          + * This implements HAService utilizing HDFS cluster. This saves master status to HDFS cluster.
          + *
          + */
          +@SuppressWarnings("unused")
          +public class HdfsServiceTracker extends HAServiceTracker {
          + private static Log LOG = LogFactory.getLog(HdfsServiceTracker.class);
          +
          + private TajoConf conf;
          +
          + private FileSystem fs;
          +
          + private String masterName;
          + private Path rootPath;
          + private Path haPath;
          + private Path activePath;
          + private Path backupPath;
          +
          + private boolean isActiveStatus = false;
          +
          + //thread which runs periodically to see the last time since a heartbeat is received.
          + private Thread checkerThread;
          + private volatile boolean stopped = false;
          +
          + private int monitorInterval;
          +
          + private String currentActiveMaster;
          +
          + public HdfsServiceTracker(TajoConf conf) throws IOException

          { + this.conf = conf; + initSystemDirectory(); + + InetSocketAddress socketAddress = conf.getSocketAddrVar(ConfVars.TAJO_MASTER_UMBILICAL_RPC_ADDRESS); + this.masterName = socketAddress.getAddress().getHostAddress() + ":" + socketAddress.getPort(); + + monitorInterval = conf.getIntVar(ConfVars.TAJO_MASTER_HA_MONITOR_INTERVAL); + }

          +
          + private void initSystemDirectory() throws IOException {
          + // Get Tajo root dir
          + this.rootPath = TajoConf.getTajoRootDir(conf);
          +
          + // Check Tajo root dir
          + this.fs = rootPath.getFileSystem(conf);
          +
          + // Check and create Tajo system HA dir
          + haPath = TajoConf.getSystemHADir(conf);
          + if (!fs.exists(haPath))

          { + fs.mkdirs(haPath, new FsPermission(TajoMaster.SYSTEM_RESOURCE_DIR_PERMISSION)); + LOG.info("System HA dir '" + haPath + "' is created"); + }

          +
          + activePath = new Path(haPath, TajoConstants.SYSTEM_HA_ACTIVE_DIR_NAME);
          + if (!fs.exists(activePath))

          { + fs.mkdirs(activePath, new FsPermission(TajoMaster.SYSTEM_RESOURCE_DIR_PERMISSION)); + LOG.info("System HA Active dir '" + activePath + "' is created"); + }

          +
          + backupPath = new Path(haPath, TajoConstants.SYSTEM_HA_BACKUP_DIR_NAME);
          + if (!fs.exists(backupPath))

          { + fs.mkdirs(backupPath, new FsPermission(TajoMaster.SYSTEM_RESOURCE_DIR_PERMISSION)); + LOG.info("System HA Backup dir '" + backupPath + "' is created"); + }

          + }
          +
          + private void startPingChecker() {
          + if (checkerThread == null)

          { + checkerThread = new Thread(new PingChecker()); + checkerThread.setName("Ping Checker"); + checkerThread.start(); + }

          + }
          +
          + @Override
          + public void register() throws IOException {
          + FileStatus[] files = fs.listStatus(activePath);
          +
          + // Phase 1: If there is not another active master, this try to become active master.
          + if (files.length == 0)

          { + createMasterFile(true); + currentActiveMaster = masterName; + LOG.info(String.format("This is added to active master (%s)", masterName)); + }

          else {
          + // Phase 2: If there is active master information, we need to check its status.
          + Path activePath = files[0].getPath();
          + currentActiveMaster = activePath.getName().replaceAll("_", ":");
          +
          + // Phase 3: If current active master is dead, this master should be active master.
          + if (!HAServiceUtil.isMasterAlive(currentActiveMaster, conf))

          { + fs.delete(activePath, true); + createMasterFile(true); + currentActiveMaster = masterName; + LOG.info(String.format("This is added to active master (%s)", masterName)); + }

          else

          { + // Phase 4: If current active master is alive, this master need to be backup master. + createMasterFile(false); + LOG.info(String.format("This is added to backup masters (%s)", masterName)); + }

          + }
          + }
          +
          + /**
          + * It will creates the following form string. It includes
          + *
          + * <pre>
          + *

          {CLIENT_RPC_HOST:PORT}

          _

          {RESOURCE_TRACKER_HOST:PORT}

          _

          {CATALOG_HOST:PORT}

          _

          {MASTER_WEB_HOST:PORT}

          + * </pre>
          + *
          + * @param isActive A boolean flag to indicate if it is for master or not.
          + * @throws IOException
          + */
          + private void createMasterFile(boolean isActive) throws IOException {
          + String fileName = masterName.replaceAll(":", "_");
          + Path path = null;
          +
          + if (isActive)

          { + path = new Path(activePath, fileName); + }

          else

          { + path = new Path(backupPath, fileName); + }

          +
          + StringBuilder sb = new StringBuilder();
          + InetSocketAddress address = getHostAddress(HAConstants.MASTER_CLIENT_RPC_ADDRESS);
          + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()).append("_");
          +
          + address = getHostAddress(HAConstants.RESOURCE_TRACKER_RPC_ADDRESS);
          + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()).append("_");
          +
          + address = getHostAddress(HAConstants.CATALOG_ADDRESS);
          + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()).append("_");
          +
          + address = getHostAddress(HAConstants.MASTER_INFO_ADDRESS);
          + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort());
          +
          + FSDataOutputStream out = fs.create(path);
          +
          + try {
          + out.writeUTF(sb.toString());
          + out.hflush();
          — End diff –

          hflush() is client side buffer flushing. Can you change to hsync() ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jinossy commented on a diff in the pull request: https://github.com/apache/tajo/pull/358#discussion_r23427872 — Diff: tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java — @@ -0,0 +1,576 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.ha; + +import com.google.common.base.Preconditions; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.fs.*; +import org.apache.hadoop.fs.permission.FsPermission; +import org.apache.hadoop.net.NetUtils; +import org.apache.tajo.TajoConstants; +import org.apache.tajo.conf.TajoConf; +import org.apache.tajo.conf.TajoConf.ConfVars; +import org.apache.tajo.master.TajoMaster; +import org.apache.tajo.service.HAServiceTracker; +import org.apache.tajo.service.ServiceTrackerException; +import org.apache.tajo.service.TajoMasterInfo; +import org.apache.tajo.util.TUtil; + +import javax.net.SocketFactory; +import java.io.IOException; +import java.net.InetSocketAddress; +import java.net.Socket; +import java.util.ArrayList; +import java.util.List; + +/** + * This implements HAService utilizing HDFS cluster. This saves master status to HDFS cluster. + * + */ +@SuppressWarnings("unused") +public class HdfsServiceTracker extends HAServiceTracker { + private static Log LOG = LogFactory.getLog(HdfsServiceTracker.class); + + private TajoConf conf; + + private FileSystem fs; + + private String masterName; + private Path rootPath; + private Path haPath; + private Path activePath; + private Path backupPath; + + private boolean isActiveStatus = false; + + //thread which runs periodically to see the last time since a heartbeat is received. + private Thread checkerThread; + private volatile boolean stopped = false; + + private int monitorInterval; + + private String currentActiveMaster; + + public HdfsServiceTracker(TajoConf conf) throws IOException { + this.conf = conf; + initSystemDirectory(); + + InetSocketAddress socketAddress = conf.getSocketAddrVar(ConfVars.TAJO_MASTER_UMBILICAL_RPC_ADDRESS); + this.masterName = socketAddress.getAddress().getHostAddress() + ":" + socketAddress.getPort(); + + monitorInterval = conf.getIntVar(ConfVars.TAJO_MASTER_HA_MONITOR_INTERVAL); + } + + private void initSystemDirectory() throws IOException { + // Get Tajo root dir + this.rootPath = TajoConf.getTajoRootDir(conf); + + // Check Tajo root dir + this.fs = rootPath.getFileSystem(conf); + + // Check and create Tajo system HA dir + haPath = TajoConf.getSystemHADir(conf); + if (!fs.exists(haPath)) { + fs.mkdirs(haPath, new FsPermission(TajoMaster.SYSTEM_RESOURCE_DIR_PERMISSION)); + LOG.info("System HA dir '" + haPath + "' is created"); + } + + activePath = new Path(haPath, TajoConstants.SYSTEM_HA_ACTIVE_DIR_NAME); + if (!fs.exists(activePath)) { + fs.mkdirs(activePath, new FsPermission(TajoMaster.SYSTEM_RESOURCE_DIR_PERMISSION)); + LOG.info("System HA Active dir '" + activePath + "' is created"); + } + + backupPath = new Path(haPath, TajoConstants.SYSTEM_HA_BACKUP_DIR_NAME); + if (!fs.exists(backupPath)) { + fs.mkdirs(backupPath, new FsPermission(TajoMaster.SYSTEM_RESOURCE_DIR_PERMISSION)); + LOG.info("System HA Backup dir '" + backupPath + "' is created"); + } + } + + private void startPingChecker() { + if (checkerThread == null) { + checkerThread = new Thread(new PingChecker()); + checkerThread.setName("Ping Checker"); + checkerThread.start(); + } + } + + @Override + public void register() throws IOException { + FileStatus[] files = fs.listStatus(activePath); + + // Phase 1: If there is not another active master, this try to become active master. + if (files.length == 0) { + createMasterFile(true); + currentActiveMaster = masterName; + LOG.info(String.format("This is added to active master (%s)", masterName)); + } else { + // Phase 2: If there is active master information, we need to check its status. + Path activePath = files [0] .getPath(); + currentActiveMaster = activePath.getName().replaceAll("_", ":"); + + // Phase 3: If current active master is dead, this master should be active master. + if (!HAServiceUtil.isMasterAlive(currentActiveMaster, conf)) { + fs.delete(activePath, true); + createMasterFile(true); + currentActiveMaster = masterName; + LOG.info(String.format("This is added to active master (%s)", masterName)); + } else { + // Phase 4: If current active master is alive, this master need to be backup master. + createMasterFile(false); + LOG.info(String.format("This is added to backup masters (%s)", masterName)); + } + } + } + + /** + * It will creates the following form string. It includes + * + * <pre> + * {CLIENT_RPC_HOST:PORT} _ {RESOURCE_TRACKER_HOST:PORT} _ {CATALOG_HOST:PORT} _ {MASTER_WEB_HOST:PORT} + * </pre> + * + * @param isActive A boolean flag to indicate if it is for master or not. + * @throws IOException + */ + private void createMasterFile(boolean isActive) throws IOException { + String fileName = masterName.replaceAll(":", "_"); + Path path = null; + + if (isActive) { + path = new Path(activePath, fileName); + } else { + path = new Path(backupPath, fileName); + } + + StringBuilder sb = new StringBuilder(); + InetSocketAddress address = getHostAddress(HAConstants.MASTER_CLIENT_RPC_ADDRESS); + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()).append("_"); + + address = getHostAddress(HAConstants.RESOURCE_TRACKER_RPC_ADDRESS); + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()).append("_"); + + address = getHostAddress(HAConstants.CATALOG_ADDRESS); + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()).append("_"); + + address = getHostAddress(HAConstants.MASTER_INFO_ADDRESS); + sb.append(address.getAddress().getHostAddress()).append(":").append(address.getPort()); + + FSDataOutputStream out = fs.create(path); + + try { + out.writeUTF(sb.toString()); + out.hflush(); — End diff – hflush() is client side buffer flushing. Can you change to hsync() ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71138119

          Hi @hyunsik

          Thanks for your contribution and simple codes by ServiceTracker looks very pretty to me.
          But unfortunately, I couldn't run Tajo HA cluster with a shell script (start-tajo.sh) on my macbook as follows:

          2015-01-23 11:15:46,876 INFO org.apache.tajo.master.rm.TajoWorkerResourceManager: Released QueryMaster (q_1421979321683_0001) resource.
          2015-01-23 11:15:46,885 INFO org.apache.tajo.util.history.HistoryWriter: Create query history file: hdfs://localhost:9010/tmp/tajo-blrunner/staging/history/20150123/query-list/query-list-111546.hist
          2015-01-23 11:15:57,739 INFO org.apache.tajo.session.SessionManager: Session f103d278-ab81-47f4-a19b-3e9f561d73fd is removed.
          2015-01-23 11:16:09,009 ERROR org.apache.tajo.master.TajoMaster: RECEIVED SIGNAL 15: SIGTERM
          2015-01-23 11:16:09,010 INFO org.apache.tajo.master.TajoMaster: ============================================
          2015-01-23 11:16:09,010 INFO org.apache.tajo.master.TajoMaster: TajoMaster received SIGINT Signal
          2015-01-23 11:16:09,010 INFO org.apache.tajo.master.TajoMaster: ============================================
          2015-01-23 11:16:09,013 INFO org.apache.tajo.catalog.CatalogServer: Catalog Server (127.0.0.1:26005) shutdown
          2015-01-23 11:16:09,020 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (CatalogProtocol) listened on 127.0.0.1:26005) shutdown
          2015-01-23 11:16:09,021 INFO org.apache.tajo.catalog.store.MySQLStore: Shutdown database (jdbc:mysql://localhost:3306/tajo?createDatabaseIfNotExist=true)
          2015-01-23 11:16:09,022 INFO org.apache.tajo.util.history.HistoryWriter: HistoryWriter_127.0.0.1_26001 stopped.
          2015-01-23 11:16:09,022 INFO org.apache.tajo.util.history.HistoryCleaner: History cleaner stopped
          2015-01-23 11:16:09,023 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (QueryCoordinatorProtocol) listened on 127.0.0.1:26001) shutdown
          2015-01-23 11:16:09,025 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (TajoMasterClientProtocol) listened on 127.0.0.1:26002) shutdown
          2015-01-23 11:16:09,026 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: SessionLivelinessMonitor thread interrupted
          2015-01-23 11:16:09,026 ERROR org.apache.tajo.master.rm.TajoWorkerResourceManager: java.lang.InterruptedException
          2015-01-23 11:16:09,027 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (TajoResourceTrackerProtocol) listened on 127.0.0.1:26003) shutdown
          2015-01-23 11:16:09,027 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: WorkerLivelinessMonitor thread interrupted
          2015-01-23 11:16:09,028 INFO org.apache.tajo.master.TajoMaster: Tajo Master main thread exiting
          2015-01-23 11:16:09,117 INFO org.apache.tajo.master.TajoMaster: SHUTDOWN_MSG:
          /************************************************************
          SHUTDOWN_MSG: Shutting down TajoMaster at jaehwaui-mbp.server.gruter.com/192.168.0.245
          ************************************************************/

          For reference, I could run Tajo HA cluster with Tajo command as follows:
          $TAJO_HOME/bin/tajo master

          Cheers
          Jaehwa

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71138119 Hi @hyunsik Thanks for your contribution and simple codes by ServiceTracker looks very pretty to me. But unfortunately, I couldn't run Tajo HA cluster with a shell script (start-tajo.sh) on my macbook as follows: 2015-01-23 11:15:46,876 INFO org.apache.tajo.master.rm.TajoWorkerResourceManager: Released QueryMaster (q_1421979321683_0001) resource. 2015-01-23 11:15:46,885 INFO org.apache.tajo.util.history.HistoryWriter: Create query history file: hdfs://localhost:9010/tmp/tajo-blrunner/staging/history/20150123/query-list/query-list-111546.hist 2015-01-23 11:15:57,739 INFO org.apache.tajo.session.SessionManager: Session f103d278-ab81-47f4-a19b-3e9f561d73fd is removed. 2015-01-23 11:16:09,009 ERROR org.apache.tajo.master.TajoMaster: RECEIVED SIGNAL 15: SIGTERM 2015-01-23 11:16:09,010 INFO org.apache.tajo.master.TajoMaster: ============================================ 2015-01-23 11:16:09,010 INFO org.apache.tajo.master.TajoMaster: TajoMaster received SIGINT Signal 2015-01-23 11:16:09,010 INFO org.apache.tajo.master.TajoMaster: ============================================ 2015-01-23 11:16:09,013 INFO org.apache.tajo.catalog.CatalogServer: Catalog Server (127.0.0.1:26005) shutdown 2015-01-23 11:16:09,020 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (CatalogProtocol) listened on 127.0.0.1:26005) shutdown 2015-01-23 11:16:09,021 INFO org.apache.tajo.catalog.store.MySQLStore: Shutdown database (jdbc:mysql://localhost:3306/tajo?createDatabaseIfNotExist=true) 2015-01-23 11:16:09,022 INFO org.apache.tajo.util.history.HistoryWriter: HistoryWriter_127.0.0.1_26001 stopped. 2015-01-23 11:16:09,022 INFO org.apache.tajo.util.history.HistoryCleaner: History cleaner stopped 2015-01-23 11:16:09,023 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (QueryCoordinatorProtocol) listened on 127.0.0.1:26001) shutdown 2015-01-23 11:16:09,025 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (TajoMasterClientProtocol) listened on 127.0.0.1:26002) shutdown 2015-01-23 11:16:09,026 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: SessionLivelinessMonitor thread interrupted 2015-01-23 11:16:09,026 ERROR org.apache.tajo.master.rm.TajoWorkerResourceManager: java.lang.InterruptedException 2015-01-23 11:16:09,027 INFO org.apache.tajo.rpc.NettyServerBase: Rpc (TajoResourceTrackerProtocol) listened on 127.0.0.1:26003) shutdown 2015-01-23 11:16:09,027 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: WorkerLivelinessMonitor thread interrupted 2015-01-23 11:16:09,028 INFO org.apache.tajo.master.TajoMaster: Tajo Master main thread exiting 2015-01-23 11:16:09,117 INFO org.apache.tajo.master.TajoMaster: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down TajoMaster at jaehwaui-mbp.server.gruter.com/192.168.0.245 ************************************************************/ For reference, I could run Tajo HA cluster with Tajo command as follows: $TAJO_HOME/bin/tajo master Cheers Jaehwa
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71138525

          For reference, after finding your comment, I'll check the patch such as JDBC, HA on YARN, auto-failover.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71138525 For reference, after finding your comment, I'll check the patch such as JDBC, HA on YARN, auto-failover.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71201445

          Thank you for your testing. I'll dig into the problem

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71201445 Thank you for your testing. I'll dig into the problem
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71309342

          @blrunner TajoMaster seems to be shutdown according to your log.

          Could you point out any abnormal line in the log? I could find only SIGTERM signal. It seems to be triggered from external something.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71309342 @blrunner TajoMaster seems to be shutdown according to your log. Could you point out any abnormal line in the log? I could find only SIGTERM signal. It seems to be triggered from external something.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71407049

          Sorry @hyunsik .

          I found that SIGTERM signal had happened by wrong configuration.
          I'm testing the patch with various environment.
          I'll give feedback as soon as possible.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71407049 Sorry @hyunsik . I found that SIGTERM signal had happened by wrong configuration. I'm testing the patch with various environment. I'll give feedback as soon as possible.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71423873

          I just fixed TajoWorkerClientService::getQueryHistory bug and triggered TravisCI.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71423873 I just fixed TajoWorkerClientService::getQueryHistory bug and triggered TravisCI.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71426866

          +1

          It looks good to me and the patch ran as expected on my testing cluster.
          Push it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71426866 +1 It looks good to me and the patch ran as expected on my testing cluster. Push it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71430098

          Hi @hyunsik

          The patch ran as expected on my testing cluster and all unit test cases finished successfully.
          But Travis CI build failed as follows:

              Running org.apache.tajo.ha.TestHAServiceHDFSImpl
              Formatting using clusterid: testClusterID
              2015-01-26 08:19:56,323 INFO: org.mortbay.log (info(67)) - jetty-6.1.14
              2015-01-26 08:19:56,329 INFO: org.mortbay.log (info(67)) - Extract jar:file:/home/travis/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/hdfs to /tmp/Jetty_localhost_60866_hdfs____ofqbhk/webapp
              2015-01-26 08:19:56,472 WARN: org.apache.hadoop.security.authentication.server.AuthenticationFilter (init(166)) - 'signature.secret' configuration not set, using a random value as secret
              2015-01-26 08:19:56,474 INFO: org.mortbay.log (info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60866
              2015-01-26 08:19:57,404 INFO: org.mortbay.log (info(67)) - jetty-6.1.14
              2015-01-26 08:19:57,412 INFO: org.mortbay.log (info(67)) - Extract jar:file:/home/travis/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/datanode to /tmp/Jetty_localhost_43395_datanode____y4jjpz/webapp
              2015-01-26 08:19:57,569 INFO: org.mortbay.log (info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43395
              2015-01-26 08:19:58,281 WARN: org.apache.tajo.storage.FileStorageManager (storageInit(100)) - does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
              2015-01-26 08:20:01,009 ERROR: org.apache.tajo.master.rm.TajoWorkerResourceManager (run(346)) - java.lang.InterruptedException
              2015-01-26 08:20:08,044 ERROR: org.apache.tajo.master.rm.TajoWorkerResourceManager (run(346)) - java.lang.InterruptedException
              2015-01-26 08:20:09,734 ERROR: org.apache.tajo.worker.WorkerHeartbeatService (run(210)) - org.apache.tajo.service.ServiceTrackerException: No active master entry
              org.apache.tajo.service.ServiceTrackerException: org.apache.tajo.service.ServiceTrackerException: No active master entry
              	at org.apache.tajo.ha.HdfsServiceTracker.getAddressElements(HdfsServiceTracker.java:436)
              	at org.apache.tajo.ha.HdfsServiceTracker.getResourceTrackerAddress(HdfsServiceTracker.java:361)
              	at org.apache.tajo.worker.WorkerHeartbeatService$WorkerHeartbeatThread.run(WorkerHeartbeatService.java:188)
              Caused by: org.apache.tajo.service.ServiceTrackerException: No active master entry
              	at org.apache.tajo.ha.HdfsServiceTracker.getAddressElements(HdfsServiceTracker.java:408)
              	... 2 more
              
          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71430098 Hi @hyunsik The patch ran as expected on my testing cluster and all unit test cases finished successfully. But Travis CI build failed as follows: Running org.apache.tajo.ha.TestHAServiceHDFSImpl Formatting using clusterid: testClusterID 2015-01-26 08:19:56,323 INFO: org.mortbay.log (info(67)) - jetty-6.1.14 2015-01-26 08:19:56,329 INFO: org.mortbay.log (info(67)) - Extract jar:file:/home/travis/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/hdfs to /tmp/Jetty_localhost_60866_hdfs____ofqbhk/webapp 2015-01-26 08:19:56,472 WARN: org.apache.hadoop.security.authentication.server.AuthenticationFilter (init(166)) - 'signature.secret' configuration not set, using a random value as secret 2015-01-26 08:19:56,474 INFO: org.mortbay.log (info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60866 2015-01-26 08:19:57,404 INFO: org.mortbay.log (info(67)) - jetty-6.1.14 2015-01-26 08:19:57,412 INFO: org.mortbay.log (info(67)) - Extract jar:file:/home/travis/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/datanode to /tmp/Jetty_localhost_43395_datanode____y4jjpz/webapp 2015-01-26 08:19:57,569 INFO: org.mortbay.log (info(67)) - Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43395 2015-01-26 08:19:58,281 WARN: org.apache.tajo.storage.FileStorageManager (storageInit(100)) - does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled') 2015-01-26 08:20:01,009 ERROR: org.apache.tajo.master.rm.TajoWorkerResourceManager (run(346)) - java.lang.InterruptedException 2015-01-26 08:20:08,044 ERROR: org.apache.tajo.master.rm.TajoWorkerResourceManager (run(346)) - java.lang.InterruptedException 2015-01-26 08:20:09,734 ERROR: org.apache.tajo.worker.WorkerHeartbeatService (run(210)) - org.apache.tajo.service.ServiceTrackerException: No active master entry org.apache.tajo.service.ServiceTrackerException: org.apache.tajo.service.ServiceTrackerException: No active master entry at org.apache.tajo.ha.HdfsServiceTracker.getAddressElements(HdfsServiceTracker.java:436) at org.apache.tajo.ha.HdfsServiceTracker.getResourceTrackerAddress(HdfsServiceTracker.java:361) at org.apache.tajo.worker.WorkerHeartbeatService$WorkerHeartbeatThread.run(WorkerHeartbeatService.java:188) Caused by: org.apache.tajo.service.ServiceTrackerException: No active master entry at org.apache.tajo.ha.HdfsServiceTracker.getAddressElements(HdfsServiceTracker.java:408) ... 2 more
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71713607

          Hi @blrunner ,

          That is a part of original unit tests; i.e.g, TestHAServiceHDFSImpl::testAutoFailOver. The failure is not related to this unit test because the following unit tests are passes well until the failure point. According to logs, the failure seems to be caused by HDFS mini cluster, which occurs occasionally. Anyway, I trigger the TravisCI.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71713607 Hi @blrunner , That is a part of original unit tests; i.e.g, TestHAServiceHDFSImpl::testAutoFailOver. The failure is not related to this unit test because the following unit tests are passes well until the failure point. According to logs, the failure seems to be caused by HDFS mini cluster, which occurs occasionally. Anyway, I trigger the TravisCI.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/358#issuecomment-71766970

          +1

          I think that the error cause was TAJO-1318.
          Shit it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/358#issuecomment-71766970 +1 I think that the error cause was TAJO-1318 . Shit it.
          Hide
          hyunsik Hyunsik Choi added a comment -

          committed.

          Show
          hyunsik Hyunsik Choi added a comment - committed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/358

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/358
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #569 (See https://builds.apache.org/job/Tajo-master-build/569/)
          TAJO-1306: HAServiceUtil should not directly use HDFS. (hyunsik: rev 4595375f7e6b62436e0d4bf88a8aef1ca680c726)

          • tajo-common/src/main/java/org/apache/tajo/service/ServiceTracker.java
          • tajo-common/src/main/java/org/apache/tajo/service/HAServiceTracker.java
          • tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java
          • tajo-core/src/main/resources/webapps/admin/cluster.jsp
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java
          • tajo-core/src/main/java/org/apache/tajo/ha/HAServiceHDFSImpl.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoContainerProxy.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoHAClientUtil.java
          • tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerFactory.java
          • tajo-common/src/main/java/org/apache/tajo/service/BaseServiceTracker.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
          • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
          • tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java
          • tajo-core/src/main/resources/webapps/admin/index.jsp
          • tajo-core/src/main/java/org/apache/tajo/ha/HAService.java
          • tajo-core/src/main/java/org/apache/tajo/worker/rule/ConnectivityCheckerRuleForTajoWorker.java
          • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/JdbcConnection.java
          • tajo-common/src/main/java/org/apache/tajo/ha/HAServiceUtil.java
          • tajo-client/src/main/java/org/apache/tajo/client/DummyServiceTracker.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/benchmark/BenchmarkSet.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageUtil.java
          • tajo-core/src/test/java/org/apache/tajo/ha/TestHAServiceHDFSImpl.java
          • tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java
          • tajo-core/src/main/resources/webapps/admin/query_executor.jsp
          • tajo-core/src/main/java/org/apache/tajo/ha/TajoMasterInfo.java
          • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoStatement.java
          • tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerException.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          • tajo-common/src/main/java/org/apache/tajo/service/TajoMasterInfo.java
          • tajo-core/src/main/java/org/apache/tajo/worker/WorkerHeartbeatService.java
          • tajo-core/src/main/resources/webapps/admin/catalogview.jsp
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java
          • tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #569 (See https://builds.apache.org/job/Tajo-master-build/569/ ) TAJO-1306 : HAServiceUtil should not directly use HDFS. (hyunsik: rev 4595375f7e6b62436e0d4bf88a8aef1ca680c726) tajo-common/src/main/java/org/apache/tajo/service/ServiceTracker.java tajo-common/src/main/java/org/apache/tajo/service/HAServiceTracker.java tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java tajo-core/src/main/resources/webapps/admin/cluster.jsp tajo-core/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java tajo-core/src/main/java/org/apache/tajo/ha/HAServiceHDFSImpl.java tajo-core/src/main/java/org/apache/tajo/master/TajoContainerProxy.java tajo-client/src/main/java/org/apache/tajo/client/TajoHAClientUtil.java tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerFactory.java tajo-common/src/main/java/org/apache/tajo/service/BaseServiceTracker.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java tajo-core/src/main/resources/webapps/admin/index.jsp tajo-core/src/main/java/org/apache/tajo/ha/HAService.java tajo-core/src/main/java/org/apache/tajo/worker/rule/ConnectivityCheckerRuleForTajoWorker.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/JdbcConnection.java tajo-common/src/main/java/org/apache/tajo/ha/HAServiceUtil.java tajo-client/src/main/java/org/apache/tajo/client/DummyServiceTracker.java tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java tajo-core/src/main/java/org/apache/tajo/benchmark/BenchmarkSet.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageUtil.java tajo-core/src/test/java/org/apache/tajo/ha/TestHAServiceHDFSImpl.java tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java tajo-core/src/main/resources/webapps/admin/query_executor.jsp tajo-core/src/main/java/org/apache/tajo/ha/TajoMasterInfo.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoStatement.java tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerException.java tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java tajo-common/src/main/java/org/apache/tajo/service/TajoMasterInfo.java tajo-core/src/main/java/org/apache/tajo/worker/WorkerHeartbeatService.java tajo-core/src/main/resources/webapps/admin/catalogview.jsp tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #208 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/208/)
          TAJO-1306: HAServiceUtil should not directly use HDFS. (hyunsik: rev 4595375f7e6b62436e0d4bf88a8aef1ca680c726)

          • tajo-core/src/main/java/org/apache/tajo/ha/HAServiceHDFSImpl.java
          • tajo-core/src/main/java/org/apache/tajo/ha/HAService.java
          • tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerException.java
          • tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerFactory.java
          • tajo-core/src/main/resources/webapps/admin/query_executor.jsp
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoContainerProxy.java
          • tajo-core/src/main/resources/webapps/admin/index.jsp
          • tajo-client/src/main/java/org/apache/tajo/client/TajoHAClientUtil.java
          • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageUtil.java
          • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoStatement.java
          • tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java
          • tajo-core/src/test/java/org/apache/tajo/ha/TestHAServiceHDFSImpl.java
          • tajo-core/src/main/java/org/apache/tajo/ha/TajoMasterInfo.java
          • tajo-core/src/main/java/org/apache/tajo/benchmark/BenchmarkSet.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java
          • tajo-core/src/main/resources/webapps/admin/cluster.jsp
          • tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java
          • tajo-common/src/main/java/org/apache/tajo/service/ServiceTracker.java
          • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/JdbcConnection.java
          • tajo-common/src/main/java/org/apache/tajo/ha/HAServiceUtil.java
          • tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java
          • tajo-common/src/main/java/org/apache/tajo/service/BaseServiceTracker.java
          • tajo-core/src/main/resources/webapps/admin/catalogview.jsp
          • tajo-client/src/main/java/org/apache/tajo/client/DummyServiceTracker.java
          • tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java
          • tajo-core/src/main/java/org/apache/tajo/worker/rule/ConnectivityCheckerRuleForTajoWorker.java
          • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
          • tajo-common/src/main/java/org/apache/tajo/service/TajoMasterInfo.java
          • tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java
          • tajo-core/src/main/java/org/apache/tajo/worker/WorkerHeartbeatService.java
          • tajo-common/src/main/java/org/apache/tajo/service/HAServiceTracker.java
          • tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java
          • tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #208 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/208/ ) TAJO-1306 : HAServiceUtil should not directly use HDFS. (hyunsik: rev 4595375f7e6b62436e0d4bf88a8aef1ca680c726) tajo-core/src/main/java/org/apache/tajo/ha/HAServiceHDFSImpl.java tajo-core/src/main/java/org/apache/tajo/ha/HAService.java tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerException.java tajo-common/src/main/java/org/apache/tajo/service/ServiceTrackerFactory.java tajo-core/src/main/resources/webapps/admin/query_executor.jsp tajo-core/src/main/java/org/apache/tajo/master/TajoMaster.java tajo-core/src/main/java/org/apache/tajo/master/TajoContainerProxy.java tajo-core/src/main/resources/webapps/admin/index.jsp tajo-client/src/main/java/org/apache/tajo/client/TajoHAClientUtil.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageUtil.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoStatement.java tajo-client/src/main/java/org/apache/tajo/client/SessionConnection.java tajo-core/src/test/java/org/apache/tajo/ha/TestHAServiceHDFSImpl.java tajo-core/src/main/java/org/apache/tajo/ha/TajoMasterInfo.java tajo-core/src/main/java/org/apache/tajo/benchmark/BenchmarkSet.java tajo-core/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java tajo-core/src/main/resources/webapps/admin/cluster.jsp tajo-core/src/main/java/org/apache/tajo/util/JSPUtil.java tajo-common/src/main/java/org/apache/tajo/service/ServiceTracker.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/JdbcConnection.java tajo-common/src/main/java/org/apache/tajo/ha/HAServiceUtil.java tajo-catalog/tajo-catalog-client/src/main/java/org/apache/tajo/catalog/AbstractCatalogClient.java tajo-core/src/main/java/org/apache/tajo/worker/TajoWorker.java tajo-common/src/main/java/org/apache/tajo/service/BaseServiceTracker.java tajo-core/src/main/resources/webapps/admin/catalogview.jsp tajo-client/src/main/java/org/apache/tajo/client/DummyServiceTracker.java tajo-core/src/main/java/org/apache/tajo/querymaster/QueryMaster.java tajo-core/src/main/java/org/apache/tajo/worker/rule/ConnectivityCheckerRuleForTajoWorker.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-common/src/main/java/org/apache/tajo/service/TajoMasterInfo.java tajo-core/src/main/java/org/apache/tajo/ha/HdfsServiceTracker.java tajo-core/src/main/java/org/apache/tajo/worker/WorkerHeartbeatService.java tajo-common/src/main/java/org/apache/tajo/service/HAServiceTracker.java tajo-client/src/main/java/org/apache/tajo/client/TajoClientImpl.java tajo-core/src/main/java/org/apache/tajo/worker/TajoWorkerClientService.java

            People

            • Assignee:
              hyunsik Hyunsik Choi
              Reporter:
              hyunsik Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development