Uploaded image for project: 'Phoenix Tephra'
  1. Phoenix Tephra
  2. TEPHRA-179

Tephra transaction manager breaks on zookeeper restart

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.8.0-incubating
    • 0.9.0-incubating
    • manager
    • None
    • OpenJDK 8 (JDK) on Alpine Linux 3.4 in Docker

    Description

      I am running HBase 1.2.2 with Phoenix 4.8.0 with the tephra transaction server in 1 docker container. In another docker container, I have Zookeeper 3.4.8 manage by Netflix Exhibitor.

      When everything first starts, I am able to create transactional table and run transactional queries.

      However, once Exhibitor restarts zookeeper and tephra reconnects to zookeeper, it no longer works correctly.

      Running transactional queries result in this error:

      Error: Error -1 (00000) : Error while executing SQL "CREATE TABLE my_table321 (k BIGINT PRIMARY KEY, v VARCHAR) TRANSACTIONAL=true": Remote driver error: RuntimeException: java.lang.Exception: Thrift error for org.apache.tephra.distributed.TransactionServiceClient$2@2361d7ab: Internal error processing startShort -> Exception: Thrift error for org.apache.tephra.distributed.TransactionServiceClient$2@2361d7ab: Internal error processing startShort -> TApplicationException: Internal error processing startShort
      SQLState:  00000
      ErrorCode: -1
      

      This is the full log:

      Fri Sep  2 00:26:50 UTC 2016 Starting tephra service on m9edd51-hmaster1.m9edd51
      -f: file size (blocks)             unlimited
      -t: cpu time (seconds)             unlimited
      -d: data seg size (kb)             unlimited
      -s: stack size (kb)                8192
      -c: core file size (blocks)        unlimited
      -m: resident set size (kb)         unlimited
      -l: locked memory (kb)             64
      -p: processes                      unlimited
      -n: file descriptors               65536
      -v: address space (kb)             unlimited
      -w: locks                          unlimited
      -e: scheduling priority            0
      -r: real-time priority             0
      Command:  /usr/lib/jvm/java-1.8-openjdk/bin/java -XX:+UseConcMarkSweepGC -cp /opt/hbase/bin/../lib/*:/opt/hbase/bin/../conf/:/opt/hbase/phoenix-client/target/*:/opt/hbase/conf:/usr/lib/jvm/java-1.8-openjdk/lib/tools.jar:/opt/hbase:/opt/hbase/lib/activation-1.1.jar:/opt/hbase/lib/antisamy-1.5.3.jar:/opt/hbase/lib/aopalliance-1.0.jar:/opt/hbase/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hbase/lib/api-asn1-api-1.0.0-M20.jar:/opt/hbase/lib/api-util-1.0.0-M20.jar:/opt/hbase/lib/asm-3.1.jar:/opt/hbase/lib/avro-1.7.4.jar:/opt/hbase/lib/batik-css-1.8.jar:/opt/hbase/lib/batik-ext-1.8.jar:/opt/hbase/lib/batik-util-1.8.jar:/opt/hbase/lib/bsh-core-2.0b4.jar:/opt/hbase/lib/commons-beanutils-1.7.0.jar:/opt/hbase/lib/commons-beanutils-core-1.8.3.jar:/opt/hbase/lib/commons-cli-1.2.jar:/opt/hbase/lib/commons-codec-1.9.jar:/opt/hbase/lib/commons-collections-3.2.2.jar:/opt/hbase/lib/commons-compress-1.4.1.jar:/opt/hbase/lib/commons-configuration-1.6.jar:/opt/hbase/lib/commons-daemon-1.0.13.jar:/opt/hbase/lib/commons-digester-1.8.jar:/opt/hbase/lib/commons-el-1.0.jar:/opt/hbase/lib/commons-fileupload-1.3.1.jar:/opt/hbase/lib/commons-httpclient-3.1.jar:/opt/hbase/lib/commons-io-2.4.jar:/opt/hbase/lib/commons-lang-2.6.jar:/opt/hbase/lib/commons-logging-1.2.jar:/opt/hbase/lib/commons-math-2.2.jar:/opt/hbase/lib/commons-math3-3.1.1.jar:/opt/hbase/lib/commons-net-3.1.jar:/opt/hbase/lib/disruptor-3.3.0.jar:/opt/hbase/lib/esapi-2.1.0.1.jar:/opt/hbase/lib/findbugs-annotations-1.3.9-1.jar:/opt/hbase/lib/guava-12.0.1.jar:/opt/hbase/lib/guice-3.0.jar:/opt/hbase/lib/guice-servlet-3.0.jar:/opt/hbase/lib/hadoop-annotations-2.5.1.jar:/opt/hbase/lib/hadoop-auth-2.5.1.jar:/opt/hbase/lib/hadoop-client-2.5.1.jar:/opt/hbase/lib/hadoop-common-2.5.1.jar:/opt/hbase/lib/hadoop-hdfs-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-app-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-common-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-core-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-client-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-common-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-server-common-2.5.1.jar:/opt/hbase/lib/hbase-annotations-1.2.2-tests.jar:/opt/hbase/lib/hbase-annotations-1.2.2.jar:/opt/hbase/lib/hbase-client-1.2.2.jar:/opt/hbase/lib/hbase-common-1.2.2-tests.jar:/opt/hbase/lib/hbase-common-1.2.2.jar:/opt/hbase/lib/hbase-examples-1.2.2.jar:/opt/hbase/lib/hbase-external-blockcache-1.2.2.jar:/opt/hbase/lib/hbase-hadoop-compat-1.2.2.jar:/opt/hbase/lib/hbase-hadoop2-compat-1.2.2.jar:/opt/hbase/lib/hbase-it-1.2.2-tests.jar:/opt/hbase/lib/hbase-it-1.2.2.jar:/opt/hbase/lib/hbase-prefix-tree-1.2.2.jar:/opt/hbase/lib/hbase-procedure-1.2.2.jar:/opt/hbase/lib/hbase-protocol-1.2.2.jar:/opt/hbase/lib/hbase-resource-bundle-1.2.2.jar:/opt/hbase/lib/hbase-rest-1.2.2.jar:/opt/hbase/lib/hbase-server-1.2.2-tests.jar:/opt/hbase/lib/hbase-server-1.2.2.jar:/opt/hbase/lib/hbase-shell-1.2.2.jar:/opt/hbase/lib/hbase-thrift-1.2.2.jar:/opt/hbase/lib/htrace-core-3.1.0-incubating.jar:/opt/hbase/lib/httpclient-4.2.5.jar:/opt/hbase/lib/httpcore-4.4.1.jar:/opt/hbase/lib/jackson-core-asl-1.9.13.jar:/opt/hbase/lib/jackson-jaxrs-1.9.13.jar:/opt/hbase/lib/jackson-mapper-asl-1.9.13.jar:/opt/hbase/lib/jackson-xc-1.9.13.jar:/opt/hbase/lib/jamon-runtime-2.4.1.jar:/opt/hbase/lib/jasper-compiler-5.5.23.jar:/opt/hbase/lib/jasper-runtime-5.5.23.jar:/opt/hbase/lib/java-xmlbuilder-0.4.jar:/opt/hbase/lib/javax.inject-1.jar:/opt/hbase/lib/jaxb-api-2.2.2.jar:/opt/hbase/lib/jaxb-impl-2.2.3-1.jar:/opt/hbase/lib/jcodings-1.0.8.jar:/opt/hbase/lib/jersey-client-1.9.jar:/opt/hbase/lib/jersey-core-1.9.jar:/opt/hbase/lib/jersey-guice-1.9.jar:/opt/hbase/lib/jersey-json-1.9.jar:/opt/hbase/lib/jersey-server-1.9.jar:/opt/hbase/lib/jets3t-0.9.0.jar:/opt/hbase/lib/jettison-1.3.3.jar:/opt/hbase/lib/jetty-6.1.26.jar:/opt/hbase/lib/jetty-sslengine-6.1.26.jar:/opt/hbase/lib/jetty-util-6.1.26.jar:/opt/hbase/lib/joni-2.1.2.jar:/opt/hbase/lib/jruby-complete-1.6.8.jar:/opt/hbase/lib/jsch-0.1.42.jar:/opt/hbase/lib/jsp-2.1-6.1.14.jar:/opt/hbase/lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/lib/jsr305-1.3.9.jar:/opt/hbase/lib/junit-4.12.jar:/opt/hbase/lib/leveldbjni-all-1.8.jar:/opt/hbase/lib/libthrift-0.9.3.jar:/opt/hbase/lib/log4j-1.2.17.jar:/opt/hbase/lib/metrics-core-2.2.0.jar:/opt/hbase/lib/nekohtml-1.9.16.jar:/opt/hbase/lib/netty-all-4.0.23.Final.jar:/opt/hbase/lib/paranamer-2.3.jar:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-client.jar:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-server.jar:/opt/hbase/lib/protobuf-java-2.5.0.jar:/opt/hbase/lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/lib/servlet-api-2.5.jar:/opt/hbase/lib/slf4j-api-1.7.7.jar:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/lib/snappy-java-1.0.4.1.jar:/opt/hbase/lib/spymemcached-2.11.6.jar:/opt/hbase/lib/xalan-2.7.0.jar:/opt/hbase/lib/xml-apis-1.3.03.jar:/opt/hbase/lib/xml-apis-ext-1.3.04.jar:/opt/hbase/lib/xmlenc-0.52.jar:/opt/hbase/lib/xom-1.2.5.jar:/opt/hbase/lib/xz-1.0.jar:/opt/hbase/lib/zookeeper-3.4.6.jar:: org.apache.tephra.TransactionServiceMain
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      2016-09-02 00:26:51,571 INFO  [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
      2016-09-02 00:26:51,571 INFO  [main] zookeeper.ZooKeeper: Client environment:host.name=m9edd51-hmaster1.m9edd51
      2016-09-02 00:26:51,571 INFO  [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_92-internal
      2016-09-02 00:26:51,571 INFO  [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
      2016-09-02 00:26:51,571 INFO  [main] zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.8-openjdk/jre
      2016-09-02 00:26:51,571 INFO  [main] zookeeper.ZooKeeper: Client environment:java.class.path=/opt/hbase/bin/../lib/phoenix-4.8.0-HBase-1.2-server.jar:/opt/hbase/bin/../lib/phoenix-4.8.0-HBase-1.2-client.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/opt/hbase/bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hbase/bin/../lib/httpcore-4.4.1.jar:/opt/hbase/bin/../lib/libthrift-0.9.3.jar:/opt/hbase/bin/../lib/hbase-hadoop2-compat-1.2.2.jar:/opt/hbase/bin/../lib/hadoop-hdfs-2.5.1.jar:/opt/hbase/bin/../lib/hbase-server-1.2.2-tests.jar:/opt/hbase/bin/../lib/commons-collections-3.2.2.jar:/opt/hbase/bin/../lib/commons-compress-1.4.1.jar:/opt/hbase/bin/../lib/jackson-jaxrs-1.9.13.jar:/opt/hbase/bin/../lib/commons-beanutils-1.7.0.jar:/opt/hbase/bin/../lib/jetty-util-6.1.26.jar:/opt/hbase/bin/../lib/hbase-annotations-1.2.2.jar:/opt/hbase/bin/../lib/jetty-6.1.26.jar:/opt/hbase/bin/../lib/jackson-core-asl-1.9.13.jar:/opt/hbase/bin/../lib/disruptor-3.3.0.jar:/opt/hbase/bin/../lib/commons-cli-1.2.jar:/opt/hbase/bin/../lib/hbase-thrift-1.2.2.jar:/opt/hbase/bin/../lib/xalan-2.7.0.jar:/opt/hbase/bin/../lib/hbase-shell-1.2.2.jar:/opt/hbase/bin/../lib/jackson-xc-1.9.13.jar:/opt/hbase/bin/../lib/activation-1.1.jar:/opt/hbase/bin/../lib/hadoop-yarn-server-common-2.5.1.jar:/opt/hbase/bin/../lib/jersey-server-1.9.jar:/opt/hbase/bin/../lib/commons-net-3.1.jar:/opt/hbase/bin/../lib/xmlenc-0.52.jar:/opt/hbase/bin/../lib/batik-ext-1.8.jar:/opt/hbase/bin/../lib/jersey-guice-1.9.jar:/opt/hbase/bin/../lib/spymemcached-2.11.6.jar:/opt/hbase/bin/../lib/bsh-core-2.0b4.jar:/opt/hbase/bin/../lib/hbase-server-1.2.2.jar:/opt/hbase/bin/../lib/hbase-rest-1.2.2.jar:/opt/hbase/bin/../lib/guava-12.0.1.jar:/opt/hbase/bin/../lib/jetty-sslengine-6.1.26.jar:/opt/hbase/bin/../lib/xz-1.0.jar:/opt/hbase/bin/../lib/jsch-0.1.42.jar:/opt/hbase/bin/../lib/jackson-mapper-asl-1.9.13.jar:/opt/hbase/bin/../lib/hbase-examples-1.2.2.jar:/opt/hbase/bin/../lib/hbase-annotations-1.2.2-tests.jar:/opt/hbase/bin/../lib/hbase-resource-bundle-1.2.2.jar:/opt/hbase/bin/../lib/snappy-java-1.0.4.1.jar:/opt/hbase/bin/../lib/commons-daemon-1.0.13.jar:/opt/hbase/bin/../lib/commons-el-1.0.jar:/opt/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/bin/../lib/hadoop-client-2.5.1.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-core-2.5.1.jar:/opt/hbase/bin/../lib/servlet-api-2.5.jar:/opt/hbase/bin/../lib/httpclient-4.2.5.jar:/opt/hbase/bin/../lib/joni-2.1.2.jar:/opt/hbase/bin/../lib/hadoop-yarn-client-2.5.1.jar:/opt/hbase/bin/../lib/jaxb-api-2.2.2.jar:/opt/hbase/bin/../lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/bin/../lib/commons-digester-1.8.jar:/opt/hbase/bin/../lib/jersey-core-1.9.jar:/opt/hbase/bin/../lib/hbase-procedure-1.2.2.jar:/opt/hbase/bin/../lib/slf4j-api-1.7.7.jar:/opt/hbase/bin/../lib/commons-logging-1.2.jar:/opt/hbase/bin/../lib/hbase-it-1.2.2.jar:/opt/hbase/bin/../lib/guice-3.0.jar:/opt/hbase/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/opt/hbase/bin/../lib/findbugs-annotations-1.3.9-1.jar:/opt/hbase/bin/../lib/api-util-1.0.0-M20.jar:/opt/hbase/bin/../lib/asm-3.1.jar:/opt/hbase/bin/../lib/hbase-hadoop-compat-1.2.2.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-common-2.5.1.jar:/opt/hbase/bin/../lib/hbase-common-1.2.2.jar:/opt/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/opt/hbase/bin/../lib/nekohtml-1.9.16.jar:/opt/hbase/bin/../lib/java-xmlbuilder-0.4.jar:/opt/hbase/bin/../lib/hbase-protocol-1.2.2.jar:/opt/hbase/bin/../lib/esapi-2.1.0.1.jar:/opt/hbase/bin/../lib/junit-4.12.jar:/opt/hbase/bin/../lib/commons-fileupload-1.3.1.jar:/opt/hbase/bin/../lib/jamon-runtime-2.4.1.jar:/opt/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/bin/../lib/hbase-it-1.2.2-tests.jar:/opt/hbase/bin/../lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/bin/../lib/xom-1.2.5.jar:/opt/hbase/bin/../lib/avro-1.7.4.jar:/opt/hbase/bin/../lib/hadoop-common-2.5.1.jar:/opt/hbase/bin/../lib/leveldbjni-all-1.8.jar:/opt/hbase/bin/../lib/commons-io-2.4.jar:/opt/hbase/bin/../lib/batik-css-1.8.jar:/opt/hbase/bin/../lib/commons-codec-1.9.jar:/opt/hbase/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/hbase/bin/../lib/hadoop-yarn-common-2.5.1.jar:/opt/hbase/bin/../lib/javax.inject-1.jar:/opt/hbase/bin/../lib/commons-httpclient-3.1.jar:/opt/hbase/bin/../lib/commons-configuration-1.6.jar:/opt/hbase/bin/../lib/zookeeper-3.4.6.jar:/opt/hbase/bin/../lib/jets3t-0.9.0.jar:/opt/hbase/bin/../lib/guice-servlet-3.0.jar:/opt/hbase/bin/../lib/jersey-client-1.9.jar:/opt/hbase/bin/../lib/hbase-common-1.2.2-tests.jar:/opt/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-app-2.5.1.jar:/opt/hbase/bin/../lib/aopalliance-1.0.jar:/opt/hbase/bin/../lib/antisamy-1.5.3.jar:/opt/hbase/bin/../lib/jsr305-1.3.9.jar:/opt/hbase/bin/../lib/metrics-core-2.2.0.jar:/opt/hbase/bin/../lib/xml-apis-1.3.03.jar:/opt/hbase/bin/../lib/protobuf-java-2.5.0.jar:/opt/hbase/bin/../lib/commons-math-2.2.jar:/opt/hbase/bin/../lib/jettison-1.3.3.jar:/opt/hbase/bin/../lib/netty-all-4.0.23.Final.jar:/opt/hbase/bin/../lib/paranamer-2.3.jar:/opt/hbase/bin/../lib/xml-apis-ext-1.3.04.jar:/opt/hbase/bin/../lib/log4j-1.2.17.jar:/opt/hbase/bin/../lib/jersey-json-1.9.jar:/opt/hbase/bin/../lib/jruby-complete-1.6.8.jar:/opt/hbase/bin/../lib/hbase-prefix-tree-1.2.2.jar:/opt/hbase/bin/../lib/commons-math3-3.1.1.jar:/opt/hbase/bin/../lib/jcodings-1.0.8.jar:/opt/hbase/bin/../lib/api-asn1-api-1.0.0-M20.jar:/opt/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/opt/hbase/bin/../lib/commons-lang-2.6.jar:/opt/hbase/bin/../lib/hadoop-annotations-2.5.1.jar:/opt/hbase/bin/../lib/commons-beanutils-core-1.8.3.jar:/opt/hbase/bin/../lib/hbase-client-1.2.2.jar:/opt/hbase/bin/../lib/hadoop-auth-2.5.1.jar:/opt/hbase/bin/../lib/batik-util-1.8.jar:/opt/hbase/bin/../lib/htrace-core-3.1.0-incubating.jar:/opt/hbase/bin/../lib/hbase-external-blockcache-1.2.2.jar:/opt/hbase/bin/../conf/:/opt/hbase/phoenix-client/target/*:/opt/hbase/conf:/usr/lib/jvm/java-1.8-openjdk/lib/tools.jar:/opt/hbase:/opt/hbase/lib/activation-1.1.jar:/opt/hbase/lib/antisamy-1.5.3.jar:/opt/hbase/lib/aopalliance-1.0.jar:/opt/hbase/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hbase/lib/api-asn1-api-1.0.0-M20.jar:/opt/hbase/lib/api-util-1.0.0-M20.jar:/opt/hbase/lib/asm-3.1.jar:/opt/hbase/lib/avro-1.7.4.jar:/opt/hbase/lib/batik-css-1.8.jar:/opt/hbase/lib/batik-ext-1.8.jar:/opt/hbase/lib/batik-util-1.8.jar:/opt/hbase/lib/bsh-core-2.0b4.jar:/opt/hbase/lib/commons-beanutils-1.7.0.jar:/opt/hbase/lib/commons-beanutils-core-1.8.3.jar:/opt/hbase/lib/commons-cli-1.2.jar:/opt/hbase/lib/commons-codec-1.9.jar:/opt/hbase/lib/commons-collections-3.2.2.jar:/opt/hbase/lib/commons-compress-1.4.1.jar:/opt/hbase/lib/commons-configuration-1.6.jar:/opt/hbase/lib/commons-daemon-1.0.13.jar:/opt/hbase/lib/commons-digester-1.8.jar:/opt/hbase/lib/commons-el-1.0.jar:/opt/hbase/lib/commons-fileupload-1.3.1.jar:/opt/hbase/lib/commons-httpclient-3.1.jar:/opt/hbase/lib/commons-io-2.4.jar:/opt/hbase/lib/commons-lang-2.6.jar:/opt/hbase/lib/commons-logging-1.2.jar:/opt/hbase/lib/commons-math-2.2.jar:/opt/hbase/lib/commons-math3-3.1.1.jar:/opt/hbase/lib/commons-net-3.1.jar:/opt/hbase/lib/disruptor-3.3.0.jar:/opt/hbase/lib/esapi-2.1.0.1.jar:/opt/hbase/lib/findbugs-annotations-1.3.9-1.jar:/opt/hbase/lib/guava-12.0.1.jar:/opt/hbase/lib/guice-3.0.jar:/opt/hbase/lib/guice-servlet-3.0.jar:/opt/hbase/lib/hadoop-annotations-2.5.1.jar:/opt/hbase/lib/hadoop-auth-2.5.1.jar:/opt/hbase/lib/hadoop-client-2.5.1.jar:/opt/hbase/lib/hadoop-common-2.5.1.jar:/opt/hbase/lib/hadoop-hdfs-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-app-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-common-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-core-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-client-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-common-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-server-common-2.5.1.jar:/opt/hbase/lib/hbase-annotations-1.2.2-tests.jar:/opt/hbase/lib/hbase-annotations-1.2.2.jar:/opt/hbase/lib/hbase-client-1.2.2.jar:/opt/hbase/lib/hbase-common-1.2.2-tests.jar:/opt/hbase/lib/hbase-common-1.2.2.jar:/opt/hbase/lib/hbase-examples-1.2.2.jar:/opt/hbase/lib/hbase-external-blockcache-1.2.2.jar:/opt/hbase/lib/hbase-hadoop-compat-1.2.2.jar:/opt/hbase/lib/hbase-hadoop2-compat-1.2.2.jar:/opt/hbase/lib/hbase-it-1.2.2-tests.jar:/opt/hbase/lib/hbase-it-1.2.2.jar:/opt/hbase/lib/hbase-prefix-tree-1.2.2.jar:/opt/hbase/lib/hbase-procedure-1.2.2.jar:/opt/hbase/lib/hbase-protocol-1.2.2.jar:/opt/hbase/lib/hbase-resource-bundle-1.2.2.jar:/opt/hbase/lib/hbase-rest-1.2.2.jar:/opt/hbase/lib/hbase-server-1.2.2-tests.jar:/opt/hbase/lib/hbase-server-1.2.2.jar:/opt/hbase/lib/hbase-shell-1.2.2.jar:/opt/hbase/lib/hbase-thrift-1.2.2.jar:/opt/hbase/lib/htrace-core-3.1.0-incubating.jar:/opt/hbase/lib/httpclient-4.2.5.jar:/opt/hbase/lib/httpcore-4.4.1.jar:/opt/hbase/lib/jackson-core-asl-1.9.13.jar:/opt/hbase/lib/jackson-jaxrs-1.9.13.jar:/opt/hbase/lib/jackson-mapper-asl-1.9.13.jar:/opt/hbase/lib/jackson-xc-1.9.13.jar:/opt/hbase/lib/jamon-runtime-2.4.1.jar:/opt/hbase/lib/jasper-compiler-5.5.23.jar:/opt/hbase/lib/jasper-runtime-5.5.23.jar:/opt/hbase/lib/java-xmlbuilder-0.4.jar:/opt/hbase/lib/javax.inject-1.jar:/opt/hbase/lib/jaxb-api-2.2.2.jar:/opt/hbase/lib/jaxb-impl-2.2.3-1.jar:/opt/hbase/lib/jcodings-1.0.8.jar:/opt/hbase/lib/jersey-client-1.9.jar:/opt/hbase/lib/jersey-core-1.9.jar:/opt/hbase/lib/jersey-guice-1.9.jar:/opt/hbase/lib/jersey-json-1.9.jar:/opt/hbase/lib/jersey-server-1.9.jar:/opt/hbase/lib/jets3t-0.9.0.jar:/opt/hbase/lib/jettison-1.3.3.jar:/opt/hbase/lib/jetty-6.1.26.jar:/opt/hbase/lib/jetty-sslengine-6.1.26.jar:/opt/hbase/lib/jetty-util-6.1.26.jar:/opt/hbase/lib/joni-2.1.2.jar:/opt/hbase/lib/jruby-complete-1.6.8.jar:/opt/hbase/lib/jsch-0.1.42.jar:/opt/hbase/lib/jsp-2.1-6.1.14.jar:/opt/hbase/lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/lib/jsr305-1.3.9.jar:/opt/hbase/lib/junit-4.12.jar:/opt/hbase/lib/leveldbjni-all-1.8.jar:/opt/hbase/lib/libthrift-0.9.3.jar:/opt/hbase/lib/log4j-1.2.17.jar:/opt/hbase/lib/metrics-core-2.2.0.jar:/opt/hbase/lib/nekohtml-1.9.16.jar:/opt/hbase/lib/netty-all-4.0.23.Final.jar:/opt/hbase/lib/paranamer-2.3.jar:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-client.jar:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-server.jar:/opt/hbase/lib/protobuf-java-2.5.0.jar:/opt/hbase/lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/lib/servlet-api-2.5.jar:/opt/hbase/lib/slf4j-api-1.7.7.jar:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/lib/snappy-java-1.0.4.1.jar:/opt/hbase/lib/spymemcached-2.11.6.jar:/opt/hbase/lib/xalan-2.7.0.jar:/opt/hbase/lib/xml-apis-1.3.03.jar:/opt/hbase/lib/xml-apis-ext-1.3.04.jar:/opt/hbase/lib/xmlenc-0.52.jar:/opt/hbase/lib/xom-1.2.5.jar:/opt/hbase/lib/xz-1.0.jar:/opt/hbase/lib/zookeeper-3.4.6.jar::
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:os.name=Linux
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:os.version=4.4.0-36-generic
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:user.name=hbase
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:user.home=/opt/hbase
      2016-09-02 00:26:51,572 INFO  [main] zookeeper.ZooKeeper: Client environment:user.dir=/opt/hbase
      2016-09-02 00:26:51,573 INFO  [main] zookeeper.ZooKeeper: Initiating client connection, connectString=m9edd51-zookeeper.m9edd51 sessionTimeout=90000 watcher=org.apache.tephra.zookeeper.TephraZKClientService$5@45c7e403
      2016-09-02 00:26:51,596 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Opening socket connection to server m9edd51-zookeeper.m9edd51/172.18.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
      2016-09-02 00:26:51,600 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Socket connection established to m9edd51-zookeeper.m9edd51/172.18.0.2:2181, initiating session
      2016-09-02 00:26:51,605 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Session establishment complete on server m9edd51-zookeeper.m9edd51/172.18.0.2:2181, sessionid = 0x156e849e51c0007, negotiated timeout = 40000
      2016-09-02 00:26:51,625 INFO  [main] inmemory.InMemoryTransactionService: Configuring TransactionService, address: 0.0.0.0, port: 15165, threads: 20, io threads: 2, max read buffer (bytes): 16777216
      2016-09-02 00:26:51,626 INFO  [main] tephra.TransactionServiceMain: Starting TransactionServiceMain
      2016-09-02 00:26:51,633 INFO  [main] zookeeper.LeaderElection: Start leader election on m9edd51-zookeeper.m9edd51/tx.service/leader with guid aa0cbcc9-5267-4618-9059-bb90a1e7e948
      2016-09-02 00:26:51,694 INFO  [leader-election-tx.service-leader] metrics.DefaultMetricsCollector: Configured metrics report to emit every 60 seconds
      2016-09-02 00:26:51,863 INFO  [ThriftRPCServer] tephra.TransactionManager: Starting transaction manager.
      2016-09-02 00:26:51,874 INFO  [DefaultMetricsCollector STARTING] metrics.DefaultMetricsCollector: Started metrics reporter
      2016-09-02 00:26:51,954 WARN  [HDFSTransactionStateStorage STARTING] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      2016-09-02 00:26:52,554 INFO  [HDFSTransactionStateStorage STARTING] persist.HDFSTransactionStateStorage: Using snapshot dir /tmp/tephra/snapshots
      2016-09-02 00:26:52,594 INFO  [ThriftRPCServer] persist.HDFSTransactionStateStorage: Creating snapshot dir at /tmp/tephra/snapshots
      2016-09-02 00:26:52,620 INFO  [ThriftRPCServer] persist.HDFSTransactionStateStorage: No snapshot files found in /tmp/tephra/snapshots
      2016-09-02 00:26:52,624 INFO  [ThriftRPCServer] tephra.TransactionManager: Starting periodic timed-out transaction cleanup every 10 seconds with default timeout of 60 seconds.
      2016-09-02 00:26:52,624 INFO  [ThriftRPCServer] tephra.TransactionManager: Starting periodic snapshot thread, frequency = 300 seconds, location = /tmp/tephra/snapshots
      2016-09-02 00:26:52,624 INFO  [ThriftRPCServer] tephra.TransactionManager: Starting periodic Metrics Emitter thread, frequency = 10000
      2016-09-02 00:26:52,631 INFO  [ThriftRPCServer] rpc.ThriftRPCServer: Starting RPC server for TTransactionServer
      2016-09-02 00:26:52,641 INFO  [ThriftRPCServer] rpc.ThriftRPCServer: Running RPC server for TTransactionServer
      2016-09-02 00:26:52,641 INFO  [ThriftRPCServer] server.TThreadedSelectorServerWithFix: Starting TThreadedSelectorServerWithFix
      2016-09-02 00:26:52,720 INFO  [leader-election-tx.service-leader] distributed.TransactionService: Transaction Thrift Service started successfully on m9edd51-hmaster1.m9edd51/172.18.0.12:15165
      2016-09-02 00:27:38,642 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x156e849e51c0007, likely server has closed socket, closing socket connection and attempting reconnect
      2016-09-02 00:27:38,746 INFO  [leader-election-tx.service-leader] zookeeper.LeaderElection: Disconnected from ZK: m9edd51-zookeeper.m9edd51 for /tx.service/leader
      2016-09-02 00:27:38,746 INFO  [leader-election-tx.service-leader] zookeeper.LeaderElection: Stepping down as leader due to disconnect: m9edd51-zookeeper.m9edd51 for /tx.service/leader
      2016-09-02 00:27:38,747 INFO  [leader-election-tx.service-leader] rpc.ThriftRPCServer: Request to stop RPC server for TTransactionServer
      2016-09-02 00:27:38,751 INFO  [ThriftRPCServer] rpc.ThriftRPCServer: Done running RPC server for TTransactionServer
      2016-09-02 00:27:38,751 INFO  [ThriftRPCServer] tephra.TransactionManager: Shutting down gracefully...
      2016-09-02 00:27:38,752 INFO  [tx-clean-timeout] tephra.TransactionManager: Exiting thread tx-clean-timeout
      2016-09-02 00:27:38,752 INFO  [tx-metrics] tephra.TransactionManager: Exiting thread tx-metrics
      2016-09-02 00:27:38,752 INFO  [tx-snapshot] tephra.TransactionManager: Writing final snapshot prior to shutdown
      2016-09-02 00:27:38,884 INFO  [tx-snapshot] tephra.TransactionManager: Exiting thread tx-snapshot
      2016-09-02 00:27:38,992 INFO  [DefaultMetricsCollector STOPPING] metrics.DefaultMetricsCollector: Stopped metrics reporter
      2016-09-02 00:27:38,992 INFO  [ThriftRPCServer] tephra.TransactionManager: Took 240.2 ms to stop
      2016-09-02 00:27:38,993 INFO  [ThriftRPCServer] rpc.ThriftRPCServer: RPC server for TTransactionServer stopped.
      2016-09-02 00:27:39,013 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Opening socket connection to server m9edd51-zookeeper.m9edd51/fe80:0:0:0:0:242:ac12:2:2181. Will not attempt to authenticate using SASL (unknown error)
      2016-09-02 00:27:39,014 WARN  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Session 0x156e849e51c0007 for server null, unexpected error, closing socket connection and attempting reconnect
      java.net.ConnectException: Connection refused
              at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
              at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
              at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
              at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
      2016-09-02 00:27:40,462 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Opening socket connection to server m9edd51-zookeeper.m9edd51/172.18.0.2:2181. Will not attempt to authenticate using SASL (unknown error)
      2016-09-02 00:27:40,463 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Socket connection established to m9edd51-zookeeper.m9edd51/172.18.0.2:2181, initiating session
      2016-09-02 00:27:40,464 INFO  [main-SendThread(m9edd51-zookeeper.m9edd51:2181)] zookeeper.ClientCnxn: Session establishment complete on server m9edd51-zookeeper.m9edd51/172.18.0.2:2181, sessionid = 0x156e849e51c0007, negotiated timeout = 40000
      2016-09-02 00:27:40,469 INFO  [leader-election-tx.service-leader] zookeeper.LeaderElection: Connected to ZK, running election: m9edd51-zookeeper.m9edd51 for /tx.service/leader
      2016-09-02 00:27:40,502 INFO  [ThriftRPCServer] rpc.ThriftRPCServer: Starting RPC server for TTransactionServer
      2016-09-02 00:27:40,502 INFO  [ThriftRPCServer] rpc.ThriftRPCServer: Running RPC server for TTransactionServer
      2016-09-02 00:27:40,502 INFO  [ThriftRPCServer] server.TThreadedSelectorServerWithFix: Starting TThreadedSelectorServerWithFix
      2016-09-02 00:27:40,509 INFO  [leader-election-tx.service-leader] distributed.TransactionService: Transaction Thrift Service started successfully on m9edd51-hmaster1.m9edd51/172.18.0.12:15165
      2016-09-02 00:28:52,892 ERROR [TTransactionServer-rpc-0] thrift.ProcessFunction: Internal error processing startShort
      java.lang.IllegalStateException: Transaction Manager is not running.
              at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
              at org.apache.tephra.TransactionManager.ensureAvailable(TransactionManager.java:709)
              at org.apache.tephra.TransactionManager.startTx(TransactionManager.java:768)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:728)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:716)
              at org.apache.tephra.distributed.TransactionServiceThriftHandler.startShort(TransactionServiceThriftHandler.java:71)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer.java:261)
              at com.sun.proxy.$Proxy17.startShort(Unknown Source)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:974)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:959)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
              at org.apache.thrift.server.Invocation.run(Invocation.java:18)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      2016-09-02 00:28:53,297 ERROR [TTransactionServer-rpc-0] thrift.ProcessFunction: Internal error processing startShort
      java.lang.IllegalStateException: Transaction Manager is not running.
              at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
              at org.apache.tephra.TransactionManager.ensureAvailable(TransactionManager.java:709)
              at org.apache.tephra.TransactionManager.startTx(TransactionManager.java:768)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:728)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:716)
              at org.apache.tephra.distributed.TransactionServiceThriftHandler.startShort(TransactionServiceThriftHandler.java:71)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer.java:261)
              at com.sun.proxy.$Proxy17.startShort(Unknown Source)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:974)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:959)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
              at org.apache.thrift.server.Invocation.run(Invocation.java:18)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      2016-09-02 00:28:54,901 ERROR [TTransactionServer-rpc-0] thrift.ProcessFunction: Internal error processing startShort
      java.lang.IllegalStateException: Transaction Manager is not running.
              at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
              at org.apache.tephra.TransactionManager.ensureAvailable(TransactionManager.java:709)
              at org.apache.tephra.TransactionManager.startTx(TransactionManager.java:768)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:728)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:716)
              at org.apache.tephra.distributed.TransactionServiceThriftHandler.startShort(TransactionServiceThriftHandler.java:71)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer.java:261)
              at com.sun.proxy.$Proxy17.startShort(Unknown Source)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:974)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:959)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
              at org.apache.thrift.server.Invocation.run(Invocation.java:18)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      2016-09-02 00:29:01,313 ERROR [TTransactionServer-rpc-0] thrift.ProcessFunction: Internal error processing startShort
      java.lang.IllegalStateException: Transaction Manager is not running.
              at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
              at org.apache.tephra.TransactionManager.ensureAvailable(TransactionManager.java:709)
              at org.apache.tephra.TransactionManager.startTx(TransactionManager.java:768)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:728)
              at org.apache.tephra.TransactionManager.startShort(TransactionManager.java:716)
              at org.apache.tephra.distributed.TransactionServiceThriftHandler.startShort(TransactionServiceThriftHandler.java:71)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer.java:261)
              at com.sun.proxy.$Proxy17.startShort(Unknown Source)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:974)
              at org.apache.tephra.distributed.thrift.TTransactionServer$Processor$startShort.getResult(TTransactionServer.java:959)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
              at org.apache.thrift.server.Invocation.run(Invocation.java:18)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      Although it thinks that the transaction manager is not running, I can still see that it is running using jps:

      bash-4.3# jps
      138 HMaster
      974 Jps
      191 TransactionServiceMain
      

      Attachments

        Issue Links

          Activity

            People

              anwar1 Ali Anwar
              francischuang Francis Chuang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: