Uploaded image for project: 'Joshua (Retired)'
  1. Joshua (Retired)
  2. JOSHUA-218

Train and testing quickstart Indian model fails in interactive shell

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Reopened
    • Resolution: Unresolved
    • None
    • 6.2
    • None
    • None

    Description

      I am unable to follow the quickstart when attempting to verify that Joshua is installed and operating correctly from within the brew Formula.
      Logging output below details the commands and output I am getting.
      Note that this is after a successful ant build with all dependencies installed.

      bash-3.2$ ln -s indian-parallel-corpora-1.0 input
      bash-3.2$ $JOSHUA/bin/pipeline.pl --source bn --target en --no-prepare --aligner berkeley --type hiero --corpus input/bn-en/tok/training.bn-en --tune input/bn-en/tok/dev.bn-en --test input/bn-en/tok/devtest.bn-en
      [source-numlines] rebuilding...
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/indian-parallel-corpora-1.0/bn-en/tok/training.bn-en.bn [CHANGED]
        cmd=cat /private/tmp/joshua-v20151002-2032-qnt9ko/indian-parallel-corpora-1.0/bn-en/tok/training.bn-en.bn | wc -l
        took 0 seconds (0s)
      [source-numlines] retrieved cached result =>    20788
      [berkeley-aligner-chunk-0] rebuilding...
        dep=alignments/0/word-align.conf [CHANGED]
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/data/train/splits/corpus.bn.0 [CHANGED]
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/data/train/splits/corpus.en.0 [CHANGED]
        dep=alignments/0/training.align [NOT FOUND]
        cmd=java -d64 -Xmx10g -jar /private/tmp/joshua-v20151002-2032-qnt9ko/lib/berkeleyaligner.jar ++alignments/0/word-align.conf
        took 270 seconds (4m30s)
      [aligner-combine] rebuilding...
        dep=alignments/0/training.align [CHANGED]
        dep=alignments/training.align [NOT FOUND]
        cmd=cat alignments/0/training.align > alignments/training.align
        took 0 seconds (0s)
      [thrax-input-file] rebuilding...
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/indian-parallel-corpora-1.0/bn-en/tok/training.bn-en.bn [CHANGED]
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/indian-parallel-corpora-1.0/bn-en/tok/training.bn-en.en [CHANGED]
        dep=alignments/training.align [CHANGED]
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/data/train/thrax-input-file [NOT FOUND]
        cmd=/private/tmp/joshua-v20151002-2032-qnt9ko/scripts/training/paste /private/tmp/joshua-v20151002-2032-qnt9ko/indian-parallel-corpora-1.0/bn-en/tok/training.bn-en.bn /private/tmp/joshua-v20151002-2032-qnt9ko/indian-parallel-corpora-1.0/bn-en/tok/training.bn-en.en alignments/training.align | perl -pe 's/\t/ ||| /g' | grep -v '()' | grep -v '||| \+$' > /private/tmp/joshua-v20151002-2032-qnt9ko/data/train/thrax-input-file
        took 0 seconds (0s)
      [thrax-prep] rebuilding...
        dep=/private/tmp/joshua-v20151002-2032-qnt9ko/data/train/thrax-input-file [CHANGED]
        dep=grammar.gz [NOT FOUND]
        cmd=hadoop/bin/hadoop fs -rm -r pipeline-bn-en-hiero-_private_tmp_joshua-v20151002-2032-qnt9ko; hadoop/bin/hadoop fs -mkdir pipeline-bn-en-hiero-_private_tmp_joshua-v20151002-2032-qnt9ko; hadoop/bin/hadoop fs -put /private/tmp/joshua-v20151002-2032-qnt9ko/data/train/thrax-input-file pipeline-bn-en-hiero-_private_tmp_joshua-v20151002-2032-qnt9ko/input-file
        JOB FAILED (return code 1)
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:WARN Failed to set property [port] to value "".
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:WARN Failed to set property [port] to value "".
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      15/10/02 14:39:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      rm: Call From localhost/127.0.0.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:WARN Failed to set property [port] to value "".
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:WARN Failed to set property [port] to value "".
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      15/10/02 14:39:31 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      mkdir: Call From localhost/127.0.0.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:WARN Failed to set property [port] to value "".
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      log4j:WARN Failed to set property [port] to value "".
      log4j:ERROR Could not connect to remote log4j server at [localhost]. We will try again later.
      15/10/02 14:39:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      put: Call From localhost/127.0.0.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            lewismc Lewis John McGibbney
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: