Uploaded image for project: 'Joshua (Retired)'
  1. Joshua (Retired)
  2. JOSHUA-311

Improve pipeline logging to indicate location on berkeley alignment log(s)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 6.0.5
    • 6.2
    • alignment, logging, pipeline
    • None

    Description

      When one runs a pipeline using --aligner berkeley, no log location is provided for user to follow progress of alignment.

      [berkeley-aligner-chunk-0] rebuilding...
        dep=alignments/0/word-align.conf [CHANGED]
        dep=/usr/local/jpl/xdata/joshua_experiments/russian_experiments/0/data/train/splits/corpus.en.0 [NOT FOUND]
        dep=/usr/local/jpl/xdata/joshua_experiments/russian_experiments/0/data/train/splits/corpus.ru.0 [NOT FOUND]
        dep=alignments/0/training.align [NOT FOUND]
        cmd=java -d64 -Xmx10g -jar /usr/local/jpl/xdata/joshua_experiments/incubator-joshua/ext/berkeleyaligner/distribution/berkeleyaligner.jar ++alignments/0/word-align.conf
      

      We could add something like

      [berkeley-aligner-chunk-0] rebuilding...
        dep=alignments/0/word-align.conf [CHANGED]
        dep=/usr/local/jpl/xdata/joshua_experiments/russian_experiments/0/data/train/splits/corpus.en.0 [NOT FOUND]
        dep=/usr/local/jpl/xdata/joshua_experiments/russian_experiments/0/data/train/splits/corpus.ru.0 [NOT FOUND]
        dep=alignments/0/training.align [NOT FOUND]
        cmd=java -d64 -Xmx10g -jar /usr/local/jpl/xdata/joshua_experiments/incubator-joshua/ext/berkeleyaligner/distribution/berkeleyaligner.jar ++alignments/0/word-align.conf logs being written to /path/to/log
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            lewismc Lewis John McGibbney
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: