Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7302

Solr MapReduce contrib needs better documentation

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 4.10.4, 5.0
    • Fix Version/s: None
    • Component/s: contrib - MapReduce
    • Labels:
      None

      Description

      The current REAME.txt contains the following instructions on this module. But I doubt it covers all commands as well as prerequisites/setups (e.g. morphlines required or not). In addition, we would need a little diagram to show all the dependencies such as morphlines (might need references to morphlines docs) and how this is related to the Solr-HDFS deployment in this wiki page (https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS).

      In summary, better documentation would make this contrib module much easier to users new to Solr-HDFS-MR deployment.

      ----------------------------------------------------------------------
      Apache Solr MapReduce

      Experimental - This contrib is currently subject to change in ways that may
      break back compatibility.

      The Solr MapReduce contrib provides an a mapreduce job that allows you to build
      Solr indexes and optionally merge them into a live Solr cluster.

      Example:

      1. Build an index with map-reduce and deploy it to SolrCloud

      source $solr_distrib/example/scripts/map-reduce/set-map-reduce-classpath.sh

      $hadoop_distrib/bin/hadoop --config $hadoop_conf_dir jar \
      $solr_distrib/dist/solr-map-reduce-*.jar -D 'mapred.child.java.opts=-Xmx500m' \
      -libjars "$HADOOP_LIBJAR" --morphline-file readAvroContainer.conf \
      --zk-host 127.0.0.1:9983 --output-dir hdfs://127.0.0.1:8020/outdir \
      --collection $collection --log4j log4j.properties --go-live \
      --verbose "hdfs://127.0.0.1:8020/indir"

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                taka.epsilon Taka
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: