Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-798

Implement Retry logic for SolrWriter

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.8.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      If Solr stops working, SolrWriter will not retry to send data to Solr. SolrWriter can be enhanced to implement retry logic.

        Activity

        Show
        eyang Eric Yang added a comment - I just committed this: https://github.com/apache/chukwa/commit/21b24284255d15fdc359ada9fc68d3a59fb60f7f
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #550 (See https://builds.apache.org/job/Chukwa-master/550/)
        CHUKWA-798. Added retry logic for SolrWriter and update to Solr 5.5.0 (eyang: rev 21b24284255d15fdc359ada9fc68d3a59fb60f7f)

        • contrib/solr/logs/conf/lang/stopwords_el.txt
        • contrib/solr/logs/conf/velocity/results_list.vm
        • contrib/solr/logs/conf/_schema_analysis_stopwords_english.json
        • contrib/solr/logs/conf/lang/stopwords_eu.txt
        • contrib/solr/logs/conf/lang/hyphenations_ga.txt
        • contrib/solr/logs/conf/lang/stoptags_ja.txt
        • contrib/solr/logs/conf/lang/stopwords_no.txt
        • contrib/solr/logs/conf/velocity/hit.vm
        • contrib/solr/logs/conf/lang/stopwords_hi.txt
        • contrib/solr/logs/conf/lang/stopwords_hu.txt
        • contrib/solr/logs/conf/lang/stopwords_ro.txt
        • contrib/solr/logs/conf/stopwords.txt
        • contrib/solr/logs/conf/lang/stopwords_hy.txt
        • contrib/solr/logs/conf/velocity/browse.vm
        • contrib/solr/logs/conf/xslt/example.xsl
        • contrib/solr/logs/conf/lang/stopwords_it.txt
        • contrib/solr/logs/conf/mapping-ISOLatin1Accent.txt
        • contrib/solr/logs/conf/scripts.conf
        • contrib/solr/logs/conf/velocity/debug.vm
        • contrib/solr/logs/conf/lang/contractions_ca.txt
        • contrib/solr/logs/conf/lang/stopwords_de.txt
        • contrib/solr/logs/conf/lang/stopwords_lv.txt
        • contrib/solr/logs/conf/lang/stopwords_ar.txt
        • contrib/solr/logs/conf/velocity/query_spatial.vm
        • contrib/solr/logs/conf/lang/stopwords_cz.txt
        • contrib/solr/logs/conf/lang/stopwords_es.txt
        • contrib/solr/logs/conf/clustering/carrot2/kmeans-attributes.xml
        • contrib/solr/logs/conf/velocity/query.vm
        • contrib/solr/logs/conf/velocity/tabs.vm
        • contrib/solr/logs/conf/lang/stopwords_ga.txt
        • contrib/solr/logs/conf/velocity/richtext_doc.vm
        • contrib/solr/logs/conf/clustering/carrot2/stc-attributes.xml
        • contrib/solr/logs/conf/lang/stopwords_pt.txt
        • contrib/solr/logs/conf/lang/stopwords_id.txt
        • contrib/solr/logs/conf/velocity/query_form.vm
        • src/main/web/hicc/ajax-solr/chukwa/js/logsearch.js
        • contrib/solr/logs/conf/velocity/product_doc.vm
        • contrib/solr/logs/conf/admin-extra.menu-bottom.html
        • contrib/solr/logs/conf/lang/contractions_ga.txt
        • contrib/solr/logs/core.properties
        • contrib/solr/logs/conf/lang/contractions_it.txt
        • contrib/solr/logs/conf/lang/contractions_fr.txt
        • contrib/solr/logs/conf/xslt/luke.xsl
        • contrib/solr/logs/conf/velocity/cluster_results.vm
        • contrib/solr/logs/conf/lang/stemdict_nl.txt
        • contrib/solr/logs/conf/elevate.xml
        • contrib/solr/logs/conf/velocity/error.vm
        • contrib/solr/logs/conf/velocity/facet_fields.vm
        • contrib/solr/logs/conf/lang/stopwords_gl.txt
        • contrib/solr/logs/conf/xslt/example_atom.xsl
        • contrib/solr/logs/conf/lang/stopwords_fa.txt
        • contrib/solr/logs/conf/lang/stopwords_ja.txt
        • contrib/solr/logs/conf/velocity/did_you_mean.vm
        • contrib/solr/logs/conf/lang/stopwords_da.txt
        • contrib/solr/logs/conf/lang/stopwords_fi.txt
        • contrib/solr/logs/conf/velocity/facets.vm
        • contrib/solr/logs/conf/velocity/main.css
        • contrib/solr/logs/conf/velocity/join_doc.vm
        • contrib/solr/logs/conf/lang/stopwords_ckb.txt
        • contrib/solr/logs/conf/_schema_analysis_synonyms_english.json
        • contrib/solr/logs/conf/xslt/example_rss.xsl
        • contrib/solr/logs/conf/lang/userdict_ja.txt
        • src/main/java/org/apache/hadoop/chukwa/datacollection/writer/solr/SolrWriter.java
        • contrib/solr/logs/conf/velocity/facet_ranges.vm
        • contrib/solr/logs/conf/lang/stopwords_sv.txt
        • contrib/solr/logs/conf/velocity/layout.vm
        • contrib/solr/logs/conf/velocity/hit_plain.vm
        • contrib/solr/logs/conf/velocity/pagination_bottom.vm
        • contrib/solr/logs/conf/lang/stopwords_fr.txt
        • conf/chukwa-agent-conf.xml
        • contrib/solr/logs/conf/lang/stopwords_ru.txt
        • contrib/solr/logs/conf/xslt/updateXml.xsl
        • contrib/solr/logs/conf/admin-extra.html
        • contrib/solr/logs/conf/velocity/suggest.vm
        • src/test/java/org/apache/hadoop/chukwa/datacollection/writer/solr/TestSolrWriter.java
        • contrib/solr/logs/README.txt
        • conf/chukwa-common.xml
        • contrib/solr/logs/conf/velocity/jquery.autocomplete.css
        • contrib/solr/logs/conf/velocity/header.vm
        • contrib/solr/logs/conf/velocity/README.txt
        • contrib/solr/logs/conf/lang/stopwords_en.txt
        • contrib/solr/chukwa/README.txt
        • contrib/solr/logs/conf/velocity/facet_queries.vm
        • contrib/solr/logs/conf/lang/stopwords_nl.txt
        • contrib/solr/logs/conf/lang/stopwords_th.txt
        • contrib/solr/logs/conf/spellings.txt
        • contrib/solr/logs/conf/velocity/cluster.vm
        • contrib/solr/logs/conf/velocity/jquery.autocomplete.js
        • src/main/web/hicc/ajax-solr/chukwa/js/logviewer.js
        • contrib/solr/logs/conf/mapping-FoldToASCII.txt
        • contrib/solr/logs/conf/lang/stopwords_ca.txt
        • contrib/solr/logs/conf/velocity/footer.vm
        • contrib/solr/logs/conf/velocity/query_group.vm
        • contrib/solr/logs/conf/synonyms.txt
        • contrib/solr/logs/conf/lang/stopwords_tr.txt
        • contrib/solr/logs/conf/currency.xml
        • contrib/solr/logs/conf/velocity/VM_global_library.vm
        • contrib/solr/logs/conf/velocity/facet_pivot.vm
        • contrib/solr/logs/conf/velocity/head.vm
        • contrib/solr/logs/conf/velocity/pagination_top.vm
        • contrib/solr/logs/conf/protwords.txt
        • contrib/solr/logs/conf/velocity/mime_type_lists.vm
        • contrib/solr/logs/conf/update-script.js
        • contrib/solr/logs/conf/velocity/hit_grouped.vm
        • contrib/solr/logs/conf/schema.xml
        • contrib/solr/logs/conf/admin-extra.menu-top.html
        • contrib/solr/logs/conf/clustering/carrot2/lingo-attributes.xml
        • contrib/solr/logs/conf/lang/stopwords_bg.txt
        • contrib/solr/logs/conf/solrconfig.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #550 (See https://builds.apache.org/job/Chukwa-master/550/ ) CHUKWA-798 . Added retry logic for SolrWriter and update to Solr 5.5.0 (eyang: rev 21b24284255d15fdc359ada9fc68d3a59fb60f7f) contrib/solr/logs/conf/lang/stopwords_el.txt contrib/solr/logs/conf/velocity/results_list.vm contrib/solr/logs/conf/_schema_analysis_stopwords_english.json contrib/solr/logs/conf/lang/stopwords_eu.txt contrib/solr/logs/conf/lang/hyphenations_ga.txt contrib/solr/logs/conf/lang/stoptags_ja.txt contrib/solr/logs/conf/lang/stopwords_no.txt contrib/solr/logs/conf/velocity/hit.vm contrib/solr/logs/conf/lang/stopwords_hi.txt contrib/solr/logs/conf/lang/stopwords_hu.txt contrib/solr/logs/conf/lang/stopwords_ro.txt contrib/solr/logs/conf/stopwords.txt contrib/solr/logs/conf/lang/stopwords_hy.txt contrib/solr/logs/conf/velocity/browse.vm contrib/solr/logs/conf/xslt/example.xsl contrib/solr/logs/conf/lang/stopwords_it.txt contrib/solr/logs/conf/mapping-ISOLatin1Accent.txt contrib/solr/logs/conf/scripts.conf contrib/solr/logs/conf/velocity/debug.vm contrib/solr/logs/conf/lang/contractions_ca.txt contrib/solr/logs/conf/lang/stopwords_de.txt contrib/solr/logs/conf/lang/stopwords_lv.txt contrib/solr/logs/conf/lang/stopwords_ar.txt contrib/solr/logs/conf/velocity/query_spatial.vm contrib/solr/logs/conf/lang/stopwords_cz.txt contrib/solr/logs/conf/lang/stopwords_es.txt contrib/solr/logs/conf/clustering/carrot2/kmeans-attributes.xml contrib/solr/logs/conf/velocity/query.vm contrib/solr/logs/conf/velocity/tabs.vm contrib/solr/logs/conf/lang/stopwords_ga.txt contrib/solr/logs/conf/velocity/richtext_doc.vm contrib/solr/logs/conf/clustering/carrot2/stc-attributes.xml contrib/solr/logs/conf/lang/stopwords_pt.txt contrib/solr/logs/conf/lang/stopwords_id.txt contrib/solr/logs/conf/velocity/query_form.vm src/main/web/hicc/ajax-solr/chukwa/js/logsearch.js contrib/solr/logs/conf/velocity/product_doc.vm contrib/solr/logs/conf/admin-extra.menu-bottom.html contrib/solr/logs/conf/lang/contractions_ga.txt contrib/solr/logs/core.properties contrib/solr/logs/conf/lang/contractions_it.txt contrib/solr/logs/conf/lang/contractions_fr.txt contrib/solr/logs/conf/xslt/luke.xsl contrib/solr/logs/conf/velocity/cluster_results.vm contrib/solr/logs/conf/lang/stemdict_nl.txt contrib/solr/logs/conf/elevate.xml contrib/solr/logs/conf/velocity/error.vm contrib/solr/logs/conf/velocity/facet_fields.vm contrib/solr/logs/conf/lang/stopwords_gl.txt contrib/solr/logs/conf/xslt/example_atom.xsl contrib/solr/logs/conf/lang/stopwords_fa.txt contrib/solr/logs/conf/lang/stopwords_ja.txt contrib/solr/logs/conf/velocity/did_you_mean.vm contrib/solr/logs/conf/lang/stopwords_da.txt contrib/solr/logs/conf/lang/stopwords_fi.txt contrib/solr/logs/conf/velocity/facets.vm contrib/solr/logs/conf/velocity/main.css contrib/solr/logs/conf/velocity/join_doc.vm contrib/solr/logs/conf/lang/stopwords_ckb.txt contrib/solr/logs/conf/_schema_analysis_synonyms_english.json contrib/solr/logs/conf/xslt/example_rss.xsl contrib/solr/logs/conf/lang/userdict_ja.txt src/main/java/org/apache/hadoop/chukwa/datacollection/writer/solr/SolrWriter.java contrib/solr/logs/conf/velocity/facet_ranges.vm contrib/solr/logs/conf/lang/stopwords_sv.txt contrib/solr/logs/conf/velocity/layout.vm contrib/solr/logs/conf/velocity/hit_plain.vm contrib/solr/logs/conf/velocity/pagination_bottom.vm contrib/solr/logs/conf/lang/stopwords_fr.txt conf/chukwa-agent-conf.xml contrib/solr/logs/conf/lang/stopwords_ru.txt contrib/solr/logs/conf/xslt/updateXml.xsl contrib/solr/logs/conf/admin-extra.html contrib/solr/logs/conf/velocity/suggest.vm src/test/java/org/apache/hadoop/chukwa/datacollection/writer/solr/TestSolrWriter.java contrib/solr/logs/README.txt conf/chukwa-common.xml contrib/solr/logs/conf/velocity/jquery.autocomplete.css contrib/solr/logs/conf/velocity/header.vm contrib/solr/logs/conf/velocity/README.txt contrib/solr/logs/conf/lang/stopwords_en.txt contrib/solr/chukwa/README.txt contrib/solr/logs/conf/velocity/facet_queries.vm contrib/solr/logs/conf/lang/stopwords_nl.txt contrib/solr/logs/conf/lang/stopwords_th.txt contrib/solr/logs/conf/spellings.txt contrib/solr/logs/conf/velocity/cluster.vm contrib/solr/logs/conf/velocity/jquery.autocomplete.js src/main/web/hicc/ajax-solr/chukwa/js/logviewer.js contrib/solr/logs/conf/mapping-FoldToASCII.txt contrib/solr/logs/conf/lang/stopwords_ca.txt contrib/solr/logs/conf/velocity/footer.vm contrib/solr/logs/conf/velocity/query_group.vm contrib/solr/logs/conf/synonyms.txt contrib/solr/logs/conf/lang/stopwords_tr.txt contrib/solr/logs/conf/currency.xml contrib/solr/logs/conf/velocity/VM_global_library.vm contrib/solr/logs/conf/velocity/facet_pivot.vm contrib/solr/logs/conf/velocity/head.vm contrib/solr/logs/conf/velocity/pagination_top.vm contrib/solr/logs/conf/protwords.txt contrib/solr/logs/conf/velocity/mime_type_lists.vm contrib/solr/logs/conf/update-script.js contrib/solr/logs/conf/velocity/hit_grouped.vm contrib/solr/logs/conf/schema.xml contrib/solr/logs/conf/admin-extra.menu-top.html contrib/solr/logs/conf/clustering/carrot2/lingo-attributes.xml contrib/solr/logs/conf/lang/stopwords_bg.txt contrib/solr/logs/conf/solrconfig.xml
        Hide
        eyang Eric Yang added a comment -

        Unit test isn't working due to Solr version change.

        Show
        eyang Eric Yang added a comment - Unit test isn't working due to Solr version change.
        Hide
        eyang Eric Yang added a comment -
        Show
        eyang Eric Yang added a comment - Fixed unit test to include solr.xml. https://github.com/apache/chukwa/commit/7eb11028f2ab146b2598085cf90f5d96809714c8
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #552 (See https://builds.apache.org/job/Chukwa-master/552/)
        CHUKWA-798. Added retry logic for SolrWriter and update to Solr 5.5.0 (eyang: rev 7eb11028f2ab146b2598085cf90f5d96809714c8)

        • pom.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #552 (See https://builds.apache.org/job/Chukwa-master/552/ ) CHUKWA-798 . Added retry logic for SolrWriter and update to Solr 5.5.0 (eyang: rev 7eb11028f2ab146b2598085cf90f5d96809714c8) pom.xml
        Show
        eyang Eric Yang added a comment - Missing file during commit. https://github.com/apache/chukwa/commit/7bd52b278fa4a4e35ee1a83d24ca9065c5dd67bd
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-master #553 (See https://builds.apache.org/job/Chukwa-master/553/)
        CHUKWA-798. Added retry logic for SolrWriter and update to Solr 5.5.0 (eyang: rev 7bd52b278fa4a4e35ee1a83d24ca9065c5dd67bd)

        • src/test/resources/solr.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-master #553 (See https://builds.apache.org/job/Chukwa-master/553/ ) CHUKWA-798 . Added retry logic for SolrWriter and update to Solr 5.5.0 (eyang: rev 7bd52b278fa4a4e35ee1a83d24ca9065c5dd67bd) src/test/resources/solr.xml

          People

          • Assignee:
            eyang Eric Yang
            Reporter:
            eyang Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development