Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3, Trunk
    • Component/s: None
    • Labels:
      None

      Description

      Currently we default to java util logging and it's terrible in my opinion.

      *It's simple built in logger is a 2 line logger.
      *You have to jump through hoops to use your own custom formatter with jetty - either putting your class in the start.jar or other pain in the butt solutions.
      *It can't roll files by date out of the box.

      I'm sure there are more issues, but those are the ones annoying me now. We should switch to log4j - it's much nicer and it's easy to get a nice single line format and roll by date, etc.

      If someone wants to use JUL they still can - but at least users could start with something decent.

      1. SOLR-3706.patch
        24 kB
        Mark Miller
      2. SOLR-3706-maven.patch
        5 kB
        Steve Rowe
      3. SOLR-3706-maven.patch
        4 kB
        Ryan McKinley
      4. SOLR-3706-solr-log4j.patch
        18 kB
        Ryan McKinley
      5. SOLR-3706-solr-log4j.patch
        15 kB
        Ryan McKinley

        Issue Links

          Activity

          Hide
          ASF subversion and git services added a comment -

          Commit 1521742 from Ryan McKinley in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1521742 ]

          SOLR-3706: getRenderedMessage() rather than + "" (merge from trunk)

          Show
          ASF subversion and git services added a comment - Commit 1521742 from Ryan McKinley in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1521742 ] SOLR-3706 : getRenderedMessage() rather than + "" (merge from trunk)
          Hide
          ASF subversion and git services added a comment -

          Commit 1521741 from Ryan McKinley in branch 'dev/trunk'
          [ https://svn.apache.org/r1521741 ]

          SOLR-3706: getRenderedMessage() rather than + ""

          Show
          ASF subversion and git services added a comment - Commit 1521741 from Ryan McKinley in branch 'dev/trunk' [ https://svn.apache.org/r1521741 ] SOLR-3706 : getRenderedMessage() rather than + ""
          Hide
          ASF subversion and git services added a comment -

          Commit 1521739 from Ryan McKinley in branch 'dev/trunk'
          [ https://svn.apache.org/r1521739 ]

          SOLR-3706: avoid NPE (merge from 4x)

          Show
          ASF subversion and git services added a comment - Commit 1521739 from Ryan McKinley in branch 'dev/trunk' [ https://svn.apache.org/r1521739 ] SOLR-3706 : avoid NPE (merge from 4x)
          Hide
          ASF subversion and git services added a comment -

          Commit 1521738 from Ryan McKinley in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1521738 ]

          SOLR-3706: avoid NPE

          Show
          ASF subversion and git services added a comment - Commit 1521738 from Ryan McKinley in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1521738 ] SOLR-3706 : avoid NPE
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1462145

          SOLR-3706: remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1462145 SOLR-3706 : remove log dependencies from core, exclude example log jars from test classpath, add versions to example log jars
          Hide
          Uwe Schindler added a comment -

          Closed after release.

          Show
          Uwe Schindler added a comment - Closed after release.
          Hide
          Hoss Man added a comment -

          Linking to SOLR-4768 where a fix had to be made to the maven build to keep in in sync with these changes, and SOLR-4766 where the smokechecker was updated to catch problems like this in the future.

          Show
          Hoss Man added a comment - Linking to SOLR-4768 where a fix had to be made to the maven build to keep in in sync with these changes, and SOLR-4766 where the smokechecker was updated to catch problems like this in the future.
          Hide
          Hoss Man added a comment -

          As covered in SOLR-4771 the behaviour of the solr.war when used w/o any slf4j jars configured in the servlet container was abysmal as a result of this change, so SOLR-4771 added an explicit check for this in SolrDispatchFilter that refers people to "http://wiki.apache.org/solr/SolrLogging" for more details.

          However: the SolrLogging has never been updated to cover the changes made by this issue, and really should be prior to 4.3 being official.

          I'm linking the two issues, and posting these specific comments in the hopes that people watching this issue who had a vested interest in this change and understand what exactly folks need to do to use the SLF4J-less solr.war in other servlet containers will step up and updated the documentation based on their expertise.

          Show
          Hoss Man added a comment - As covered in SOLR-4771 the behaviour of the solr.war when used w/o any slf4j jars configured in the servlet container was abysmal as a result of this change, so SOLR-4771 added an explicit check for this in SolrDispatchFilter that refers people to "http://wiki.apache.org/solr/SolrLogging" for more details. However: the SolrLogging has never been updated to cover the changes made by this issue, and really should be prior to 4.3 being official. I'm linking the two issues, and posting these specific comments in the hopes that people watching this issue who had a vested interest in this change and understand what exactly folks need to do to use the SLF4J-less solr.war in other servlet containers will step up and updated the documentation based on their expertise.
          Hide
          Mark Miller added a comment - - edited

          Adding

          <pathelement path="${example}/resources"/>

          to the test classpath in the solrj build file takes care of finding the log4j config file for the example tests - I don't know why they won't pick it up from solrj/src/test-files.

          However, I'm still seeing the intermittent glitch (same as above)...

          Show
          Mark Miller added a comment - - edited Adding <pathelement path="${example}/resources"/> to the test classpath in the solrj build file takes care of finding the log4j config file for the example tests - I don't know why they won't pick it up from solrj/src/test-files. However, I'm still seeing the intermittent glitch (same as above)...
          Hide
          Mark Miller added a comment -

          Thanks Shawn.

          Show
          Mark Miller added a comment - Thanks Shawn.
          Hide
          Shawn Heisey added a comment -

          Mark Miller The two *-excl-slf4j build targets are now superfluous. Because people may be relying on them in build automation, it is probably not a good idea to remove them from 4x, but doing so in trunk strikes me as a good idea. I can open a new issue for that.

          Show
          Shawn Heisey added a comment - Mark Miller The two *-excl-slf4j build targets are now superfluous. Because people may be relying on them in build automation, it is probably not a good idea to remove them from 4x, but doing so in trunk strikes me as a good idea. I can open a new issue for that.
          Hide
          Mark Miller added a comment -

          Hmm...things are at least...better...passing all but that one random time so far...weird...

          Show
          Mark Miller added a comment - Hmm...things are at least...better...passing all but that one random time so far...weird...
          Hide
          Mark Miller added a comment -

          Another interesting data point - just running test-solrj has never caused a problem for me.

          Show
          Mark Miller added a comment - Another interesting data point - just running test-solrj has never caused a problem for me.
          Hide
          Mark Miller added a comment -

          Bah - and I just hit it again - guess I just had a couple lucky runs...

          Show
          Mark Miller added a comment - Bah - and I just hit it again - guess I just had a couple lucky runs...
          Hide
          Mark Miller added a comment -

          A couple test ant runs seemed at the old speed - last one was fairly slow though, looks like due to:

          [junit4:junit4] HEARTBEAT J2 PID(20759@totalmetal): 2013-03-28T11:09:27, stalled for 64.6s at: SolrExampleStreamingTest.testChineseDefaults
          [junit4:junit4] HEARTBEAT J2 PID(20759@totalmetal): 2013-03-28T11:10:27, stalled for  125s at: SolrExampleStreamingTest.testChineseDefaults
          [junit4:junit4] HEARTBEAT J2 PID(20759@totalmetal): 2013-03-28T11:11:27, stalled for  185s at: SolrExampleStreamingTest.testChineseDefaults
          [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest
          
          Show
          Mark Miller added a comment - A couple test ant runs seemed at the old speed - last one was fairly slow though, looks like due to: [junit4:junit4] HEARTBEAT J2 PID(20759@totalmetal): 2013-03-28T11:09:27, stalled for 64.6s at: SolrExampleStreamingTest.testChineseDefaults [junit4:junit4] HEARTBEAT J2 PID(20759@totalmetal): 2013-03-28T11:10:27, stalled for 125s at: SolrExampleStreamingTest.testChineseDefaults [junit4:junit4] HEARTBEAT J2 PID(20759@totalmetal): 2013-03-28T11:11:27, stalled for 185s at: SolrExampleStreamingTest.testChineseDefaults [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest
          Hide
          Mark Miller added a comment -

          Okay, committed to 5x and 4x - things should be in better shape at this point.

          Show
          Mark Miller added a comment - Okay, committed to 5x and 4x - things should be in better shape at this point.
          Hide
          Mark Miller added a comment -

          The jars in lib/ext do not have version numbers in the filenames.

          I'll address this.

          Show
          Mark Miller added a comment - The jars in lib/ext do not have version numbers in the filenames. I'll address this.
          Hide
          Mark Miller added a comment -

          More data point - ant test seems to go fine on my mac and is hit or miss on my other linux box running jenkins.

          Show
          Mark Miller added a comment - More data point - ant test seems to go fine on my mac and is hit or miss on my other linux box running jenkins.
          Hide
          Mark Miller added a comment -

          Okay, so I'll remove from core in the morning (must...sleep), but the problem appears to be having the jars in ivy in both the solrj and example modules. Need a creative way to get around it I guess...

          Show
          Mark Miller added a comment - Okay, so I'll remove from core in the morning (must...sleep), but the problem appears to be having the jars in ivy in both the solrj and example modules. Need a creative way to get around it I guess...
          Hide
          Mark Miller added a comment -

          Odd - Ryans first patch was giving me issues without the jars in core - but perhaps I just hadn't refreshed ivy right or something. Removing from core.

          Show
          Mark Miller added a comment - Odd - Ryans first patch was giving me issues without the jars in core - but perhaps I just hadn't refreshed ivy right or something. Removing from core.
          Hide
          Mark Miller added a comment -

          Between the one run i saw fail locally and your stuff above, it looks like just example based tests. And we have the jars both in the example and in solrj and in core (hence the warnings, though they never caused me any grief in my testing before). Not sure how to get around that (or if it is indeed the problem).

          Show
          Mark Miller added a comment - Between the one run i saw fail locally and your stuff above, it looks like just example based tests. And we have the jars both in the example and in solrj and in core (hence the warnings, though they never caused me any grief in my testing before). Not sure how to get around that (or if it is indeed the problem).
          Hide
          Mark Miller added a comment -

          It's late and I'm going to bed. All I can say is that I heavily tested the logging output for ant test, so I have no clue what has changed at the moment.

          Show
          Mark Miller added a comment - It's late and I'm going to bed. All I can say is that I heavily tested the logging output for ant test, so I have no clue what has changed at the moment.
          Hide
          Mark Miller added a comment -

          Hmm...seeing this on trunk as well - though I ran ant test a bunch of times previously to test the logging output...

          Show
          Mark Miller added a comment - Hmm...seeing this on trunk as well - though I ran ant test a bunch of times previously to test the logging output...
          Hide
          Hoss Man added a comment -

          Something is still broken as far as how log4j is being configured when tests are run...

          Possibly related, but as of this change, i can't get "ant test" to pass for the solrj tests.

          Using trunk, i can run the following...

          svn update -r 1461586 && ant clean && cd solr/solrj/ && ant test -Dtests.dups=10
          

          ...and everything runs great.

          But as soon as i try to use r1461587 (or later) at least one of the solrj tests using jetty will fail (even w/o trying to run duplicate tests) with a NoHttpResponseException...

          svn update -r 1461587 && ant clean && cd solr/solrj/ && ant test
          
          ...
          
          [junit4:junit4] HEARTBEAT J2 PID(18228@frisbee): 2013-03-27T17:40:10, stalled for  125s at: SolrExampleJettyTest.testUpdateMultiValuedField
          [junit4:junit4] HEARTBEAT J2 PID(18228@frisbee): 2013-03-27T17:41:10, stalled for  185s at: SolrExampleJettyTest.testUpdateMultiValuedField
          [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleJettyTest
          [junit4:junit4]   2> SLF4J: Class path contains multiple SLF4J bindings.
          [junit4:junit4]   2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/solrj/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
          [junit4:junit4]   2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/core/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
          [junit4:junit4]   2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/example/lib/ext/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
          [junit4:junit4]   2> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
          [junit4:junit4]   2> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
          [junit4:junit4]   2> log4j:WARN No appenders could be found for logger (org.apache.solr.SolrJettyTestBase).
          [junit4:junit4]   2> log4j:WARN Please initialize the log4j system properly.
          [junit4:junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
          [junit4:junit4]   2> Creating dataDir: /home/hossman/lucene/dev/solr/build/solr-solrj/test/J2/./solrtest-SolrExampleJettyTest-1364431081119
          [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrExampleJettyTest -Dtests.method=testUpdateMultiValuedField -Dtests.seed=D7EFA4209757AF4C -Dtests.slow=true -Dtests.locale=uk -Dtests.timezone=America/Argentina/Rio_Gallegos -Dtests.file.encoding=US-ASCII
          [junit4:junit4] ERROR    201s J2 | SolrExampleJettyTest.testUpdateMultiValuedField <<<
          [junit4:junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:55761/solr
          [junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([D7EFA4209757AF4C:C5EA294B445B7894]:0)
          [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:416)
          [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
          [junit4:junit4]    > 	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
          [junit4:junit4]    > 	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
          [junit4:junit4]    > 	at org.apache.solr.client.solrj.SolrExampleTests.testUpdateMultiValuedField(SolrExampleTests.java:1404)
          [junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
          [junit4:junit4]    > Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
          [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
          [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
          [junit4:junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
          [junit4:junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
          [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
          [junit4:junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
          [junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
          [junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
          [junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
          [junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
          [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
          [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
          [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
          [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
          [junit4:junit4]    > 	... 44 more
          [junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=uk, timezone=America/Argentina/Rio_Gallegos
          [junit4:junit4]   2> NOTE: Linux 3.2.0-40-generic amd64/Oracle Corporation 1.7.0_15 (64-bit)/cpus=4,threads=1,free=171547592,total=337838080
          [junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrExampleJettyTest]
          [junit4:junit4] Completed on J2 in 213.23s, 27 tests, 1 error <<< FAILURES!
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.client.solrj.request.TestCoreAdmin
          [junit4:junit4] Completed on J2 in 0.65s, 2 tests
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.LargeVolumeBinaryJettyTest
          [junit4:junit4] Completed on J2 in 1.03s, 1 test
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.common.util.TestNamedListCodec
          [junit4:junit4] Completed on J2 in 0.31s, 4 tests
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.common.params.SolrParamTest
          [junit4:junit4] Completed on J2 in 0.02s, 1 test
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest
          [junit4:junit4] Completed on J2 in 18.64s, 27 tests
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.common.util.TestXMLEscaping
          [junit4:junit4] Completed on J2 in 0.07s, 7 tests
          [junit4:junit4] 
          [junit4:junit4] Suite: org.apache.solr.client.solrj.response.FacetFieldTest
          [junit4:junit4] Completed on J2 in 0.01s, 1 test
          [junit4:junit4] 
          [junit4:junit4] 
          [junit4:junit4] Tests with failures:
          [junit4:junit4]   - org.apache.solr.client.solrj.embedded.SolrExampleJettyTest.testUpdateMultiValuedField
          [junit4:junit4] 
          [junit4:junit4] 
          [junit4:junit4] JVM J0:     1.01 ..    79.72 =    78.71s
          [junit4:junit4] JVM J1:     1.23 ..    79.74 =    78.52s
          [junit4:junit4] JVM J2:     1.02 ..   235.99 =   234.97s
          [junit4:junit4] Execution time total: 3 minutes 56 seconds
          [junit4:junit4] Tests summary: 45 suites, 266 tests, 1 error
          

          ...the failures happen in different tests at different places, but all the ones i've seen have happened in the middle of tests – never the "first" attempt at talking to the solr server, suggesting that the server is in fact up, and responds to at least some of the add/delete/commit requests but then later it suddenly vanishes. (I have no idea why this might be happening, or how it might related to the changes in r1461587 ... there is no hint as to why the server has suddenly vanished, because of the previously mentioned problem of log4j not finding config).

          Show
          Hoss Man added a comment - Something is still broken as far as how log4j is being configured when tests are run... Possibly related, but as of this change, i can't get "ant test" to pass for the solrj tests. Using trunk, i can run the following... svn update -r 1461586 && ant clean && cd solr/solrj/ && ant test -Dtests.dups=10 ...and everything runs great. But as soon as i try to use r1461587 (or later) at least one of the solrj tests using jetty will fail (even w/o trying to run duplicate tests) with a NoHttpResponseException... svn update -r 1461587 && ant clean && cd solr/solrj/ && ant test ... [junit4:junit4] HEARTBEAT J2 PID(18228@frisbee): 2013-03-27T17:40:10, stalled for 125s at: SolrExampleJettyTest.testUpdateMultiValuedField [junit4:junit4] HEARTBEAT J2 PID(18228@frisbee): 2013-03-27T17:41:10, stalled for 185s at: SolrExampleJettyTest.testUpdateMultiValuedField [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleJettyTest [junit4:junit4] 2> SLF4J: Class path contains multiple SLF4J bindings. [junit4:junit4] 2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/solrj/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit4:junit4] 2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/core/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit4:junit4] 2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/example/lib/ext/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit4:junit4] 2> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit4:junit4] 2> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] [junit4:junit4] 2> log4j:WARN No appenders could be found for logger (org.apache.solr.SolrJettyTestBase). [junit4:junit4] 2> log4j:WARN Please initialize the log4j system properly. [junit4:junit4] 2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [junit4:junit4] 2> Creating dataDir: /home/hossman/lucene/dev/solr/build/solr-solrj/test/J2/./solrtest-SolrExampleJettyTest-1364431081119 [junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SolrExampleJettyTest -Dtests.method=testUpdateMultiValuedField -Dtests.seed=D7EFA4209757AF4C -Dtests.slow=true -Dtests.locale=uk -Dtests.timezone=America/Argentina/Rio_Gallegos -Dtests.file.encoding=US-ASCII [junit4:junit4] ERROR 201s J2 | SolrExampleJettyTest.testUpdateMultiValuedField <<< [junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:55761/solr [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D7EFA4209757AF4C:C5EA294B445B7894]:0) [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:416) [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181) [junit4:junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) [junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168) [junit4:junit4] > at org.apache.solr.client.solrj.SolrExampleTests.testUpdateMultiValuedField(SolrExampleTests.java:1404) [junit4:junit4] > at java.lang.Thread.run(Thread.java:722) [junit4:junit4] > Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond [junit4:junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95) [junit4:junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62) [junit4:junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254) [junit4:junit4] > at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289) [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252) [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191) [junit4:junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300) [junit4:junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127) [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717) [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522) [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353) [junit4:junit4] > ... 44 more [junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=uk, timezone=America/Argentina/Rio_Gallegos [junit4:junit4] 2> NOTE: Linux 3.2.0-40-generic amd64/Oracle Corporation 1.7.0_15 (64-bit)/cpus=4,threads=1,free=171547592,total=337838080 [junit4:junit4] 2> NOTE: All tests run in this JVM: [SolrExampleJettyTest] [junit4:junit4] Completed on J2 in 213.23s, 27 tests, 1 error <<< FAILURES! [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.client.solrj.request.TestCoreAdmin [junit4:junit4] Completed on J2 in 0.65s, 2 tests [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.LargeVolumeBinaryJettyTest [junit4:junit4] Completed on J2 in 1.03s, 1 test [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.common.util.TestNamedListCodec [junit4:junit4] Completed on J2 in 0.31s, 4 tests [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.common.params.SolrParamTest [junit4:junit4] Completed on J2 in 0.02s, 1 test [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest [junit4:junit4] Completed on J2 in 18.64s, 27 tests [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.common.util.TestXMLEscaping [junit4:junit4] Completed on J2 in 0.07s, 7 tests [junit4:junit4] [junit4:junit4] Suite: org.apache.solr.client.solrj.response.FacetFieldTest [junit4:junit4] Completed on J2 in 0.01s, 1 test [junit4:junit4] [junit4:junit4] [junit4:junit4] Tests with failures: [junit4:junit4] - org.apache.solr.client.solrj.embedded.SolrExampleJettyTest.testUpdateMultiValuedField [junit4:junit4] [junit4:junit4] [junit4:junit4] JVM J0: 1.01 .. 79.72 = 78.71s [junit4:junit4] JVM J1: 1.23 .. 79.74 = 78.52s [junit4:junit4] JVM J2: 1.02 .. 235.99 = 234.97s [junit4:junit4] Execution time total: 3 minutes 56 seconds [junit4:junit4] Tests summary: 45 suites, 266 tests, 1 error ...the failures happen in different tests at different places, but all the ones i've seen have happened in the middle of tests – never the "first" attempt at talking to the solr server, suggesting that the server is in fact up, and responds to at least some of the add/delete/commit requests but then later it suddenly vanishes. (I have no idea why this might be happening, or how it might related to the changes in r1461587 ... there is no hint as to why the server has suddenly vanished, because of the previously mentioned problem of log4j not finding config).
          Hide
          Shawn Heisey added a comment -

          A couple of notes after taking a look at things:

          1) Included in example/lib/ext is jul-to-slf4j.jar, but I don't have this jar in my custom log4j setup. This might be because I do not use any features that depend directly on JUL. Searching the entire checkout for java.util.logging, I can see some explicit imports, even outside the admin UI classes specifically made for it. Below is a list of all the .java files in trunk that import JUL classes. I suspect that the spatial and facet code should use slf4j instead. The test code might have a legitimate reason to import JUL directly, and the log formatter for the admin UI might still have need for it. Is there anything here that needs follow-up?

          ./lucene/spatial/src/test/org/apache/lucene/spatial/StrategyTestCase.java
          ./lucene/spatial/src/java/org/apache/lucene/spatial/util/ShapeFieldCacheProvider.java
          ./lucene/facet/src/java/org/apache/lucene/facet/search/StandardFacetsAccumulator.java
          ./lucene/facet/src/java/org/apache/lucene/facet/sampling/RepeatableSampler.java
          ./lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
          ./lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java
          ./solr/core/src/test/org/apache/solr/handler/admin/LoggingHandlerTest.java
          ./solr/core/src/java/org/apache/solr/SolrLogFormatter.java
          ./solr/core/src/java/org/apache/solr/logging/jul/JulInfo.java
          ./solr/core/src/java/org/apache/solr/logging/jul/RecordHandler.java
          ./solr/core/src/java/org/apache/solr/logging/jul/JulWatcher.java
          ./solr/test-framework/src/java/org/apache/solr/SolrTestCaseJ4.java
          

          2) The jars in lib/ext do not have version numbers in the filenames. This doesn't affect operation at all, but it does mean that a casual observer will not know what version is being included, so it'll be harder to track down potential bugs in those libraries. Even examining the filenames within the jars reveals no version information.

          Show
          Shawn Heisey added a comment - A couple of notes after taking a look at things: 1) Included in example/lib/ext is jul-to-slf4j.jar, but I don't have this jar in my custom log4j setup. This might be because I do not use any features that depend directly on JUL. Searching the entire checkout for java.util.logging, I can see some explicit imports, even outside the admin UI classes specifically made for it. Below is a list of all the .java files in trunk that import JUL classes. I suspect that the spatial and facet code should use slf4j instead. The test code might have a legitimate reason to import JUL directly, and the log formatter for the admin UI might still have need for it. Is there anything here that needs follow-up? ./lucene/spatial/src/test/org/apache/lucene/spatial/StrategyTestCase.java ./lucene/spatial/src/java/org/apache/lucene/spatial/util/ShapeFieldCacheProvider.java ./lucene/facet/src/java/org/apache/lucene/facet/search/StandardFacetsAccumulator.java ./lucene/facet/src/java/org/apache/lucene/facet/sampling/RepeatableSampler.java ./lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java ./lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java ./solr/core/src/test/org/apache/solr/handler/admin/LoggingHandlerTest.java ./solr/core/src/java/org/apache/solr/SolrLogFormatter.java ./solr/core/src/java/org/apache/solr/logging/jul/JulInfo.java ./solr/core/src/java/org/apache/solr/logging/jul/RecordHandler.java ./solr/core/src/java/org/apache/solr/logging/jul/JulWatcher.java ./solr/test-framework/src/java/org/apache/solr/SolrTestCaseJ4.java 2) The jars in lib/ext do not have version numbers in the filenames. This doesn't affect operation at all, but it does mean that a casual observer will not know what version is being included, so it'll be harder to track down potential bugs in those libraries. Even examining the filenames within the jars reveals no version information.
          Hide
          Hoss Man added a comment -

          Something is still broken as far as how log4j is being configured when tests are run...

          hossman@frisbee:~/lucene/dev/solr/solrj$ ant test -Dtestcase=SolrExampleXMLTest
          ...
          common.test:
          [junit4:junit4] <JUnit4> says hi! Master seed: 7C26743BA1BE32C5
          [junit4:junit4] Executing 1 suite with 1 JVM.
          [junit4:junit4] 
          [junit4:junit4] Started J0 PID(6428@frisbee).
          [junit4:junit4] Suite: org.apache.solr.client.solrj.SolrExampleXMLTest
          [junit4:junit4]   2> SLF4J: Class path contains multiple SLF4J bindings.
          [junit4:junit4]   2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/solrj/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
          [junit4:junit4]   2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/core/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
          [junit4:junit4]   2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/example/lib/ext/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
          [junit4:junit4]   2> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
          [junit4:junit4]   2> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
          [junit4:junit4]   2> log4j:WARN No appenders could be found for logger (org.apache.solr.SolrJettyTestBase).
          [junit4:junit4]   2> log4j:WARN Please initialize the log4j system properly.
          [junit4:junit4]   2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
          
          Show
          Hoss Man added a comment - Something is still broken as far as how log4j is being configured when tests are run... hossman@frisbee:~/lucene/dev/solr/solrj$ ant test -Dtestcase=SolrExampleXMLTest ... common.test: [junit4:junit4] <JUnit4> says hi! Master seed: 7C26743BA1BE32C5 [junit4:junit4] Executing 1 suite with 1 JVM. [junit4:junit4] [junit4:junit4] Started J0 PID(6428@frisbee). [junit4:junit4] Suite: org.apache.solr.client.solrj.SolrExampleXMLTest [junit4:junit4] 2> SLF4J: Class path contains multiple SLF4J bindings. [junit4:junit4] 2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/solrj/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit4:junit4] 2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/core/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit4:junit4] 2> SLF4J: Found binding in [jar:file:/home/hossman/lucene/dev/solr/example/lib/ext/slf4j-log4j12.jar!/org/slf4j/impl/StaticLoggerBinder.class] [junit4:junit4] 2> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [junit4:junit4] 2> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] [junit4:junit4] 2> log4j:WARN No appenders could be found for logger (org.apache.solr.SolrJettyTestBase). [junit4:junit4] 2> log4j:WARN Please initialize the log4j system properly. [junit4:junit4] 2> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
          Hide
          Steve Rowe added a comment -

          yup it worked.

          Thanks for testing, Ryan.

          Committed to trunk and branch_4x.

          Show
          Steve Rowe added a comment - yup it worked. Thanks for testing, Ryan. Committed to trunk and branch_4x.
          Hide
          Ryan McKinley added a comment -

          yup it worked. thanks Steve Rowe

          Show
          Ryan McKinley added a comment - yup it worked. thanks Steve Rowe
          Hide
          Steve Rowe added a comment -

          Patch, minor mods from Ryan McKinley's patch (mostly adding org.slf4j:jul-to-slf4j as an optional dep in the solr parent POM).

          The .war now has the same set of .jar files under WEB-INF/lib/ (after SOLR-4649 removes two test-only jars from the Ant-built .war) - I had to make the morfologik dependency optional in the solr-core POM in order to achieve this.

          Also, compilation succeeds, and all Solr tests pass under the Maven build.

          Ryan, could you try this patch and see if it fixes your compilation problem?

          Show
          Steve Rowe added a comment - Patch, minor mods from Ryan McKinley 's patch (mostly adding org.slf4j:jul-to-slf4j as an optional dep in the solr parent POM). The .war now has the same set of .jar files under WEB-INF/lib/ (after SOLR-4649 removes two test-only jars from the Ant-built .war) - I had to make the morfologik dependency optional in the solr-core POM in order to achieve this. Also, compilation succeeds, and all Solr tests pass under the Maven build. Ryan, could you try this patch and see if it fixes your compilation problem?
          Hide
          Mark Miller added a comment -

          Whoops - I have to deal with some license issues.

          Show
          Mark Miller added a comment - Whoops - I have to deal with some license issues.
          Hide
          Ryan McKinley added a comment -

          I still follow "the committer does not need deal with maven, maven dudes will deal with it"

          that is totally fine. It is easier to do in two passes anyway

          Show
          Ryan McKinley added a comment - I still follow "the committer does not need deal with maven, maven dudes will deal with it" that is totally fine. It is easier to do in two passes anyway
          Hide
          Steve Rowe added a comment -

          That is, Steve Rowe

          Cool, I got a "you got mentioned in JIRA" email, which prodded me to do more than TL;DR this issue's comments - I didn't know about this JIRA feature.

          We still need to update the maven dependencies. This is my first attempt, but it still fails with: [...] I won't be able to look at this for a while, so someone else can pick it up or I will try to tackle it later

          I'll take a look today.

          Show
          Steve Rowe added a comment - That is, Steve Rowe Cool, I got a "you got mentioned in JIRA" email, which prodded me to do more than TL;DR this issue's comments - I didn't know about this JIRA feature. We still need to update the maven dependencies. This is my first attempt, but it still fails with: [...] I won't be able to look at this for a while, so someone else can pick it up or I will try to tackle it later I'll take a look today.
          Hide
          Mark Miller added a comment -

          That is, Steve Rowe

          Show
          Mark Miller added a comment - That is, Steve Rowe
          Hide
          Mark Miller added a comment -

          Someone will jump on it - I still follow "the committer does not need deal with maven, maven dudes will deal with it" that fell out of the big maven discussion Unfortunately, your one of those maven dudes, so you can't duck as nicely, but I'd bet Steve or someone will be willing to help us out.

          Show
          Mark Miller added a comment - Someone will jump on it - I still follow "the committer does not need deal with maven, maven dudes will deal with it" that fell out of the big maven discussion Unfortunately, your one of those maven dudes, so you can't duck as nicely, but I'd bet Steve or someone will be willing to help us out.
          Hide
          Ryan McKinley added a comment -

          We still need to update the maven dependencies. This is my first attempt, but it still fails with:

          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project solr-analysis-extras: Compilation failure: Compilation failure:
          [ERROR] /Users/ryan/workspace/apache/lucene_4x/solr/contrib/analysis-extras/src/java/org/apache/solr/schema/ICUCollationField.java:[37,30] package org.apache.solr.common does not exist
          [ERROR] /Users/ryan/workspace/apache/lucene_4x/solr/contrib/analysis-extras/src/java/org/apache/solr/schema/ICUCollationField.java:[38,44] package org.apache.solr.
          

          I won't be able to look at this for a while, so someone else can pick it up or I will try to tackle it later

          Show
          Ryan McKinley added a comment - We still need to update the maven dependencies. This is my first attempt, but it still fails with: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile ( default -compile) on project solr-analysis-extras: Compilation failure: Compilation failure: [ERROR] /Users/ryan/workspace/apache/lucene_4x/solr/contrib/analysis-extras/src/java/org/apache/solr/schema/ICUCollationField.java:[37,30] package org.apache.solr.common does not exist [ERROR] /Users/ryan/workspace/apache/lucene_4x/solr/contrib/analysis-extras/src/java/org/apache/solr/schema/ICUCollationField.java:[38,44] package org.apache.solr. I won't be able to look at this for a while, so someone else can pick it up or I will try to tackle it later
          Hide
          Jan Høydahl added a comment -

          You can't mix and match. So the CHANGES.txt should mention what do do for custom deploys. I guess also solr/SYSTEM_REQUIREMENTS.txt could be updated with a paragraph saying that the Solr WAR depends on SLF4j being installed.

          Show
          Jan Høydahl added a comment - You can't mix and match. So the CHANGES.txt should mention what do do for custom deploys. I guess also solr/SYSTEM_REQUIREMENTS.txt could be updated with a paragraph saying that the Solr WAR depends on SLF4j being installed.
          Hide
          Mark Miller added a comment -

          My question was only about the slf4j-api.jar file not any of the implementation jars

          I believe others have reported you cannot mix and match this way. I think that makes sense given the different class loaders that would be involved and how jetty will use slf4j as well.

          If we don't include the -api.jar and there is no logging configured in the container it will crash with a ClassNotFound exception. If we add the -api.jar file it will write a line to stdout and stderr that say "no logging framework found" and then hide all logging messages. (this is the default slf4j behavior)

          I think if we can do something more explicit, that would be very helpful, but I think we are working from the classnotfound base point, because I don't think we can mix.

          Show
          Mark Miller added a comment - My question was only about the slf4j-api.jar file not any of the implementation jars I believe others have reported you cannot mix and match this way. I think that makes sense given the different class loaders that would be involved and how jetty will use slf4j as well. If we don't include the -api.jar and there is no logging configured in the container it will crash with a ClassNotFound exception. If we add the -api.jar file it will write a line to stdout and stderr that say "no logging framework found" and then hide all logging messages. (this is the default slf4j behavior) I think if we can do something more explicit, that would be very helpful, but I think we are working from the classnotfound base point, because I don't think we can mix.
          Hide
          Ryan McKinley added a comment -

          My question was only about the slf4j-api.jar file not any of the implementation jars

          If we don't include the -api.jar and there is no logging configured in the container it will crash with a ClassNotFound exception. If we add the -api.jar file it will write a line to stdout and stderr that say "no logging framework found" and then hide all logging messages. (this is the default slf4j behavior)

          Show
          Ryan McKinley added a comment - My question was only about the slf4j-api.jar file not any of the implementation jars If we don't include the -api.jar and there is no logging configured in the container it will crash with a ClassNotFound exception. If we add the -api.jar file it will write a line to stdout and stderr that say "no logging framework found" and then hide all logging messages. (this is the default slf4j behavior)
          Hide
          Mark Miller added a comment -

          It's a much better model really - it also lets jetty hook into the logging right away (or tomcat or whatever you container). Tons of upsides. Logging freedom. Finally. Yay.

          It might be a good idea to have a very early classloader check in SolrDispatchFilter that checks if SLF4J is there and if not then gives you a very helpful message to remedy it – i.e. you need to place SLF4J jars in your app-server somewhere or re-pack the WAR with them.

          You bring up a good point - we should look at that and see how friendly we can make it.

          Show
          Mark Miller added a comment - It's a much better model really - it also lets jetty hook into the logging right away (or tomcat or whatever you container). Tons of upsides. Logging freedom. Finally. Yay. It might be a good idea to have a very early classloader check in SolrDispatchFilter that checks if SLF4J is there and if not then gives you a very helpful message to remedy it – i.e. you need to place SLF4J jars in your app-server somewhere or re-pack the WAR with them. You bring up a good point - we should look at that and see how friendly we can make it.
          Hide
          David Smiley added a comment -

          As long it's also easy to handle an SLF4J-less war in Tomcat, then I think it's fine too. Nonetheless this is going to be a new thing a deployer of Solr needs to check the box in their checklist. It might be a good idea to have a very early classloader check in SolrDispatchFilter that checks if SLF4J is there and if not then gives you a very helpful message to remedy it – i.e. you need to place SLF4J jars in your app-server somewhere or re-pack the WAR with them.

          Show
          David Smiley added a comment - As long it's also easy to handle an SLF4J-less war in Tomcat, then I think it's fine too. Nonetheless this is going to be a new thing a deployer of Solr needs to check the box in their checklist. It might be a good idea to have a very early classloader check in SolrDispatchFilter that checks if SLF4J is there and if not then gives you a very helpful message to remedy it – i.e. you need to place SLF4J jars in your app-server somewhere or re-pack the WAR with them.
          Hide
          Shawn Heisey added a comment -

          Do we want slf4j-api in the .war or nothing? I think nothing is OK (as you have it)

          I favor having none of the slf4j jars in the war. If logging jars in lib/ext will be the new standard, it's better to have all the related jars there and none of them in the war.

          Show
          Shawn Heisey added a comment - Do we want slf4j-api in the .war or nothing? I think nothing is OK (as you have it) I favor having none of the slf4j jars in the war. If logging jars in lib/ext will be the new standard, it's better to have all the related jars there and none of them in the war.
          Hide
          Mark Miller added a comment -

          Do we want slf4j-api in the .war or nothing? I think nothing is OK (as you have it)

          I don't think so - if we do that, everyone has claimed you cannot easily change the logging impl without building a custom war.

          Unrelated... the .war includes restlet stuff? When I poke around the code that is only used in tests. Is the restlet infrastructure part of solr core now?

          Yes, the schema read side rest api uses it - new in 4.2.

          Show
          Mark Miller added a comment - Do we want slf4j-api in the .war or nothing? I think nothing is OK (as you have it) I don't think so - if we do that, everyone has claimed you cannot easily change the logging impl without building a custom war. Unrelated... the .war includes restlet stuff? When I poke around the code that is only used in tests. Is the restlet infrastructure part of solr core now? Yes, the schema read side rest api uses it - new in 4.2.
          Hide
          Ryan McKinley added a comment -

          Do we want slf4j-api in the .war or nothing? I think nothing is OK (as you have it)

          Unrelated... the .war includes restlet stuff? When I poke around the code that is only used in tests. Is the restlet infrastructure part of solr core now?

          I'm testing maven stuff and will let you know if that looks OK

          Show
          Ryan McKinley added a comment - Do we want slf4j-api in the .war or nothing? I think nothing is OK (as you have it) Unrelated... the .war includes restlet stuff? When I poke around the code that is only used in tests. Is the restlet infrastructure part of solr core now? I'm testing maven stuff and will let you know if that looks OK
          Hide
          Mark Miller added a comment -

          Hmm..looks like ant test fails in the solrj module are not picking up the logging config file in core. For the moment I think I can just put it in the solrj test-files dir as well. It may be better longer term to get a test-file dir for the test-framework module and have one copy there if that is possible.

          Show
          Mark Miller added a comment - Hmm..looks like ant test fails in the solrj module are not picking up the logging config file in core. For the moment I think I can just put it in the solrj test-files dir as well. It may be better longer term to get a test-file dir for the test-framework module and have one copy there if that is possible.
          Hide
          Mark Miller added a comment -

          Just some general sanity review would be good. I tried running tests in eclipse as well as making sure test fails with ant test still logged. Someone might have comments on the logging format, I got it from the wiki or something. I also tested run-example and it logs. I updated the README to mention you might want to turn off the console logging and that we use log4j by default.

          If someone can review, I'll commit it.

          Show
          Mark Miller added a comment - Just some general sanity review would be good. I tried running tests in eclipse as well as making sure test fails with ant test still logged. Someone might have comments on the logging format, I got it from the wiki or something. I also tested run-example and it logs. I updated the README to mention you might want to turn off the console logging and that we use log4j by default. If someone can review, I'll commit it.
          Hide
          Ryan McKinley added a comment -

          Mark – anything you want me to look at?

          What are the things that need done before we can commit? (I think SolrLogFormatter should come as a follow on issue)

          Show
          Ryan McKinley added a comment - Mark – anything you want me to look at? What are the things that need done before we can commit? (I think SolrLogFormatter should come as a follow on issue)
          Hide
          Mark Miller added a comment -

          This builds on Ryan's patch.

          Show
          Mark Miller added a comment - This builds on Ryan's patch.
          Hide
          Ryan McKinley added a comment -

          I'm going to say lets get log4j fully integrated and working well

          +1 log4j is common and the infrastructure changes we need to make that work are the same needed for logback or log4j2.

          Once we have things working, converting the SolrLogFormatter to Log4j would be great.

          Show
          Ryan McKinley added a comment - I'm going to say lets get log4j fully integrated and working well +1 log4j is common and the infrastructure changes we need to make that work are the same needed for logback or log4j2. Once we have things working, converting the SolrLogFormatter to Log4j would be great.
          Hide
          Mark Miller added a comment -

          One thing that we would lose in tests with the current approach is Yonik Seeley's SolrLogFormatter. We would probably want to re-implement it. I'd much prefer that to trying to straddle different logging impls for example and tests. I think devs should see the logging users will see. And did I mention the single line logging?

          Show
          Mark Miller added a comment - One thing that we would lose in tests with the current approach is Yonik Seeley 's SolrLogFormatter. We would probably want to re-implement it. I'd much prefer that to trying to straddle different logging impls for example and tests. I think devs should see the logging users will see. And did I mention the single line logging?
          Hide
          Mark Miller added a comment -

          I've got this mostly working, though there are still a few little chads to work out. One line logging...it's beautiful

          I'm going to say lets get log4j fully integrated and working well - I think most of us know it best and it's already setup to work with the admin UI.

          If someone that knows logback can then use this as a starting base to switch to logback (should be much easier to build off this), that would be fine, else we can consider it later, or simply move to log4j2 later. Someone has said it offers the same benefits as logback.

          Show
          Mark Miller added a comment - I've got this mostly working, though there are still a few little chads to work out. One line logging...it's beautiful I'm going to say lets get log4j fully integrated and working well - I think most of us know it best and it's already setup to work with the admin UI. If someone that knows logback can then use this as a starting base to switch to logback (should be much easier to build off this), that would be fine, else we can consider it later, or simply move to log4j2 later. Someone has said it offers the same benefits as logback.
          Hide
          Satheesh Akkinepally added a comment -

          We got logback working on our jetty/solr installation. logBack also provides configurable http access logs (log4j does not have this for jetty) out of the box and it works great with jetty and solr. But yes for this to work, we had to unwar the solr.war remove the native jdk logging jar and place the logBack implementation in the war.
          It would be good if we dont have to unwar and war and can use the solr.war out of the box for any custom logging implementation that we want to use. Also, the web app class loader seems to ignore the log4j implementation found in the jetty lib if slf4j is available within the war.

          Show
          Satheesh Akkinepally added a comment - We got logback working on our jetty/solr installation. logBack also provides configurable http access logs (log4j does not have this for jetty) out of the box and it works great with jetty and solr. But yes for this to work, we had to unwar the solr.war remove the native jdk logging jar and place the logBack implementation in the war. It would be good if we dont have to unwar and war and can use the solr.war out of the box for any custom logging implementation that we want to use. Also, the web app class loader seems to ignore the log4j implementation found in the jetty lib if slf4j is available within the war.
          Hide
          Mark Miller added a comment -

          Sorry Ryan - fell off my radar this weekend. I'll try and look at this sometime after I wake up today.

          Show
          Mark Miller added a comment - Sorry Ryan - fell off my radar this weekend. I'll try and look at this sometime after I wake up today.
          Hide
          Ryan McKinley added a comment -

          Mark, can you take a look at this?

          removes all logging from solr.war and tries to copy files to example/lib

          for some reason the log4j dependency does not copy into the that folder

          Show
          Ryan McKinley added a comment - Mark, can you take a look at this? removes all logging from solr.war and tries to copy files to example/lib for some reason the log4j dependency does not copy into the that folder
          Hide
          Mark Miller added a comment -

          I will switch things around to pull the logging jars out of the war and put them in the example lib folder

          I'm happy to help out too - I've been meaning to get to this and just have not yet. Happy to let someone else do it, but if you need any help, I have a strong interest in making this work well.

          Show
          Mark Miller added a comment - I will switch things around to pull the logging jars out of the war and put them in the example lib folder I'm happy to help out too - I've been meaning to get to this and just have not yet. Happy to let someone else do it, but if you need any help, I have a strong interest in making this work well.
          Hide
          Mark Miller added a comment -

          I can dig it up, but am much happier to drop it from the .war!

          Great, this is my feeling - a lot has changed in a year - we have a new discussion here that seems to be making progress and has a lot of visibility. If someone wants to toss a monkey wrench in, the lane is wide open

          Show
          Mark Miller added a comment - I can dig it up, but am much happier to drop it from the .war! Great, this is my feeling - a lot has changed in a year - we have a new discussion here that seems to be making progress and has a lot of visibility. If someone wants to toss a monkey wrench in, the lane is wide open
          Hide
          Ryan McKinley added a comment -

          > Where and what is someone else arguing?

          I am remembering discussion from a long time ago (year+) the last time I really paid attention to this discussion. I can dig it up, but am much happier to drop it from the .war!

          I will switch things around to pull the logging jars out of the war and put them in the example lib folder

          Are there concrete patches on other issues that I ignored? The key stuff in this one is for the admin UI managing log4j levels.

          Show
          Ryan McKinley added a comment - > Where and what is someone else arguing? I am remembering discussion from a long time ago (year+) the last time I really paid attention to this discussion. I can dig it up, but am much happier to drop it from the .war! I will switch things around to pull the logging jars out of the war and put them in the example lib folder Are there concrete patches on other issues that I ignored? The key stuff in this one is for the admin UI managing log4j levels.
          Hide
          Mark Miller added a comment -

          but last we discussed inertia pointed towords keeping concrete logging dependencies in solr.war

          Why? I don't see anyone arguing for that here.

          There seems to be plenty of advantageous to getting it out of the webapp and plenty of downsides to having it in.

          Where and what is someone else arguing?

          Show
          Mark Miller added a comment - but last we discussed inertia pointed towords keeping concrete logging dependencies in solr.war Why? I don't see anyone arguing for that here. There seems to be plenty of advantageous to getting it out of the webapp and plenty of downsides to having it in. Where and what is someone else arguing?
          Hide
          Christian Moen added a comment -

          Mark, have you tried Logback? That's a good logging implementation; arguably a better one.

          David and Mark, I believe Log4J 2 addresses a lot of the weaknesses in Log4J 1.x also addressed by Logback. However, Log4J 2 hasn't been released yet.

          To me it sounds like a good idea to use Log4J 1.x now and move to Log4J 2 in the future.

          Show
          Christian Moen added a comment - Mark, have you tried Logback? That's a good logging implementation; arguably a better one. David and Mark, I believe Log4J 2 addresses a lot of the weaknesses in Log4J 1.x also addressed by Logback. However, Log4J 2 hasn't been released yet. To me it sounds like a good idea to use Log4J 1.x now and move to Log4J 2 in the future.
          Hide
          Ryan McKinley added a comment -

          Does that mean:

          • no logging slf4j/log4j in .war (like dist-war-excl-slf4j)
          • put logging files slf4j+log4j in example/lib

          I'm a big +1 to that

          but last we discussed inertia pointed towords keeping concrete logging dependencies in solr.war

          Show
          Ryan McKinley added a comment - Does that mean: no logging slf4j/log4j in .war (like dist-war-excl-slf4j) put logging files slf4j+log4j in example/lib I'm a big +1 to that but last we discussed inertia pointed towords keeping concrete logging dependencies in solr.war
          Hide
          Mark Miller added a comment -

          I'd prefer a solution closer to what we are discussing.

          There are also other things to look at I think:

          *existing log ant targets - are they now broken?
          *existing pre logging setup in jetty.xml and stuff in the README
          *an example log4j conf file rather than the java util one
          *consider switching our tests to it so that devs actually deal with what users will see

          I think that Jan is on the right track for how we should tackle the switch.

          Show
          Mark Miller added a comment - I'd prefer a solution closer to what we are discussing. There are also other things to look at I think: *existing log ant targets - are they now broken? *existing pre logging setup in jetty.xml and stuff in the README *an example log4j conf file rather than the java util one *consider switching our tests to it so that devs actually deal with what users will see I think that Jan is on the right track for how we should tackle the switch.
          Hide
          Ryan McKinley added a comment -

          This patch switches things to log4j

          I'm not wild about adding log4j to the .war, but that is the closest to what we currently do with JUL

          Docs and sample configs would need to be updated too...

          Show
          Ryan McKinley added a comment - This patch switches things to log4j I'm not wild about adding log4j to the .war, but that is the closest to what we currently do with JUL Docs and sample configs would need to be updated too...
          Hide
          Mark Miller added a comment -

          Mark, have you tried Logback?

          I have not, I'll take a look!

          Show
          Mark Miller added a comment - Mark, have you tried Logback? I have not, I'll take a look!
          Hide
          David Smiley added a comment -

          I hear ya on JUL sucking. My first involvement with the Solr community was on this very issue as I railed against it and pushed for SLF4J. I'm glad me and those that pushed with me (Ryan, ...) won out.

          Mark, have you tried Logback? That's a good logging implementation; arguably a better one.

          Show
          David Smiley added a comment - I hear ya on JUL sucking. My first involvement with the Solr community was on this very issue as I railed against it and pushed for SLF4J. I'm glad me and those that pushed with me (Ryan, ...) won out. Mark, have you tried Logback? That's a good logging implementation; arguably a better one.
          Hide
          Mark Miller added a comment -

          Thanks for the details Shawn!

          Show
          Mark Miller added a comment - Thanks for the details Shawn!
          Hide
          Shawn Heisey added a comment - - edited

          here's the logging jars, in jetty's lib/ext:

          [root@bigindy5 ~]# ls -al /opt/solr4/lib/ext/
          total 548
          drwxrwxr-x  2 ncindex ncindex   4096 Nov 30 10:59 .
          drwxr-xr-x. 3 ncindex ncindex   4096 Feb  3 13:38 ..
          -rw-r--r--  1 ncindex ncindex  16457 Oct 20 13:41 jcl-over-slf4j-1.7.2.jar
          -rw-rw-r--  1 ncindex ncindex 489883 Oct 20 13:41 log4j-1.2.17.jar
          -rw-r--r--  1 ncindex ncindex  26083 Oct 20 13:41 slf4j-api-1.7.2.jar
          -rw-r--r--  1 ncindex ncindex   8819 Oct 20 13:41 slf4j-log4j12-1.7.2.jar
          

          and for comparison purposes, here's the lib in solr.solr.home:

          [root@bigindy5 ~]# ls /index/solr4/lib
          icu4j-49.1.jar
          lucene-analyzers-icu-4.2-SNAPSHOT.jar
          mysql-connector-java-5.1.22-bin.jar
          solr-dataimporthandler-4.2-SNAPSHOT.jar
          solr-dataimporthandler-extras-4.2-SNAPSHOT.jar
          
          Show
          Shawn Heisey added a comment - - edited here's the logging jars, in jetty's lib/ext: [root@bigindy5 ~]# ls -al /opt/solr4/lib/ext/ total 548 drwxrwxr-x 2 ncindex ncindex 4096 Nov 30 10:59 . drwxr-xr-x. 3 ncindex ncindex 4096 Feb 3 13:38 .. -rw-r--r-- 1 ncindex ncindex 16457 Oct 20 13:41 jcl-over-slf4j-1.7.2.jar -rw-rw-r-- 1 ncindex ncindex 489883 Oct 20 13:41 log4j-1.2.17.jar -rw-r--r-- 1 ncindex ncindex 26083 Oct 20 13:41 slf4j-api-1.7.2.jar -rw-r--r-- 1 ncindex ncindex 8819 Oct 20 13:41 slf4j-log4j12-1.7.2.jar and for comparison purposes, here's the lib in solr.solr.home: [root@bigindy5 ~]# ls /index/solr4/lib icu4j-49.1.jar lucene-analyzers-icu-4.2-SNAPSHOT.jar mysql-connector-java-5.1.22-bin.jar solr-dataimporthandler-4.2-SNAPSHOT.jar solr-dataimporthandler-extras-4.2-SNAPSHOT.jar
          Hide
          Shawn Heisey added a comment -

          I think you need to remove all slf4j jars from the war for it to work. When slf4j-api.jar is in the war you won't be able to load slf4j-log4j*.jar from external location. Also I believe various servlet container implementations may require you to put your logging classes in different places for it to work. For wrapping up /example/ I believe it will work put them all in /example/lib. Try it.

          This is exactly right. I created a patch that made a "dist-excl-slf4j" build target, and modified the dist-war-excl-slf4j target. Hoss committed it for me in SOLR-3918, on the condition that I updated the wiki, which I did. Now all slf4j jars are removed from the war.

          I use the jetty included with the example in production, and log4j finally worked when I put the slf4j and log4j jars in lib/ext.

          Show
          Shawn Heisey added a comment - I think you need to remove all slf4j jars from the war for it to work. When slf4j-api.jar is in the war you won't be able to load slf4j-log4j*.jar from external location. Also I believe various servlet container implementations may require you to put your logging classes in different places for it to work. For wrapping up /example/ I believe it will work put them all in /example/lib. Try it. This is exactly right. I created a patch that made a "dist-excl-slf4j" build target, and modified the dist-war-excl-slf4j target. Hoss committed it for me in SOLR-3918 , on the condition that I updated the wiki, which I did. Now all slf4j jars are removed from the war. I use the jetty included with the example in production, and log4j finally worked when I put the slf4j and log4j jars in lib/ext.
          Hide
          Mark Miller added a comment -

          Also, if I remember right, the reason I put it in the jetty lib dir in the past and another advantage is that jetty will then use it as well - instead of it's weird mortbay whatever logging stuff.

          Show
          Mark Miller added a comment - Also, if I remember right, the reason I put it in the jetty lib dir in the past and another advantage is that jetty will then use it as well - instead of it's weird mortbay whatever logging stuff.
          Hide
          Mark Miller added a comment -

          Great, that will probably save me some time working this out. I'll play around with this when I get home next week.

          Show
          Mark Miller added a comment - Great, that will probably save me some time working this out. I'll play around with this when I get home next week.
          Hide
          Jan Høydahl added a comment -

          I tried something like that a while back. I think you need to remove all slf4j jars from the war for it to work. When slf4j-api.jar is in the war you won't be able to load slf4j-log4j*.jar from external location. Also I believe various servlet container implementations may require you to put your logging classes in different places for it to work. For wrapping up /example/ I believe it will work put them all in /example/lib. Try it.

          I agree that log4j is a better out-of-the box experience with the Jetty we ship, and a decent config which rotates etc.

          Show
          Jan Høydahl added a comment - I tried something like that a while back. I think you need to remove all slf4j jars from the war for it to work. When slf4j-api.jar is in the war you won't be able to load slf4j-log4j*.jar from external location. Also I believe various servlet container implementations may require you to put your logging classes in different places for it to work. For wrapping up /example/ I believe it will work put them all in /example/lib. Try it. I agree that log4j is a better out-of-the box experience with the Jetty we ship, and a decent config which rotates etc.
          Hide
          Mark Miller added a comment -

          solr/example/lib

          Have you actually tried that? I was talking to someone the other day that said they had to package this in the war to get it to work - but I'm pretty sure I remember getting it to work by putting it int he jetty lib dir in the past.

          Show
          Mark Miller added a comment - solr/example/lib Have you actually tried that? I was talking to someone the other day that said they had to package this in the war to get it to work - but I'm pretty sure I remember getting it to work by putting it int he jetty lib dir in the past.
          Hide
          Mark Miller added a comment -

          I don't really where the jars end up - I just want decent logging out of the box - that pretty much leaves you with log4j.

          Show
          Mark Miller added a comment - I don't really where the jars end up - I just want decent logging out of the box - that pretty much leaves you with log4j.
          Hide
          Jan Høydahl added a comment -

          I still think it would be better to not inlcude any slf4j binding in the war but place our chosen default in solr/example/lib, so people easily can swap things out without repacking solr.war. I don't have an explicit log preference, it should be up to the customer to choose at deploy time.

          Show
          Jan Høydahl added a comment - I still think it would be better to not inlcude any slf4j binding in the war but place our chosen default in solr/example/lib, so people easily can swap things out without repacking solr.war. I don't have an explicit log preference, it should be up to the customer to choose at deploy time.

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development