Bug 18462 - System.err gets overriden to output to System.out
Summary: System.err gets overriden to output to System.out
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Catalina (show other bugs)
Version: trunk
Hardware: Other other
: P3 enhancement with 1 vote (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
: 33606 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-03-28 12:35 UTC by Nikolay Metchev
Modified: 2011-01-12 19:58 UTC (History)
1 user (show)



Attachments
[PATCH] Remove SystemLogHandler in start() (809 bytes, patch)
2003-04-22 13:08 UTC, Andy Chapman
Details | Diff
[PATCH] Add support for attribute in <server> tag of server.xml to allow switching of output redirect behaviour (2.87 KB, patch)
2003-04-25 10:10 UTC, Andy Chapman
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Nikolay Metchev 2003-03-28 12:35:32 UTC
I am using Eclipse to do my development and I am running tomcat 4.1.24 from 
within eclipse. One of the nice features of eclipse is that anything you 
display with System.out comes out in blue and anything you send to System.err 
is displayed in red. However when tomcat is running Catalina overrides 
System.err and makes it point to System.out. This is very inconvinient because 
its much harder to search for severe error messages in a list of millions of 
trivial ones.
Comment 1 Andy Chapman 2003-04-22 13:08:05 UTC
Created attachment 5946 [details]
[PATCH] Remove SystemLogHandler in start()
Comment 2 Andy Chapman 2003-04-22 13:09:35 UTC
Redirection of System.out and System.err in 
org.apache.catalina.startup.Catalina.start() prevents any other application 
(for instance an IDE) from redirecting System.out and System.err. It also 
redirects them to the same output (the standard system output stream) 
preventing any differentiation of the two streams.

I cannot see any value in creating a custom SystemLogHandler in this instance 
the attached patch removes the following code.

>> SystemLogHandler log = new SystemLogHandler(System.out);
>> System.setOut(log);
>> System.setErr(log);

Comment 3 Glenn Nielsen 2003-04-22 15:26:17 UTC
Tomcat as a container is used in a number of different ways.
Some use it with an IDE as you do, but there are many others who use
it as a production web application server.

The redirection of System.out and System.err is there to support those
who run Tomcat as a production application server.  It does this so that
application code which generates output to stdout or stderr gets placed in
the appropriate tomcat log for the application.  This makes it easier to
track down problems for a production application because all output generated
by the application will be in the same log.

Your patch is not acceptable because it would break this behaviour for
those who need it on a production server.

You might consider researching this further and submitting a patch
that will work for both cases.  Perhaps a test to not execute the code
if Tomcat is running embedded.  Or a new attribute to disable this.

Comment 4 Michael Eriksson 2003-04-22 15:55:05 UTC
Personally I consider redirection of System.out/System.err in code a bad thing
for other reasons. The point is that when you use e.g. System.out, you
do so because you want it to appear on System.out/stdout/console or whatever
terminology you favour. Thus, the application should never redirekt. Any
and all redirektion should occur outside the application, e.g in a shell script
or a bat file. (Yes, I realize that a lot of people confuse "System.out"
and "logging framework", but that is really their problem.)
Comment 5 Andy Chapman 2003-04-23 12:48:24 UTC
Users redirected System.out via the startup scripts in previous versions of 
Tomcat and this seemed acceptable to all parties. You could leave the output 
going to the console or redirect it to a log file in the standalone 
(production) case, and additionally allow redirection when used within a 
container app such as an IDE. This seemed to be the most comprehensive 
solution, I personally think redirecting System.out from within the app is less 
flexible.

I believe I understand the function of the SystemLogHandler, but think there is 
a difference to the logging requirement and the System.out requirement.
Comment 6 Glenn Nielsen 2003-04-23 15:22:05 UTC
Without the SysteLogHandler and the way it redirects System.out and System.err
to the appropriate tomcat log debugging web application problems on a 
production server where you have 20-30 different web applications with
multiple virtual hosts is a nightmare.  There are too many supporting API's
being used in web applicaitons which don't know they are running in a container
and will print stack traces, etc. to System.out or System.err.  Having output
from the same applicaiton in different logs, or output from multiple applications
in the same log because they are printing to System.out or System.err is a pain.

I will not make the change recommended by the patch.

As I said earlier

"You might consider researching this further and submitting a patch
that will work for both cases.  Perhaps a test to not execute the code
if Tomcat is running embedded.  Or a new attribute to disable this."
Comment 7 Andy Chapman 2003-04-24 11:22:45 UTC
Okay points taken and understood.

Alternatively what I would like to propose is adding an optional attribute to 
the <Server> tag in server.xml. The attribute will be named "noRedirect" and 
will take a boolean value. So the whole tag would look like:

server.xml
...
<Server port="8005" shutdown="SHUTDOWN" debug="0" noRedirect="true">
...

If no attribute is set then the default is false (current behaviour). Setting 
the attribute to "true" (as above) will mean that System.out and System.err are 
NOT redirected.

Summary of changes:

org.apache.catalina.Server
--------------------------
ADD public void setNoRedirect(boolean noRedirect);
ADD public boolean getNoRedirect();

org.apache.catalina.core.StandardServer
---------------------------------------
ADD private boolean noRedirect = false;
ADD public void setNoRedirect(boolean noRedirect) {...}
ADD public boolean getNoRedirect() {...}

org.apache.catalina.startup.Catalina
------------------------------------
CHANGE protected void start() {...

<<        // Replace System.out and System.err with a custom PrintStream
<<        SystemLogHandler log = new SystemLogHandler(System.out);
<<        System.setOut(log);
<<        System.setErr(log);

>>        // Replace System.out and System.err with a custom PrintStream if the
>>        // noRedirect switch is false (default)
>>        if (server.getNoRedirect() == false) {
>>          SystemLogHandler log = new SystemLogHandler(System.out);
>>          System.setOut(log);
>>          System.setErr(log);
>>        }

...}

This involves changing the Server interface however, which is not to be taken 
lightly. An alternative may be to leave the interface and just add the 
noRedirect to StandardServer (as debug is currently), then check the server 
class as follows:

>>        // Replace System.out and System.err with a custom PrintStream if the
>>        // using StandardServer and noRedirect switch is false (default)
>>        if (server instanceof StandardServer) {
>>          if (server.getNoRedirect() == false) {
>>            SystemLogHandler log = new SystemLogHandler(System.out);
>>            System.setOut(log);
>>            System.setErr(log);
>>          }
>>        } else {
>>          SystemLogHandler log = new SystemLogHandler(System.out);
>>          System.setOut(log);
>>          System.setErr(log);
>>        }

I think the change to Server is the most contentious aspect of this proposal. I 
have a patch for the former implementation (with changes to Server). This is 
neater in some respects adding this functionality to all Server 
implementations, but obviously "breaking" any existing implementations other 
than StandardServer.

I'll await comment and if acceptable I'll submit a patch.
Comment 8 Nikolay Metchev 2003-04-24 12:00:38 UTC
Calling the attribute "noRedirect" will cause a lot of confusion because it may 
be interpreted as all kinds of redirection such as HTTP redirection. A better 
name might be "errorStreamRedirect"
Comment 9 Andy Chapman 2003-04-25 10:05:32 UTC
Yes you have a valid point. I don't particularly like the double negative 
noRedirect=false either.

I'll change the patch to use outputRedirect which by default will be true. So 
you would use the following in server.xml to prevent the redirection:

server.xml
...
<Server port="8005" shutdown="SHUTDOWN" debug="0" outputRedirect="false">
...
Comment 10 Andy Chapman 2003-04-25 10:10:08 UTC
Created attachment 6008 [details]
[PATCH] Add support for attribute in <server> tag of server.xml to allow switching of output redirect behaviour
Comment 11 Mark Thomas 2004-04-14 21:09:20 UTC
Marking this an an enhancement.

Not sure how much interest there is in the user community for this patch. A 
quick scan of the last 12 months of tomcat-user didn't highlight this as a 
requirement although I might have missed something.
Comment 12 Konstantin Kolinko 2011-01-10 18:39:36 UTC
Moving from Tomcat 4/4.1.24 to Tomcat 7.

My concern is when Tomcat is installed on Windows: it is by default configured to log err and out to separate files.

Steps to reproduce:
1. Install apache-tomcat-6.0.30.exe with the Examples webapp.
2. Launch Tomcat and navigate to
http://localhost:8080/examples/jsp/simpletag/foo.jsp
3. Look into logs/tomcat6-stdout.2011-01-10.log
Actual result: The file contains the following:
===============================
2011-01-10 23:33:27 Commons Daemon procrun stdout initialized
Did you see me on the stderr window?


Did you see me on the browser window as well?
===============================
Expected result: The "Did you see me on the stderr window?" message should have been printed into *stderr* log, not in *stdout* log.

At the same time the tomcat6-stderr.2011-01-10.log file contains a lot of text, printed by the logging subsystem. It is java.util.logging.ConsoleHandler that prints there.

As was noticed earlier, the culprit is Embedded#initStreams() (TC5.5,6,7) and Catalina#initStreams() (TC7).
>>  // Replace System.out and System.err with a custom PrintStream
>>  SystemLogHandler systemlog = new SystemLogHandler(System.out);
>>  System.setOut(systemlog);
>>  System.setErr(systemlog);

Initialization of JULI happens before this redirection, and thus java.util.logging.ConsoleHandler prints to the original System.err.

My thoughts of this:
1) SystemLogHandler can be enhanced to support separate out and err streams.
2) If we do not fix this, then maybe configure commons-daemon to log both streams to the same file by default.
It is done by setting Stderr name in its configuration to be an empty string. -- See method redirectStdStreams() in prunsrv.c.
3) It is worth being mentioned in the FAQ.
4) This feature is not needed at all if no Context is configured with "swallowOutput=true". So maybe make it configurable and turn it off by default. The name for the configuration property might be "redirectStreams", like it is already used as the field name Embedded#redirectStreams.
Comment 13 Konstantin Kolinko 2011-01-10 18:58:27 UTC
*** Bug 33606 has been marked as a duplicate of this bug. ***
Comment 14 Mark Thomas 2011-01-12 19:58:15 UTC
Thanks to the way SystemLogHandler is written there is a really nice way to handle this whilst retaining the swallow output functionality.

I have applied a patch to 7.0.x that will be in 7.0.7 onwards that does the following:
- treats stdout and stderr separately
- swallowOutput redirects both
- console operation remains unchanged although catalina.sh can now be modified to log stderr and stdout separately if desired
- when running as a windows server, stderr and stdout are logged separately by default