Bug 14095 - Change default Content-Type (DefaultType) in default config
Summary: Change default Content-Type (DefaultType) in default config
Status: RESOLVED DUPLICATE of bug 13986
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: Other Modules (show other bugs)
Version: 2.5-HEAD
Hardware: All other
: P3 normal with 18 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
: 27499 27500 (view as bug list)
Depends on:
Blocks:
 
Reported: 2002-10-30 17:22 UTC by Simon Woodside
Modified: 2007-08-27 10:31 UTC (History)
10 users (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Woodside 2002-10-30 17:22:05 UTC
The default config for Content-Type (set by DefaultType) on Mac OS X is
text/plain (I can't confirm this on other systems). This is just wrong. Perhaps
some time ago, back in the unix everything-is-a-text-file days this made sense,
but today, random unknown flies that aren't in the MIME types table are far more
likely to be binary data than text/plain.

The new default should be either application/octet-stream (which is in the MIME
spec) or application/x-unknown-content-type (which is new to me, but maybe
appropriate). application/octet-stream will allow the user agent to save the
file or prompt the user to make a decision, while
application/x-unknown-content-type will allow the user agent to actually sniff
at the data and try to figure out what it is.

This is particularly a problem for Mozilla. see
http://bugzilla.mozilla.org/show_bug.cgi?id=175848 amongst other bugs (that's
for Chimera light-weight mozilla on Mac OS X).
Comment 1 Boris Zbarsky 2002-10-30 17:34:38 UTC
"application/x-unknown-content-type" is not a registered type and is just
something used internally within Mozilla to mark content with no type specified.
 Please do not start sending this as an actual content-type over the Net, under
any circumstances....

See also bug 13986
Comment 2 Daniel R. Tobias 2002-10-30 18:44:05 UTC
It would be better not to send a content type at all if the type is unknown,
rather than a bogus type.  Technically, under the standards, a user agent is
allowed to sniff for content if no MIME type is sent, but not if one is sent
(even if it's some "x-" value that is unknown).
Comment 3 Simon Woodside 2002-10-30 21:38:51 UTC
OK, forget about application/x-unknown-content-type, obviously I was off in
la-la-land with that one.

As Dan said, once bug 13986 is fixed then the DefaultType should change to
nothing. until then, application/octet-stream is better than status quo (text/plain)
Comment 4 Erik Abele 2004-01-05 21:57:22 UTC

*** This bug has been marked as a duplicate of 13986 ***
Comment 5 Boris Zbarsky 2004-01-05 23:00:55 UTC
Erik, I'm sorry, but I have to reopen this.  The other bug is rather difficult
to resolve, involving changes in core architecture and such (see the objections
raised to my proposed patch).  It's not likely to be a change that happens in a
minor point release.

This bug is requesting a one-line change in the default config file that can be
made by anyone who can type.  It could happen in a maintenance release, imo...
(and should have happened by now if the Apache group is at all interested in
trying to resolve the browser makers' concerns about this behavior).

To be truthful, I am _very_ close to just hardcoding a check for the Apache
server in Mozilla and just ignoring any type sent by such servers, since it's
clearly bogus so much of the time.  Our built-in content-sniffing would do a
better job.  It pains me to say this, but IIS has the right behavior here; I
can't figure out why the Apache group is so set on screwing over people trying
to write browsers complying with the HTTP standards.
Comment 6 André Malo 2004-01-05 23:57:08 UTC
you're  at the wrong group of people. We are not responsible for people who are
not configuring theirs servers properly.

That is, I don't see, how a sample server config file a browser's HTTP
compliance affects. Please calm down a bit.

Said that, I've startet a thread on dev@ wrt the issue anyway.
Comment 7 André Malo 2004-01-05 23:58:08 UTC
"you're looking at the wrong ..."
Comment 8 Erik Abele 2004-01-06 00:23:13 UTC
Boris, no need to be sorry for reopening, but I've to disagree with you :)

You're right that #13986 is much more complex to resolve and I'm basically in favour of the 
proposed solution but the issue raised here at #14095 is simply a workaround for the other bug.

As Dan and Simon already put it: "once bug #13986 is fixed then the DefaultType should change to 
nothing. until then, application/octet-stream is better than status quo (text/plain)".

> This bug is requesting a one-line change in the default config file that can be
> made by anyone who can type.

Right and everyone can do so in his/her httpd.conf. No problem here. IMHO the problem would be 
the same with 'application/octet-stream', just the other way round... Again, the _real_ solution 
would be to have #13986 fixed, at least in my eyes.
Comment 9 Boris Zbarsky 2004-01-06 00:37:49 UTC
André, I realize that you just distribute a sample config.  Too bad most people
just use it without modifying it...

The reason your config affects browser compliance is that IE ignores the type
you send.  That means that all other browsers look "broken" when they follow the
HTTP, since IE clearly "works" (does what the person serving the page wanted it
to do).  This means that other browsers either have to fail on many real-world
pages or break the HTTP spec; in practice they are starting to choose the latter
at this point.

I've been pretty calm for over a year now (ever since bug 13986 got filed and I
attempted to start a thread on the issue on the dev list only to be completely
ignored).  In that time, I have had to deal with dozens of Mozilla bug reports
requesting that we simply ignore the type because of Apache's default config (I
am not counting the hundreds of bug reports filed in the same period of time
that got reassigned to evangelism to contact the site and ask them to pretty
please make the change that this bug requests be made in the default config). 
It's hard to be calm when so much of my time is being wasted on this issue,
which is why I am considering explicitly checking for Apache at this point -- it
would simply save me enormous amounts of time, especially if there are no plans
to address either this bug or bug 13986.

Erik, as I pointed out the default setup of IIS is to send
application/octet-stream as the "unknown data" type.  There would be no problem
in the reverse direction, because as soon as Apache started sending
application/octet-stream people who fail to configure their servers would see
their pages "break" in IE AND WOULD FIX THEIR SETUPS.  Which is what we all
want.  It's unfortunate that the MIME bugs in IE make the text/plain behavior
acceptable because, on average, the users simply don't notice it.  Changing to a
type they _do_ notice would have very beneficial consequences, imo.
Comment 10 Henri Sivonen 2004-01-06 18:22:19 UTC
This is a very real problem at the browser end. The easiest way to fix it (even
for people who take "sample" files for face value and don't double-check
everything themselves) is in the Apache default config. The fix is extremely
easy to implement. Why, oh why, can't the fix go in now? We could *then* wait
for bug 13986. 

To turn the situation around: Anyone who is informed and still wants the
text/plain default could be expected to change the default config. (And I think
turning the expectation around this way is more realistic than just shrugging
the issue off by saying that each and every server admins should fix this
separately.)
Comment 11 André Malo 2004-01-06 19:01:08 UTC
Funny. What are you talking about? Guess what I'm going to do if there there
come no objections in the thread I've started? Did you read my posting?

Folks, please tone down. There's no reason to become rude. We're not the ones
who fail to configure their server. I don't see why someone should take any
action if he listens these attitudes all the time.
Comment 12 Joe Llywelyn Griffith Blakesley 2004-01-06 19:21:03 UTC
I would like to point out that this bug (xor bug 13986 -- whichever way you look
at it) causes the binary executables of the Apache HTTP Server for Microsoft
Windows (on the http://apache.org download page) to be sent as plain/text (so
they display within the browser window), where the mantainers of the mirrors of
the Apache.org server have made the (incorrect) assumption that Apache works
correctly (i.e.: sends correct Internet content-types) by default (whereas, in
actual fact, it sends incorrect and particularly specific* ones by default when
it has absolutely no idea of the actual content-type of the data).  I filed this
problem as bug 25650 a while back.

I would also like to second the penultimate and the triultimate post which
explain why it *is* _preferable_ to sometimes send application/octet-stream as
the content type when it should be text/plain, as opposed to sometimes sending
text/plain as the content type when it should be application/octet-stream . 
Obviously it is vastly preferable to either method to send nothing when unsure
(i.e.:to fix bug 13986) but in the mean time...

*that is, application/octet-stream is IMO less specific than text/plain .
Comment 13 Joshua Slive 2004-01-06 21:00:28 UTC
As I pointed out in bug 25650, the underlying problem is configuration,
not any bug in the server.  Apache expects users to configure mime types.

The problems here are:
1. Users who don't configure mime types correctly; and
2. MSIE, which created an expectation that incorrect configurations
work.  This is not only wrong, it is also a security problem.

Individually, neither of these problems would be a big deal.  Together,
they create an issue that we probably should address.

One thing to think about, Andre: DefaultType actually serves two purposes.
1. It determines the type for files with unmatched extensions.
2. It determines the type for files with NO extensions.

Files of the second type are quite likely to be text/plain
(think README, etc), which is probably why that is currently the default.  
And Apache provides no easy way to set the type of these files other than
DefaultType.

But given the balance of harm, it is probably still a good idea both to
change the DefeaultType, and to allow for no Content-Type.

Everyone should be aware, however, that this won't magically fix all the
broken servers out there.  Servers with this type of problem are also the
ones that are least likely to have an admin that will carefully inspect
changes in default config files.
Comment 14 William A. Rowe Jr. 2004-01-13 22:05:49 UTC
  Quoting an earlier comment...

  "application/octet-stream will allow the user agent to save the file or prompt 
   the user to make a decision, while application/x-unknown-content-type will 
   allow the user agent to actually sniff at the data and try to figure out what 
   it is."

  is entirely wrong w.r.t. Microsoft IE.

  IE has the irritating distinction of treating BOTH text/plain and also
  application/octet-stream as content type '*', parsing the data for some 
  possible meaning.  This causes IE to succumb to all sorts of mallicious
  purposes and undesirable side-effects of file naming.  (e.g. refusing to
  simply display a nice text/plain result of a result uri named 'test.pl',
  but instead insisting it aught to be locally executed or saved.)

  I agree with the spirit of the proposal, that our default become
  application/octet-stream until such time as we provide *no* content-type 
  whatsoever if the user has left it unconfigured.
Comment 15 Boris Zbarsky 2004-01-13 22:13:37 UTC
Note that apparently the next IE service pack will significantly scale back the
types of sniffing IE will do (for security reasons, amusingly enough).  As a
result, eg, it will no longer treat text/plain files as HTML even if they look
like HTML.
Comment 16 fantasai 2004-02-25 01:30:53 UTC
Sorry, I'm a little bit confused here. I've read through this bug report, and
there seems to be a disagreement. But I can't figure out over what, unless it's
the duplicate-marking of this report.

If one were to create a patch that changed the default DefaultType in the default
config, as requested by the reporter, would that patch be accepted?
Comment 17 Boris Zbarsky 2004-02-25 01:36:52 UTC
Dirk-Willem (see bug 13986 comment made at 2004-02-14 00:35) says that just
making that change would break the following:

1)  Sending "README" files as plaintext
2)  Interaction with systems like akamai
3)  interaction with certain caching proxies

So the impression I got is that the answer is "no"
Comment 18 Joshua Slive 2004-02-25 15:28:21 UTC
> Dirk-Willem (see bug 13986 comment made at 2004-02-14 00:35) says that just
> making that change would break the following:

> 1)  Sending "README" files as plaintext

This is a legitimate problem.  The ideal solution it to allow the "no extension"
case to be configured separately from the "unknown extension" case.

But I don't think it would hurt anyone severly to switch the config-file 
default to application/octet-stream. People who need text/plain can always
make the switch, and apache now requires README.txt or README.html for
its autoindex files anyway. 

> 2)  Interaction with systems like akamai
> 3)  interaction with certain caching proxies

I have no idea what this means.  How could changing from one wrong mime type
to another (slightly less) wrong mime type hurt proxies?
Comment 19 Boris Zbarsky 2004-02-25 15:47:19 UTC
No idea.  You'll have to ask Dirk-Willem; I never quite understood his
explanation of the proxy issue.
Comment 20 Joshua Slive 2004-03-07 01:16:49 UTC
*** Bug 27499 has been marked as a duplicate of this bug. ***
Comment 21 Joshua Slive 2004-03-07 01:17:46 UTC
*** Bug 27500 has been marked as a duplicate of this bug. ***
Comment 22 Jeff Walden 2007-08-02 12:02:37 UTC
So, honestly, if you're going to WONTFIX I think you should at least explain why...
Comment 23 Nick Kew 2007-08-02 12:25:47 UTC
Changing a long-standing default is a big/disruptive thing on a stable product.
 Let alone a long-obsolete product like Apache 1.3 (the 1998 release).

Feel free to argue your case for future releases.  But you'd do that on the dev
mailinglist, not bugzilla.
Comment 24 Boris Zbarsky 2007-08-02 12:36:42 UTC
For what it's worth, when I brought this up on the dev mailing list almost five
years ago (when the bug was filed) the answer was a deafening silence.  I have
no reason to believe that things would be any different now.

As I said in bug 13986, at this point things are pretty much moot: all the
browsers have been forced to work around this Apache issue by breaking the HTTP
spec and either sniffing for the "real" type (e.g. IE) or just trying to guess
whether the data should really be application/octet-stream (e.g. Gecko) when
they get a type of text/plain.

Given that, trying further to get you guys to change this broken behavior is
just a waste of time: chances are there will be no response anyway, and even if
there is it won't help HTTP UAs, who'll have to continue working around this
problem in all the existing Apache installs.
Comment 25 Nick Kew 2007-08-02 12:50:23 UTC
**** DON'T **** add other people's names to a Cc: list, SPAMMER.

(In reply to comment #24)
> For what it's worth, when I brought this up on the dev mailing list almost five
> years ago (when the bug was filed) the answer was a deafening silence.

You have the advantage of me.  I don't recollect your post, so I can't say
whether it had anything like a coherent argument.

> As I said in bug 13986, at this point things are pretty much moot: all the
> browsers have been forced to work around this Apache issue by breaking the HTTP
> spec and either sniffing for the "real" type (e.g. IE) or just trying to guess
> whether the data should really be application/octet-stream (e.g. Gecko) when
> they get a type of text/plain.

Utter rubbish.  MSIE has *always* broken HTTP.  That's how the first email worms
got into Outlook (which shares the same bug), back in about 1997.

> Given that, trying further to get you guys to change this broken behavior is
> just a waste of time: chances are there will be no response anyway, and even if
> there is it won't help HTTP UAs, who'll have to continue working around this
> problem in all the existing Apache installs.

As it happens, I'd support changing this particular default.  But not
retrospectively.  Nor so strongly as to take the driving seat.  And I'm
certainly not going to argue the case in bugzilla: I just need to remove myself
from getting mailed twice due to some moron messing with Cc: lists.
Comment 26 Boris Zbarsky 2007-08-02 13:01:53 UTC
> **** DON'T **** add other people's names to a Cc: list, SPAMMER.

Customs apparently differ.  In other bugzillas you do that if you're having a
discussion with someone in a bug so that they don't miss part of it.

> Utter rubbish.  MSIE has *always* broken HTTP.

Indeed.  Apache's behavior has just force Gecko, Webkit, Opera, to follow suit.

> But not retrospectively. 

No one's asking for this to be changed in shipped stable software.  At the time
this bug was filed it was filed on what was the current version.  I can't help
the fact that your bug tracking system doesn't have a way to track bugs
applicable to "whatever the current development version is".
Comment 27 Boris Zbarsky 2007-08-25 12:17:54 UTC
OK, let's try getting this fixed in a more recent release.  See also discussion
on the public-html@w3.org mailing list about issues related to server type
misconfiguration and MIME type sniffing on the client side.
Comment 28 Boris Zbarsky 2007-08-25 12:30:46 UTC
Though really, fixing bug 13986 would be much much better.  This bug was just
meant as a temporary workaround pending the work needed to handle modules, etc,
well for that bug...
Comment 29 Roy T. Fielding 2007-08-27 10:31:10 UTC
Changing the default to application/octet-stream is not going to happen.
Anyone can do that in their own server using DefaultType, and if we did
it on all servers that would just lead to yet another sniffed type.

Fixing #13986 is easier (socially, not technically).  Marking this
as a duplicate because it has shared state.


*** This bug has been marked as a duplicate of 13986 ***