Bug 47609 - Failsafe EOL conversion
Summary: Failsafe EOL conversion
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 5
Classification: Unclassified
Component: Native:Packaging (show other bugs)
Version: 5.5.27
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-07-30 11:52 UTC by Sebb
Modified: 2010-03-04 20:30 UTC (History)
0 users



Attachments
List of extensions used in 5.5.28 (572 bytes, text/plain)
2009-07-30 11:52 UTC, Sebb
Details
List of files with no extension (505 bytes, text/plain)
2009-07-30 11:53 UTC, Sebb
Details
fixcrlf use include rather than exclude (2.00 KB, patch)
2009-08-02 08:24 UTC, Sebb
Details | Diff
Improved patch - detab (1.93 KB, patch)
2009-08-02 08:29 UTC, Sebb
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sebb 2009-07-30 11:52:49 UTC
Created attachment 24066 [details]
List of extensions used in 5.5.28

The builds/build.xml file has the following code:

  <target name="package-src-tgz">
    <fixcrlf srcdir="${tomcat.dist}/src"
        excludes="**/*.bin,**/*.bmp,**/*.dia,**/*.exe,**/*.gif,**/*.ico,**/*.jar,**/*.jpg,**/*.pdf,**/*.rtf,**/*.war"
        eol="lf"
        encoding="ISO-8859-1" fixlast="false" />

The intention is to fix all the source files so that they are readable on Unix (LF) systems.

However, the command is inherently unsafe - if a new binary file is added with an extension that is not listed, then the file will be damaged. In general it is impossible to reverse LF-conversion on a binary file.

It would be much safer to use includes instead. If a new source file type is added, then the worst that can happen is that the file is awkard to read on Unix. Unlike the case with binary files, the file can still be converted successfully.

The current list of extensions that are present in the 5.5.28 source archive are included in the file "extensions.txt".

At least two binary extensions (.db, .keystore) are currently missing from the excludes list.

There are quite a few files with no extension. These appear to be all text files, and are listed in the file "no-extension.txt" (duplicate names have been removed).
Comment 1 Sebb 2009-07-30 11:53:24 UTC
Created attachment 24067 [details]
List of files with no extension
Comment 2 Filip Hanik 2009-07-31 16:04:23 UTC
So the only safe way to do it, would be to specify an includes list instead of using an excludes.
Patches welcome
Comment 3 Sebb 2009-08-02 07:26:27 UTC
OK, I will create a patch shortly
Comment 4 Sebb 2009-08-02 08:24:37 UTC
Created attachment 24080 [details]
fixcrlf use include rather than exclude
Comment 5 Sebb 2009-08-02 08:29:53 UTC
Created attachment 24081 [details]
Improved patch - detab
Comment 6 Mark Thomas 2009-11-22 11:31:59 UTC
Modified patch applied to trunk and proposed for 6.0.x and 5.5.x
Comment 7 Konstantin Kolinko 2010-03-04 20:30:20 UTC
Fixed in 6.0.22, 5.5.29.