Maven WAR Plugin
  1. Maven WAR Plugin
  2. MWAR-164

Support for specifying which encoding to use when filtering resources

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.1-alpha-1
    • Fix Version/s: 2.3
    • Component/s: filtering
    • Labels:
      None

      Description

      Quoting Hervé:

      Maven filtering provides an encoding parameter to set encoding used when
      reading/writing files. But war plugin uses null value, which means platform
      encoding... Sorry, encoding support won't be totally "free"

      I added TODOs in the code.

      For web.xml and container config XML file, I set encoding to UTF-8, which is a
      better default value than platform encoding.

      For other filtered resources, you'll need to add an encoding attribute to
      o.a.m.model.Resource class, to let the user define which encoding he wants to
      use when filtering. Perhaps using project.build.sourceEncoding as a default
      value is a good idea.
      Seems like this is worth a Jira issue to track this new feature.

        Issue Links

          Activity

          Hide
          Dennis Lundberg added a comment -

          I stumbled upon this issue today when using filteringDeploymentDescriptors.

          Filtering the web.xml file should be able to use the encoding specified in the web.xml file itself. We specify ISO-8859-1 as the encoding in our web.xml, but it ends up scrambled into UTF-8 after being filtered.

          This is our configuration:

                 <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-war-plugin</artifactId>
                    <version>2.1-alpha-2</version>
                    <configuration>
                      <archive>
                        <manifest>
                          <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
                          <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
                        </manifest>
                      </archive>
                      <archiveClasses>true</archiveClasses>
                      <filteringDeploymentDescriptors>true</filteringDeploymentDescriptors>
                    </configuration>
                  </plugin>
          
          Show
          Dennis Lundberg added a comment - I stumbled upon this issue today when using filteringDeploymentDescriptors. Filtering the web.xml file should be able to use the encoding specified in the web.xml file itself. We specify ISO-8859-1 as the encoding in our web.xml, but it ends up scrambled into UTF-8 after being filtered. This is our configuration: <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>2.1-alpha-2</version> <configuration> <archive> <manifest> <addDefaultImplementationEntries>true</addDefaultImplementationEntries> <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries> </manifest> </archive> <archiveClasses>true</archiveClasses> <filteringDeploymentDescriptors>true</filteringDeploymentDescriptors> </configuration> </plugin>
          Hide
          Dennis Lundberg added a comment -

          Bug fix for filteringDeploymentDescriptors coming up.

          Show
          Dennis Lundberg added a comment - Bug fix for filteringDeploymentDescriptors coming up.
          Hide
          Dennis Lundberg added a comment -

          Created MWAR-183 specifically for filteringDeploymentDescriptors.

          Show
          Dennis Lundberg added a comment - Created MWAR-183 specifically for filteringDeploymentDescriptors.
          Hide
          Stephane Nicoll added a comment -

          Hey Dennis, are we sure we want to do this?

          Why do we have two strategies here (enconding specified in the xml and a parameter for filtered resources). The resources plugin does not offer such feature AFAIK ...

          I'd go with something similar to the resources plugin and that would be enough,right?

          Show
          Stephane Nicoll added a comment - Hey Dennis, are we sure we want to do this? Why do we have two strategies here (enconding specified in the xml and a parameter for filtered resources). The resources plugin does not offer such feature AFAIK ... I'd go with something similar to the resources plugin and that would be enough,right?
          Hide
          Benjamin Bentmann added a comment -

          Why do we have two strategies here (enconding specified in the xml and a parameter for filtered resources).

          XML files come with the luxus of an embedded encoding declaration and this declaration has to be obeyed or interpolation fails, leaving the input either unaltered or corrupted (it's a fatal error if the encoding does not match the file's byte stream). Most other resources files (i.e. non-XML documents), are not equipped with an embedded encoding declaration so the user has to provide the encoding in form of a parameter.

          The Resources Plugin or better the filtering component should ideally be enhanced to detect XML files and respect their encoding.

          Show
          Benjamin Bentmann added a comment - Why do we have two strategies here (enconding specified in the xml and a parameter for filtered resources). XML files come with the luxus of an embedded encoding declaration and this declaration has to be obeyed or interpolation fails, leaving the input either unaltered or corrupted (it's a fatal error if the encoding does not match the file's byte stream). Most other resources files (i.e. non-XML documents), are not equipped with an embedded encoding declaration so the user has to provide the encoding in form of a parameter. The Resources Plugin or better the filtering component should ideally be enhanced to detect XML files and respect their encoding.
          Hide
          Dennis Lundberg added a comment -

          I agree with Stephane and Benjamin. We should simply use any encoding specified in the files to be filtered. This will at least work for xml files, since they have the ability for the author to specify the encoding being used. Ideally this should be handled by the shared filtering component.

          Show
          Dennis Lundberg added a comment - I agree with Stephane and Benjamin. We should simply use any encoding specified in the files to be filtered. This will at least work for xml files, since they have the ability for the author to specify the encoding being used. Ideally this should be handled by the shared filtering component.
          Hide
          Olivier Lamy (*$^¨%`£) added a comment - - edited

          So we need to detect *.xml files (in a harcoded way ?).
          This need some changes in DefaultMavenFileFilter#copyFile.
          Something like
          pseudo code without any formating

          if ( file.endWith (*.xml) )
          encoding = ReaderFactory.newXmlReader( from ).getEncoding();
          

          BTW this new way of filtering should be off by default (and configurable).
          As this can break some builds.
          WDYT ?

          Show
          Olivier Lamy (*$^¨%`£) added a comment - - edited So we need to detect *.xml files (in a harcoded way ?). This need some changes in DefaultMavenFileFilter#copyFile. Something like pseudo code without any formating if ( file.endWith (*.xml) ) encoding = ReaderFactory.newXmlReader( from ).getEncoding(); BTW this new way of filtering should be off by default (and configurable). As this can break some builds. WDYT ?
          Hide
          Ognjen Blagojevic added a comment -

          Herve is right. "Perhaps using project.build.sourceEncoding as a default value is a good idea."

          Plus a Boolean parameter (searchEncodingInFiles) which will specify to open XML/JSP/HTML... files to look for specified encoding. This is of course harder to implement and less valuable, so maybe could be separate bug?

          project.build.sourceEncoding covers most use cases, I think.

          Regards,
          Ognjen

          Show
          Ognjen Blagojevic added a comment - Herve is right. "Perhaps using project.build.sourceEncoding as a default value is a good idea." Plus a Boolean parameter (searchEncodingInFiles) which will specify to open XML/JSP/HTML... files to look for specified encoding. This is of course harder to implement and less valuable, so maybe could be separate bug? project.build.sourceEncoding covers most use cases, I think. Regards, Ognjen
          Hide
          Florian Fray added a comment -

          I was facing this problem in our builds as well, so I've worked around it by setting the JVM file.encoding-property.
          It works for me, but not for my team, so I'd like to fix it permanently.

          For now my preference is to keep the existing behaviour as default (which is odd, but well known). In my patch I've added a property named resourceEncoding to the AbstractWarMojo.

          I've added this to my parent-poms:

          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-war-plugin</artifactId>
            <version>2.2-SNAPSHOT</version>
            <configuration>
              <resourceEncoding>${project.build.sourceEncoding}</resourceEncoding>
            </configuration>
          </plugin>
          

          In order to test this I've modified the web-resources-filtering-Integration-Test and hope this will pass the review

          Please find my patch attached to this bug-report.

          Best regards,

          Florian

          Show
          Florian Fray added a comment - I was facing this problem in our builds as well, so I've worked around it by setting the JVM file.encoding -property. It works for me, but not for my team, so I'd like to fix it permanently. For now my preference is to keep the existing behaviour as default (which is odd, but well known). In my patch I've added a property named resourceEncoding to the AbstractWarMojo. I've added this to my parent-poms: <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-war-plugin</artifactId> <version>2.2-SNAPSHOT</version> <configuration> <resourceEncoding>${project.build.sourceEncoding}</resourceEncoding> </configuration> </plugin> In order to test this I've modified the web-resources-filtering -Integration-Test and hope this will pass the review Please find my patch attached to this bug-report. Best regards, Florian
          Hide
          Florian Fray added a comment -

          Patch-file for MWAR-164 adding a "resourceEncoding"-property to the AbstractWarMojo.

          Show
          Florian Fray added a comment - Patch-file for MWAR-164 adding a "resourceEncoding"-property to the AbstractWarMojo.
          Hide
          Florian Fray added a comment -

          Hi!

          Has somebody had the chance to review this patch ?

          Best regards,

          Florian

          Show
          Florian Fray added a comment - Hi! Has somebody had the chance to review this patch ? Best regards, Florian
          Hide
          Florian Fray added a comment -

          Could somebody please review the patch and respond to this issue?

          Best regards,

          Florian

          Show
          Florian Fray added a comment - Could somebody please review the patch and respond to this issue? Best regards, Florian
          Hide
          Steve Davids added a comment - - edited

          Please review patch and/or fix differently, thanks.

          12 Votes and patch exists for almost a year now ...

          I am in favor of being consistent with the resources plugin: http://maven.apache.org/plugins/maven-resources-plugin/resources-mojo.html#encoding

          http://docs.codehaus.org/display/MAVENUSER/POM+Element+for+Source+File+Encoding

          Show
          Steve Davids added a comment - - edited Please review patch and/or fix differently, thanks. 12 Votes and patch exists for almost a year now ... I am in favor of being consistent with the resources plugin: http://maven.apache.org/plugins/maven-resources-plugin/resources-mojo.html#encoding http://docs.codehaus.org/display/MAVENUSER/POM+Element+for+Source+File+Encoding
          Hide
          Dennis Lundberg added a comment -

          I have committed a fix for this in http://svn.apache.org/viewvc?view=revision&revision=1388368 complete with an IT. The default encoding used is project.build.sourceEncoding, as suggested.

          I went the more difficult path and did not set the configured encoding for xml files. Intead for xml files (currently files ending in ".xml") the encoding is read from the files themselves. This means that you can have a mix of encodings in your resource files. The IT has a properties file encoded in ISO-8859-1 and an xml file encoded in UTF-8 which is also specified in the files xml header. Apart from the IT I've successfully tried this on several local projects that suffer from this bug.

          I have not yet deployed a new SNAPSHOT, but will do so later today. Please help test this.

          Also I want to add some documentation for this, apart from the parameter documentation that went in with this commit, before I close the issue.

          Show
          Dennis Lundberg added a comment - I have committed a fix for this in http://svn.apache.org/viewvc?view=revision&revision=1388368 complete with an IT. The default encoding used is project.build.sourceEncoding, as suggested. I went the more difficult path and did not set the configured encoding for xml files. Intead for xml files (currently files ending in ".xml") the encoding is read from the files themselves. This means that you can have a mix of encodings in your resource files. The IT has a properties file encoded in ISO-8859-1 and an xml file encoded in UTF-8 which is also specified in the files xml header. Apart from the IT I've successfully tried this on several local projects that suffer from this bug. I have not yet deployed a new SNAPSHOT, but will do so later today. Please help test this. Also I want to add some documentation for this, apart from the parameter documentation that went in with this commit, before I close the issue.
          Hide
          Dennis Lundberg added a comment -

          A new 2.3-SNAPSHOT has now been deployed.
          Please help us test this issue.

          Show
          Dennis Lundberg added a comment - A new 2.3-SNAPSHOT has now been deployed. Please help us test this issue.
          Hide
          Dennis Lundberg added a comment -

          Documentation added in r1391312.

          Show
          Dennis Lundberg added a comment - Documentation added in r1391312 .

            People

            • Assignee:
              Dennis Lundberg
              Reporter:
              Kai Lilleby
            • Votes:
              12 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development