1. Ivy
  2. IVY-1060

ApacheURLLister.retrieveListing() fails if the encoding of the URL list is different from the default encoding


    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0, 2.1.0, 2.3.0-RC1
    • Fix Version/s: 2.3.0-RC2, 2.4.0-RC1
    • Component/s: Core
    • Labels:
    • Environment:

      OS: z/OS 1.9

      java version "1.6.0"
      Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))


      ApacheURLLister.retrieveListing() assumes that the list of URLs is encoded in the same encoding as the system's default encoding.

      The problematic code is:

      BufferedReader r = new BufferedReader(new InputStreamReader(URLHandlerRegistry.getDefault().openStream(url)));
      String htmlText = FileUtil.readEntirely(r);

      FileUtil.readEntirely() converts the the content of the BufferedReader r to a String. Because no encoding is specified in the InputStreamReader constructor, the default encoding is used. If the default encoding does not match the actual encoding of the data read from url, htmlText ends up as a garbage String and the URL pattern matcher fails.

      This causes an issue on z/OS, where the default encoding is EBCDIC (e.g. IBM-1047) but the data containing the list of URLs is typically retrieved from the network as ASCII (ISO-8559-1).

      A workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1.


        Robin Fernandes created issue -
        Robin Fernandes made changes -
        Field Original Value New Value
        Attachment patch.ivy.URLListerEncoding.diff [ 12404991 ]
        Robin Fernandes made changes -
        Environment Observed on z/OS OS: z/OS 1.9

        java version "1.6.0"
        Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
        Nicolas Lalevée made changes -
        Labels patch
        Nicolas Lalevée made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Nicolas Lalevée [ hibou ]
        Fix Version/s 2.3.0 [ 12320743 ]
        Fix Version/s trunk [ 12320744 ]
        Resolution Fixed [ 1 ]
        Maarten Coene made changes -
        Fix Version/s 2.4.0-RC1 [ 12326648 ]
        Fix Version/s trunk [ 12320744 ]


          • Assignee:
            Nicolas Lalevée
            Robin Fernandes
          • Votes:
            0 Vote for this issue
            2 Start watching this issue


            • Created: