Uploaded image for project: 'OODT'
  1. OODT
  2. OODT-483

NumberFormatException when using RSS service to view transfers for large files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5
    • Component/s: file manager
    • Labels:

      Description

      I used File Manager to ingest a large file and then attempted to view the progress of the transfer using the 'cas-product' RSS web application (RSSProductTransferServlet [1]). This caused a NumberFormatException as follows:

      java.lang.NumberFormatException: For input string: "11100111001101000000000000"
      java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
      java.lang.Long.parseLong(Long.java:422)
      java.lang.Long.parseLong(Long.java:468)
      org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusFromXmlRpc(XmlRpcStructFactory.java:86)
      org.apache.oodt.cas.filemgr.util.XmlRpcStructFactory.getFileTransferStatusesFromXmlRpc(XmlRpcStructFactory.java:112)
      org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.getCurrentFileTransfers(XmlRpcFileManagerClient.java:398)
      org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doIt(RSSProductTransferServlet.java:151)
      org.apache.oodt.cas.product.rss.RSSProductTransferServlet.doGet(RSSProductTransferServlet.java:138)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
      

      From further testing I found that this exception occurs for files approximately 10MB in size or larger.

      This exception occurs because the XmlRpcStructFactory class is attempting to set the value of a long variable using a binary representation (i.e. treating the binary number as a decimal number). The size of the binary representation exceeds the maximum capacity of the long because it is not being converted to a decimal representation.

      I traced the problem to the following methods in class XmlRpcStructFactory [2]:

      public static Hashtable<String, Object> getXmlRpcFileTransferStatus(FileTransferStatus status)
      {
        Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
        statusHash.put("bytesTransferred",Long.toBinaryString(status.getBytesTransferred()));
        statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
        statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
        return statusHash;
      }
      
      public static FileTransferStatus getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash) 
      {
        FileTransferStatus status = new FileTransferStatus();
        status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
        status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>) statusHash.get("parentProduct")));
        status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
      statusHash.get("fileRef")));
        return status;
      }
      

      In the getXmlRpcFileTransferStatus method shown above, the following line converts the value returned by 'status.getBytesTransferred()' to a binary representation and stores it as a String in the 'statusHash' Hashtable:

        statusHash.put("bytesTransferred", Long.toBinaryString(status.getBytesTransferred()));
      

      But when the value is retrieved from statusHash in method getFileTransferStatusFromXmlRpc, it isn't assumed to be a binary number:

        status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
      

      For large files, the binary representation exceeds the capacity of a long. In the example above, the input string "11100111001101000000000000" converted directly to a long (i.e. treated as a decimal) will exceed the maximum size of a long (9,223,372,036,854,775,807).

      A simple fix for this would be to add a radix argument to Long.parseLong in method getFileTransferStatusFromXmlRpc, as follows:

        status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(), 2));

      I tested this out on a large file (over 100MB) and it seemed to solve the problem. Would this be an acceptable fix? I'll attach a patch to this issue for reference.

      An alternative solution would be not to store a binary representation in the statusHash Hashtable. But I'm assuming there's a reason why it's converted to binary for the hash.

      [1] http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
      [2] http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java

        Attachments

        1. OODT-483.rlaidlaw.2012-08-06.patch.1.txt
          0.8 kB
          Ross Laidlaw
        2. OODT-483.rlaidlaw.2012-08-06.patch.2.txt
          1 kB
          Ross Laidlaw

          Activity

            People

            • Assignee:
              rlaidlaw Ross Laidlaw
              Reporter:
              rlaidlaw Ross Laidlaw
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: