Uploaded image for project: 'Daffodil'
  1. Daffodil
  2. DAFFODIL-1933

Performance regression with new Base64/Layering changes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: Back End
    • Labels:
      None

      Description

      Nightly tests showed a fairly drastic performance regression (50% reduction in speed) in commit 1ea2290f28: All properties for Base64/layering implemented.

      Skimming the code, one change jumped out at me as a likely cause in ByteBufferDataInputStream.scala:

      @@ -91,7 +90,13 @@ object ByteBufferDataInputStream {
             case _ => {
               // copy the contents of the stream into an array of bytes
               val bos = new ByteArrayOutputStream
      -        IOUtils.copy(in, bos)
      +        var b: Int = 0
      +        while ({
      +          b = in.read()
      +          b != -1
      +        }) {
      +          bos.write(b)
      +        }
               bos.flush()
               bos.close()
               in.close()
      

      This changes the copy from a bulk copy to a byte-by-byte copy, which appears to have pretty drastic performance. A few simple quick tests shows that reverting this change recovers all performance losses.

      Mike, can you confirm if this change can be reverted, or if there was some other reason related to layering to switch to a byte-by-byte copy?

        Attachments

          Activity

            People

            • Assignee:
              dfthompson Dave Thompson
              Reporter:
              slawrence Steve Lawrence
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: