Uploaded image for project: 'Commons Codec'
  1. Commons Codec
  2. CODEC-166

Base64 could be faster

    XMLWordPrintableJSON

Details

    • Wish
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • 1.7
    • 2.0
    • None

    Description

      Our Base64 consistently performs 3 times slower compared to MiGBase64 and iHarder in the byte[] and String encode() methods.

      We are pretty good on decode(), though a little slower (approx. 33% slower) than MiGBase64.

      We always win in the Streaming methods (MiGBase64 doesn't do streaming). Yay!

      I put together a benchmark. Here's a typical run:

        LARGE DATA new byte[12345]
      
      iHarder...
      encode 486.0 MB/s    decode 158.0 MB/s
      encode 491.0 MB/s    decode 148.0 MB/s
      
      MiGBase64...
      encode 499.0 MB/s    decode 222.0 MB/s
      encode 493.0 MB/s    decode 226.0 MB/s
      
      Apache Commons Codec...
      encode 142.0 MB/s    decode 146.0 MB/s
      encode 138.0 MB/s    decode 150.0 MB/s
      

      I believe the main approach we can consider to improve performance is to avoid array copies at all costs. MiGBase64 even counts the number of valid Base64 characters ahead of time on decode() to precalculate the result's size and avoid any array copying!

      I suspect this will mean writing out separate execution paths for the String and byte[] methods, and keeping them out of the streaming logic, since the streaming logic is founded on array copy.

      Unfortunately this means we will diminish internal reuse of the streaming implementation, but I think it's the only way to improve performance, if we want to.

      Attachments

        1. base64bench.zip
          251 kB
          Julius Davies
        2. CODEC-166_speed.patch
          9 kB
          Thomas Neidhart
        3. CODEC-166.patch
          55 kB
          Julius Davies

        Issue Links

          Activity

            People

              jochen@apache.org Jochen Wiedmann
              juliusdavies Julius Davies
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: