Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10694

Remove synchronized input streams from Writable deserialization

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Remove invisible synchronization primitives from DataInputBuffer

      Description

      Writable deserialization is slowing down due to a synchronized block within DataInputBuffer$Buffer.

      ByteArrayInputStream::read() is synchronized and this shows up as a slow uncontested lock.

      Hive ships with its own faster thread-unsafe version with hive.common.io.NonSyncByteArrayInputStream.

      The DataInputBuffer and Writable deserialization should not require a lock per readInt()/read().

        Attachments

        1. writable-read-sync.png
          255 kB
          Gopal V
        2. HADOOP-10694.1.patch
          2 kB
          Gopal V
        3. HADOOP-10694.2.patch
          3 kB
          Rajesh Balamohan

          Issue Links

            Activity

              People

              • Assignee:
                gopalv Gopal V
                Reporter:
                gopalv Gopal V
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: