Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8533

DataInput#readVInt() supports negative numbers although not documented

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 7.6, 8.0
    • None
    • None
    • New, Patch Available

    Description

      readVInt() has to return positive numbers (and zero), throw some exception in case of negative numbers.

      While for the sequence of bytes [-1, -1, -1, -1, 15] it returns -1.

      simplifying readVInt up to last readByte (exclusive):

      int i = ((byte)-1) & 0x7F;
      i |= (((byte)-1) & 0x7F) << 7;
      i |= (((byte)-1) & 0x7F) << 14;
      i |= (((byte)-1) & 0x7F) << 21;
      

      Here i = 268435455 or in binary format is 00001111_11111111_11111111_11111111

      Keeping in mind that int is a signed type we have only 3 more bits before overflow happens or in another words (Integer.MAX_VALUE - i) >> 28 == 7 - that's max value could be stored in 5th byte to avoid overflow.

      Instead of

      i |= (b & 0x0F) << 28;
      if ((b & 0xF0) == 0) return i;
      

      has to be

      i |= (b & 0x07) << 28;
      if ((b & 0xF8) == 0) return i;
      

      Attachments

        1. LUCENE-8533_fix_readVInt_javadoc.patch
          0.8 kB
          Vladimir Dolzhenko
        2. readVInt.patch
          5 kB
          Vladimir Dolzhenko

        Activity

          People

            uschindler Uwe Schindler
            vladimir.dolzhenko Vladimir Dolzhenko
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: