Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8533

DataInput#readVInt() supports negative numbers although not documented

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.6, 8.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      readVInt() has to return positive numbers (and zero), throw some exception in case of negative numbers.

      While for the sequence of bytes [-1, -1, -1, -1, 15] it returns -1.

      simplifying readVInt up to last readByte (exclusive):

      int i = ((byte)-1) & 0x7F;
      i |= (((byte)-1) & 0x7F) << 7;
      i |= (((byte)-1) & 0x7F) << 14;
      i |= (((byte)-1) & 0x7F) << 21;
      

      Here i = 268435455 or in binary format is 00001111_11111111_11111111_11111111

      Keeping in mind that int is a signed type we have only 3 more bits before overflow happens or in another words (Integer.MAX_VALUE - i) >> 28 == 7 - that's max value could be stored in 5th byte to avoid overflow.

      Instead of

      i |= (b & 0x0F) << 28;
      if ((b & 0xF0) == 0) return i;
      

      has to be

      i |= (b & 0x07) << 28;
      if ((b & 0xF8) == 0) return i;
      

        Attachments

        1. LUCENE-8533_fix_readVInt_javadoc.patch
          0.8 kB
          Vladimir Dolzhenko
        2. readVInt.patch
          5 kB
          Vladimir Dolzhenko

          Activity

            People

            • Assignee:
              uschindler Uwe Schindler
              Reporter:
              vladimir.dolzhenko Vladimir Dolzhenko
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: