Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1456

Use page index, ParquetFileReader throw ArrayIndexOutOfBoundsException

    XMLWordPrintableJSON

Details

    Description

      hi, We use page index to adaptive spark with master branch find a concurrent problem, class org.apache.parquet.internal.column.columnindex.ColumnIndexBuilder#build cached ColumnIndexBuilder but not lock it,

      PrimitiveTypeName typeName = type.getPrimitiveTypeName();
      ColumnIndexBuilder builder = BUILDERS.get(typeName);
      if (builder == null)

      { builder = createNewBuilder(type, Integer.MAX_VALUE); BUILDERS.put(typeName, builder); }

      Attachments

        Issue Links

          Activity

            People

              gszadovszky Gabor Szadovszky
              hn5092 yiming.xu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: