Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17130

Configuration.getValByRegex() shouldn't update the results while fetching.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.3
    • 3.2.2, 3.3.1
    • common
    • None

    Description

      We have seen this stacktrace while using ABFS file system. After analysing the stack trace we can see that getValByRegex() is reading the properties and substituting the value in the same call. This may cause the ConcurrentModificationException. 

      Caused by: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1877) ... 18 more Caused by: java.util.ConcurrentModificationException at java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at org.apache.hadoop.conf.Configuration.getValByRegex(Configuration.java:3855) at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.validateStorageAccountKeys(AbfsConfiguration.java:689) at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.<init>(AbfsConfiguration.java:237) at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:154) at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:113) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3396) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3456) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3424) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:518) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
       
      
      
      

      Attachments

        Issue Links

          Activity

            People

              mukund-thakur Mukund Thakur
              mukund-thakur Mukund Thakur
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: