Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17130

Configuration.getValByRegex() shouldn't update the results while fetching.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.3
    • Fix Version/s: 3.2.2, 3.3.1
    • Component/s: common
    • Labels:
      None
    • Target Version/s:

      Description

      We have seen this stacktrace while using ABFS file system. After analysing the stack trace we can see that getValByRegex() is reading the properties and substituting the value in the same call. This may cause the ConcurrentModificationException. 

      Caused by: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1877) ... 18 more Caused by: java.util.ConcurrentModificationException at java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at org.apache.hadoop.conf.Configuration.getValByRegex(Configuration.java:3855) at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.validateStorageAccountKeys(AbfsConfiguration.java:689) at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.<init>(AbfsConfiguration.java:237) at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:154) at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:113) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3396) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:158) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3456) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3424) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:518) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
       
      
      
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mukund-thakur Mukund Thakur
                Reporter:
                mukund-thakur Mukund Thakur
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: