Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11506

Configuration variable expansion regex expensive for long values

          Details

          • Target Version/s:
          • Hadoop Flags:
            Reviewed

            Description

            Profiling several large Hadoop jobs, we discovered that a surprising amount of time was spent inside Configuration.get, more specifically, in regex matching caused by the substituteVars call.

              Attachments

              1. HADOOP-11506.001.patch
                4 kB
                Gera Shegalov
              2. HADOOP-11506.002.patch
                6 kB
                Gera Shegalov
              3. HADOOP-11506.003.patch
                9 kB
                Gera Shegalov
              4. HADOOP-11506.004.patch
                9 kB
                Gera Shegalov

                Issue Links

                breaks

                Bug - A problem which impairs or prevents the functions of the product. HADOOP-13699 Configuration does not substitute multiple references to the same var

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved
                relates to

                Improvement - An improvement or enhancement to an existing feature or task. HADOOP-463 variable expansion in Configuration

                • Major - Major loss of function.
                • Closed

                  Activity

                    People

                    • Assignee:
                      jira.shegalov Gera Shegalov
                      Reporter:
                      dvryaboy Dmitriy V. Ryaboy
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      15 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: