[HADOOP-8587] HarFileSystem access of harMetaCache isn't threadsafe - ASF JIRA

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 1.0.0, 2.0.0-alpha
Fix Version/s: 1.2.0, 0.23.3, 2.0.2-alpha
Component/s: fs
Labels:
None

Hadoop Flags:

Reviewed

Description

HarFileSystem's use of the static harMetaCache map is not threadsafe. Credit to Todd for pointing this out.

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

hadoop-8587.txt

11/Jul/12 04:44

1 kB

Eli Collins
hadoop-8587.txt

11/Jul/12 09:25

2 kB

Eli Collins
hadoop-8587-b1.txt

11/Jul/12 19:33

1 kB

Eli Collins

Activity

Ascending order - Click to sort in descending order

Eli Collins created issue - 11/Jul/12 04:38

Eli Collins made changes - 11/Jul/12 04:44

Field	Original Value	New Value
Attachment		hadoop-8587.txt [ 12535980 ]

Eli Collins made changes - 11/Jul/12 04:44

Assignee		Eli Collins [ eli ]
Affects Version/s		1.0.0 [ 12318244 ]
Target Version/s	2.0.1-alpha [ 12321441 ]	1.2.0, 2.0.1-alpha [ 12321659, 12321441 ]
Priority	Major [ 3 ]	Minor [ 4 ]
Component/s		fs [ 12310689 ]

Eli Collins made changes - 11/Jul/12 04:44

Status

Open [ 1 ]

Patch Available [ 10002 ]

Eli Collins made changes - 11/Jul/12 09:25

Attachment

hadoop-8587.txt [ 12535996 ]

Eli Collins made changes - 11/Jul/12 19:28

Comment

[ Integrated in Hadoop-Mapreduce-trunk #1133 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/])
Revert previous commit, accidentally included ~~HADOOP-8587~~. (Revision 1360055)

Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
]

Eli Collins made changes - 11/Jul/12 19:28

Comment

[ Integrated in Hadoop-Common-trunk-Commit #2447 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2447/])
Revert previous commit, accidentally included ~~HADOOP-8587~~. (Revision 1360055)

Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
]

Eli Collins made changes - 11/Jul/12 19:28

Comment

[ Integrated in Hadoop-Hdfs-trunk-Commit #2514 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2514/])
Revert previous commit, accidentally included ~~HADOOP-8587~~. (Revision 1360055)

Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
]

Eli Collins made changes - 11/Jul/12 19:28

Comment

[ Integrated in Hadoop-Mapreduce-trunk-Commit #2466 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2466/])
Revert previous commit, accidentally included ~~HADOOP-8587~~. (Revision 1360055)

Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
]

Eli Collins made changes - 11/Jul/12 19:28

Comment

[ Integrated in Hadoop-Hdfs-trunk #1100 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/])
Revert previous commit, accidentally included ~~HADOOP-8587~~. (Revision 1360055)

Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1360055
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/HttpAuthentication.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HarFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/AuthenticationFilterInitializer.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/resources/httpfs-default.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/UsingHttpTools.apt.vm
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/apt/index.apt.vm
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebHDFS.apt.vm
]

Eli Collins made changes - 11/Jul/12 19:28

Comment

[ -1 overall. Here are the results of testing the latest attachment
  http://issues.apache.org/jira/secure/attachment/12535980/hadoop-8587.txt
  against trunk revision .

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 javac. The patch appears to cause the build to fail.

Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1189//console

This message is automatically generated. ]

Eli Collins made changes - 11/Jul/12 19:28

Status

Patch Available [ 10002 ]

Open [ 1 ]

Eli Collins made changes - 11/Jul/12 19:33

Attachment

hadoop-8587-b1.txt [ 12536088 ]

Eli Collins made changes - 11/Jul/12 23:48

Status	Open [ 1 ]	Resolved [ 5 ]
Hadoop Flags		Reviewed [ 10343 ]
Target Version/s	1.2.0, 2.0.1-alpha [ 12321659, 12321441 ]
Fix Version/s		1.2.0 [ 12321659 ]
Fix Version/s		2.0.1-alpha [ 12321441 ]
Resolution		Fixed [ 1 ]

Robert Joseph Evans made changes - 12/Jul/12 16:59

Fix Version/s

0.23.3 [ 12320059 ]

Arun C Murthy made changes - 07/Sep/12 22:01

Fix Version/s		2.0.2-alpha [ 12322473 ]
Fix Version/s	2.1.0-alpha [ 12321441 ]
Fix Version/s	1.2.0 [ 12321659 ]

Arun C Murthy made changes - 11/Oct/12 18:45

Status

Resolved [ 5 ]

Closed [ 6 ]

Matt Foley made changes - 06/May/13 04:16

Fix Version/s

1.2.0 [ 12321659 ]

People

Assignee:

Eli Collins

Reporter:

Eli Collins

Votes:

0 Vote for this issue

Watchers:

6 Start watching this issue

Dates

Created:

11/Jul/12 04:38

Updated:

06/May/13 04:16

Resolved:

11/Jul/12 23:48

Development

Agile

View on Board