-
Type:
New Feature
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 2.8.0
-
Fix Version/s: 2.8.0, 3.0.0-alpha1
-
Component/s: None
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
While we wait for YARN-2942 to become viable, it would still be great to improve the aggregated logs problem. We can write a tool that combines aggregated log files into a single HAR file per application, which should solve the too many files and too many blocks problems. See the design document for details.
See YARN-2942 for more context.
- depends upon
-
YARN-3950 Add unique YARN_SHELL_ID environment variable to DistributedShell
-
- Resolved
-
- relates to
-
MAPREDUCE-7236 HadoopArchiveLogs will use token to create proxy user when kerberos on
-
- Open
-
-
MAPREDUCE-7027 HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails
-
- Resolved
-
-
MAPREDUCE-6480 archive-logs tool may miss applications
-
- Resolved
-
-
MAPREDUCE-6494 Permission issue when running archive-logs tool as different users
-
- Resolved
-
-
MAPREDUCE-6495 Docs for archive-logs tool
-
- Resolved
-
-
MAPREDUCE-6503 archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME
-
- Resolved
-
-
MAPREDUCE-6550 archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor
-
- Resolved
-
-
MAPREDUCE-7023 TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun
-
- Resolved
-
-
MAPREDUCE-7235 NPE in HadoopArchiveLogs#filterAppsByAggregatedStatus
-
- Patch Available
-
-
YARN-2942 Aggregated Log Files should be combined
-
- Resolved
-
-
MAPREDUCE-6970 archive-logs tool should throttle container requests
-
- Open
-
-
YARN-4086 Allow Aggregated Log readers to handle HAR files
-
- Resolved
-
-
YARN-4946 RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
-
- Resolved
-