[YARN-10884] EntityGroupFSTimelineStore fails to parse log files which has empty owner

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.3.1
• Component/s: timelineserver
• Assignee: SwathiChandrashekar
• Reporter: Prabhu Joseph
• Created: 15/Aug/21 11:53
• Updated: 07/Sep/21 16:13
• Resolved: 07/Sep/21 16:13

Description

Due to HADOOP-17848 - Wasb FileSystem sets owner as empty during append operation. 

ATS1.5 fails to read such files with below error 

 java.lang.IllegalArgumentException: Null user
        at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1271)
        at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1258)
        at org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:141)
        at org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:114)
        at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:701)
        at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:675)
        at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:888)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

It gets ownership of the file to check ACL. In case of disabled ACL check, this is not required. Will suggest to add anonymous user in case of empty user.

    if (owner.isEmpty()) {
      user = "anonymous";
    } else {
      user = owner;
    }

Issue Links

• links to: GitHub Pull Request #3318