[HDDS-2250] Generated configs missing from ozone-filesystem-lib jars

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.5.0
• Component/s: build, Ozone Filesystem
• Labels: pull-request-available
• Target Version/s: 0.5.0
• Assignee: Attila Doroszlai
• Reporter: Attila Doroszlai
• Created: 04/Oct/19 12:22
• Updated: 26/Mar/21 10:58
• Resolved: 04/Oct/19 21:31

Description

Hadoop 3.1 and 3.2 acceptance tests started failing with HDDS-1720, which added a new, annotated configuration class.

The change itself looks fine. The problem is that the packaging process for ozone-filesystem-lib jars keeps only 1 or 2 ozone-default-generated.xml files. With the new config in place, client configs are missing, so Ratis client gets evicted immediately due to scm.container.client.idle.threshold = 0. This results in NPE:

https://elek.github.io/ozone-ci-q4/pr/pr-hdds-1720-trunk-rd9ht/acceptance/summary.html#s1-s5-t1-k2-k2

Running command 'hdfs dfs -put /opt/hadoop/NOTICE.txt o3fs://bucket1.vol1/ozone-14607
...
-put: Fatal internal error
java.lang.NullPointerException: client is null
	at java.util.Objects.requireNonNull(Objects.java:228)
	at org.apache.hadoop.hdds.scm.XceiverClientRatis.getClient(XceiverClientRatis.java:208)
	at org.apache.hadoop.hdds.scm.XceiverClientRatis.sendRequestAsync(XceiverClientRatis.java:234)
	at org.apache.hadoop.hdds.scm.XceiverClientRatis.sendCommandAsync(XceiverClientRatis.java:332)
	at org.apache.hadoop.hdds.scm.storage.ContainerProtocolCalls.writeChunkAsync(ContainerProtocolCalls.java:310)
...

Issue Links

• is related to: HDDS-5035 Config not loaded when using an Ozone assembled jar
• links to: GitHub Pull Request #1597