[HDDS-6588] Recon shows zero datanodes with `./compose/upgrade/compose/ha/`

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: Ozone Recon
• Labels: pull-request-available
• Assignee: Zita Dombi
• Reporter: Siyao Meng
• Created: 15/Apr/22 01:20
• Updated: 14/Jul/22 16:32
• Resolved: 14/Jul/22 15:03

Description

Need to investigate. Possibly `./compose/upgrade/compose/ha/docker-compose.yaml` has incorrect (SCM, or Recon, or DN) configs that prevents Recon from getting Datanodes information?

Repro (just boot the compose cluster up, without doing anything else):

mvn clean install -Pdist -DskipTests -e -Dmaven.javadoc.skip=true
cd hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/compose/upgrade/compose/ha

docker-compose up -d --scale datanode=3

# Wait for a minute for the Docker Compose cluster to boot up

Then navigate to http://localhost:9888/, you will be greeted with 0/0 Datanodes:

Clicking into the Datanodes page gives nothing:

But in the Pipelines page it clearly shows live datanodes with OPEN Ratis 1 and 3 pipelines:

SCM Web UI shows 3 healthy, in-service datanodes correctly:

This bug was also observed in this comment (unrelated issue), which uses the same docker-compose.yaml.

Note that ./compose/ozone wasn't bothered with the same issue (Recon displays datanodes just fine).

Issue Links

• links to: GitHub Pull Request #3597