[HDDS-7327] Recon to note down replica states

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Recon
• Assignee: Zita Dombi
• Reporter: Siyao Meng
• Created: 13/Oct/22 20:56
• Updated: 18/Nov/22 02:34

Description

Related previous discussion: HDDS-7098

Right now it seems that Recon only takes note of the overall container health state in the Recon SQL DB:

ij version 10.14
ij> connect 'jdbc:derby:ozone_recon_derby.db';
ij> show tables;
TABLE_SCHEM         |TABLE_NAME                    |REMARKS
------------------------------------------------------------------------
...
SYSIBM              |SYSDUMMY1                     |
RECON               |CLUSTER_GROWTH_DAILY          |
RECON               |FILE_COUNT_BY_SIZE            |
RECON               |GLOBAL_STATS                  |
RECON               |RECON_TASK_STATUS             |
RECON               |UNHEALTHY_CONTAINERS          |

28 rows selected
ij> select * from RECON.UNHEALTHY_CONTAINERS;
container_id        |container_state |in_state_since      |expected_r&|actual_rep&|replica_de&|reason
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1                   |UNDER_REPLICATED|1665692819704       |3          |2          |1          |NULL

but Recon does not record the health state of individual replicas in the container. This will be useful for users to check replica states in Recon.

We might want to persist the info to Recon SQL DB only when datanodes report that a replica is unhealthy. Do not persist healthy ones to avoid too many writes (can lead to performance issues)

Issue Links

• is a child of: HDDS-7098 Provide a way for admin to identify all unhealthy container replicas