[HDDS-7926] [hsync] Recon throws ClassCastException

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Labels: pull-request-available
• Assignee: Arafat Khan
• Reporter: Wei-Chiu Chuang
• Created: 08/Feb/23 14:47
• Updated: 16/Jul/24 08:45
• Resolved: 01/Mar/23 12:25

Description

Recon server doesn't show the correct number of files/keys.

Found this error in Recon server:

2023-02-08 14:30:30,774 ERROR org.apache.hadoop.ozone.recon.tasks.FileSizeCountTask: Unexpected exception while processing key /vol1/bucket1/hbase/MasterData/WALs/weic
hiu-1.weichiu.root.hwx.site,22001,1675748204950/weichiu-1.weichiu.root.hwx.site%2C22001%2C1675748204950.1675865324614.
java.lang.ClassCastException: org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo cannot be cast to org.apache.hadoop.ozone.om.helpers.OmKeyInfo
        at org.apache.hadoop.ozone.recon.tasks.FileSizeCountTask.process(FileSizeCountTask.java:144)
        at org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.lambda$consumeOMEvents$0(ReconTaskControllerImpl.java:113)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2023-02-08 14:30:30,775 ERROR org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl: Unexpected error :
java.util.concurrent.ExecutionException: java.lang.ClassCastException: org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo cannot be cast to org.apache.hadoop.ozone.om.helpers.OmKeyInfo
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.processTaskResults(ReconTaskControllerImpl.java:247)
        at org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.consumeOMEvents(ReconTaskControllerImpl.java:118)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.syncDataFromOM(OzoneManagerServiceProviderImpl.java:503)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.lambda$startSyncDataFromOM$0(OzoneManagerServiceProviderImpl.java:258)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassCastException: org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo cannot be cast to org.apache.hadoop.ozone.om.helpers.OmKeyInfo
        at org.apache.hadoop.ozone.recon.tasks.NSSummaryTaskWithLegacy.processWithLegacy(NSSummaryTaskWithLegacy.java:93)
        at org.apache.hadoop.ozone.recon.tasks.NSSummaryTask.process(NSSummaryTask.java:99)
        at org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.lambda$consumeOMEvents$0(ReconTaskControllerImpl.java:113)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ... 3 more

I suspect this is because the value of the deletedTable OMMetadataManager.getDeletedTable() is RepeatedOmKeyInfo, unlike like other tables where the value is OmKeyInfo.

 

We need to do better type check here.

Issue Links

• relates to: HDDS-11187 Fix Event Handling Corruption in OMDBUpdatesHandler to Prevent ClassCastException in Recon Server
• links to: GitHub Pull Request #4266