Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
DeleteBlockCommands can be sent out of order, so even DeleteBlockCommand's tx is smaller than container's deleteTransactionId, we cannot tell it's duplicate or not. But if the DeleteBlockCommand's tx is equal to container's current deleteTransactionId, then it's duplicate no doubt. Tx ID is unique ID. There will be no Tx with same ID but different pending deleting blocks. In user's production cluster, it's full of following messages in DN log files.
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2370 is either received out of order or retried, 278569799 <= 278569799
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2373 is either received out of order or retried, 278569801 <= 278569801
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2366 is either received out of order or retried, 278569797 <= 278569797
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2371 is either received out of order or retried, 278569800 <= 278569800
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2381 is either received out of order or retried, 278569803 <= 278569803
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2388 is either received out of order or retried, 278569805 <= 278569805
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2403 is either received out of order or retried, 278569807 <= 278569807
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2409 is either received out of order or retried, 278569809 <= 278569809
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2413 is either received out of order or retried, 278569810 <= 278569810
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2415 is either received out of order or retried, 278569811 <= 278569811
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2417 is either received out of order or retried, 278569813 <= 278569813
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2416 is either received out of order or retried, 278569812 <= 278569812
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2445 is either received out of order or retried, 278569819 <= 278569819
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2487 is either received out of order or retried, 278569823 <= 278569823
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2490 is either received out of order or retried, 278569824 <= 278569824
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2495 is either received out of order or retried, 278569826 <= 278569826
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2506 is either received out of order or retried, 278569833 <= 278569833
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2501 is either received out of order or retried, 278569828 <= 278569828
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2504 is either received out of order or retried, 278569831 <= 278569831
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2538 is either received out of order or retried, 278569839 <= 278569839
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2540 is either received out of order or retried, 278569840 <= 278569840
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2551 is either received out of order or retried, 278569842 <= 278569842
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2554 is either received out of order or retried, 278569844 <= 278569844
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2561 is either received out of order or retried, 278569845 <= 278569845
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2583 is either received out of order or retried, 278569852 <= 278569852
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2568 is either received out of order or retried, 278569849 <= 278569849
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2567 is either received out of order or retried, 278569848 <= 278569848
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2589 is either received out of order or retried, 278569854 <= 278569854
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2563 is either received out of order or retried, 278569846 <= 278569846
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2594 is either received out of order or retried, 278569859 <= 278569859
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2599 is either received out of order or retried, 278569864 <= 278569864
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2606 is either received out of order or retried, 278569871 <= 278569871
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2607 is either received out of order or retried, 278569872 <= 278569872
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2616 is either received out of order or retried, 278569881 <= 278569881
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2619 is either received out of order or retried, 278569884 <= 278569884
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2622 is either received out of order or retried, 278569887 <= 278569887
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2621 is either received out of order or retried, 278569886 <= 278569886
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2635 is either received out of order or retried, 278569900 <= 278569900
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2625 is either received out of order or retried, 278569890 <= 278569890
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2634 is either received out of order or retried, 278569899 <= 278569899
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2636 is either received out of order or retried, 278569901 <= 278569901
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2638 is either received out of order or retried, 278569903 <= 278569903
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2640 is either received out of order or retried, 278569905 <= 278569905
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2648 is either received out of order or retried, 278569913 <= 278569913
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2656 is either received out of order or retried, 278569921 <= 278569921
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2657 is either received out of order or retried, 278569922 <= 278569922
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2661 is either received out of order or retried, 278569926 <= 278569926
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2672 is either received out of order or retried, 278569937 <= 278569937
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2675 is either received out of order or retried, 278569940 <= 278569940
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2676 is either received out of order or retried, 278569941 <= 278569941
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2680 is either received out of order or retried, 278569945 <= 278569945
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2683 is either received out of order or retried, 278569948 <= 278569948
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2688 is either received out of order or retried, 278569953 <= 278569953
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2689 is either received out of order or retried, 278569954 <= 278569954
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2690 is either received out of order or retried, 278569955 <= 278569955
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2697 is either received out of order or retried, 278569962 <= 278569962
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2707 is either received out of order or retried, 278569972 <= 278569972
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2698 is either received out of order or retried, 278569963 <= 278569963
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2708 is either received out of order or retried, 278569973 <= 278569973
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2709 is either received out of order or retried, 278569974 <= 278569974
2024-04-16 12:14:34,583 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2704 is either received out of order or retried, 278569969 <= 278569969
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2711 is either received out of order or retried, 278569976 <= 278569976
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2715 is either received out of order or retried, 278569980 <= 278569980
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2717 is either received out of order or retried, 278569982 <= 278569982
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2720 is either received out of order or retried, 278569985 <= 278569985
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2723 is either received out of order or retried, 278569988 <= 278569988
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2725 is either received out of order or retried, 278569990 <= 278569990
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2736 is either received out of order or retried, 278570001 <= 278570001
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2740 is either received out of order or retried, 278570005 <= 278570005
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2750 is either received out of order or retried, 278570015 <= 278570015
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2753 is either received out of order or retried, 278570018 <= 278570018
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2743 is either received out of order or retried, 278570008 <= 278570008
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2754 is either received out of order or retried, 278570019 <= 278570019
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2764 is either received out of order or retried, 278570020 <= 278570020
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2749 is either received out of order or retried, 278570014 <= 278570014
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 2770 is either received out of order or retried, 278570021 <= 278570021
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 3243 is either received out of order or retried, 278570025 <= 278570025
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 3217 is either received out of order or retried, 278570024 <= 278570024
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 3299 is either received out of order or retried, 278570027 <= 278570027
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 3326 is either received out of order or retried, 278570029 <= 278570029
2024-04-16 12:14:34,584 INFO org.apache.hadoop.ozone.container.common.statemachine.commandhandler.DeleteBlocksCommandHandler: Delete blocks for containerId: 3194 is either received out of order or retried, 278570022 <= 278570022
Attachments
Issue Links
- links to