case when (key = '0') then 123.456BD else 0.0BD end
from src limit 2
FAILED: SemanticException [Error 10016]: Line 3:44 Argument type mismatch '0.0BD': The expression after ELSE should have the same type as those after THEN: "decimal(6,3)" is expected but "decimal(1,0)" is found
The return type checking is too strict and won't allow different decimal types to be returned if they are not the exact same type (precision/scale). There are similar issues with char/varchar length, but even in the general case it seems odd that you wouldn't be able to specify 1 and 0.0 in the same case statement.
I would propose setting returnOIResolver so that it is able to convert the return values to common type.