I have an engine running on spark2 using a DecisionTreeClassifier model using the CrossValidator.
I want to migrate from spark2 to spark3. I've run DecisionTreeClassifier on the same data with the same parameter values. But unfortunately my results are completely different, especially in terms of tree structure. I have trees with less depth and fewer splits on spark3. I've tried to read the documentation but I haven't found an answer to my question.
Can you help me find a solution to this problem?
Thanks in advance for your help