Major Description
Livy's build currently uses a different version of netty depending on the Spark version. That is dangerous because at runtime, Livy will be using the configured Spark's version of netty, which may not match the one Livy was built with.
That can cause issues if the netty API changes in an incompatible way (as it has in some areas between 4.0 and 4.1, which are the versions used by Spark 2.2 and 2.3 respectively).
Livy currently also adds its own netty jar to the Spark application, but that basically has no effect unless you use Spark's "userClassPathFirst" configs.
Shading makes sure the Livy code is using its expected version of netty regardless of the version Spark is using.