[AVRO-2616] Do not use Hadoop FS for local files with avro-tools

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: java
• Assignee: Unassigned
• Reporter: Ryan Skraba
• Created: 04/Nov/19 13:44
• Updated: 17/Jan/20 21:31

Description

The avro-tools jar includes the Hadoop dependencies inside the fat jar. This is probably so that the CLI can operate with files that are located on the cluster (or other URIs that have Hadoop FileSystem implementations).

This is useful if the user is accessing HDFS with the same version as the Hadoop FileSystem, and mostly neutral if the user is accessing other URIs, including the local filesystem.

Hadoop doesn't currently officially support any version after JDK 8 (at time of writing). We might want to access the local filesystem bypassing the Hadoop jars to avoid requiring this dependency when it is not needed.

Issue Links

• is related to: AVRO-2561 java.lang.StringIndexOutOfBoundsException: begin 0, end 3, length 2