I'm checking with a following method for judging whether user-facing api use Guava's object, though it's not elegant one:

1. Executing find . -name "*.java" | xargs grep "google.common" under libraries which seems user-facing api (hadoop-hdfs-project/hadoop-hdfs-client/, hadoop-common-project, , hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-client, hadoop-yarn-client}, hadoop-mapreduce-project/hadoop-mapreduce-client, )
2. Open code and whether it has @InterfaceAudience.Private. If so, we don't need to check the code.
3. Check the usage of guava's objects.

I checked hadoop-hdfs-project/hadoop-hdfs-client/, and there are no exposure of Guava's object.

Since HADOOP-10101 was committed, is the plan to finish this investigation, then shade Guava everywhere?

A more sure way of finding API pollution would be writing a bytecode analysis tool. We'd filter for classes and methods annotated IA.Public and LimitedPrivate, then check those method signatures for Guava class references.

Another possible quick thing you could try is using JACC or JDiff to compare current vs. shaded Guava.

Thanks Andrew for the information. Annotation File Utilities, being included in Checker Framework, can extract Annotations included in classes. Hence, we can use it to check whether classes have IA.Public and LimitedPrivate. Let me try.

https://checkerframework.org/annotation-file-utilities/#extract-annotations

I'm upgrading this to a blocker, since it's a prereq for shading. I'm also going to file a blocker JIRA for shading guava everywhere.

Hi Tsuyoshi Ozawa, do you have an update for this JIRA?

Can we resolve this given that the shaded client work seems to be working okay?

Andrew Wang Vinod Kumar Vavilapalli Can we use java8 supplier here, instead of guava to avoid this issue?

Andrew Wang Vinod Kumar Vavilapalli As far as I know and as Bharat Viswanadham mentioned, it would be ideal to substitute Guava's Supplier with java's one.

Tsuyoshi Ozawa As this umbrella jira, created a subtask for updating AMRMClient and AMRMClientAysnc.java files.

Hi Bharat Viswanadham is this JIRA tracking towards completion for beta1 (end-of-week)?

Sean Busbey mentioned the apilyzer plugin to me which Accumulo uses, it might help for finding these classes.

Andrew Wang As this is umbrella jira, created subtask for AMRMClient.
And in the comments it is mentioned that hadoop-hdfs-project/hadoop-hdfs-client/ does not have any references.

I can look over this apilyzer plugin, but i have not played with that and it may take some time for me, if any one has idea about that, they can take this up work item if it is needed work item by end of week.

Okay, thanks Bharat. I wrote a little shell script possibly useful with apilyzer to generate a regex for fully-qualified classnames that mention "InterfaceAudience.Public":

echo -n "("
for f in `ag --java -l InterfaceAudience.Public`; do
    package=$(grep package "$f" | cut -d " " -f 2 | tr -d ";")
    classname=$(basename "$f" ".java")
    echo -n $package.$classname"|"
done
echo -n ")"

I couldn't get apilyzer to work after playing with it for about an hour. A different tool might still be better.

If we've taken care of all the problems we know about, let's close this out. We can open new jiras for new problems.

Sad to hear apilyzer didn't work out. Andrew Wang could you send them a ping about the difficulty? Maybe we can get one of its maintainers to walk us through use.

Thanks Sean, I downgraded this to critical to get it off the blocker list.

Sure, I can ping the apilyzer folks.