[HBASE-20553] Add dependency CVE checking to nightly tests

• Type: Umbrella
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha-1
• Component/s: dependencies
• Assignee: Sakthi
• Reporter: Sean Busbey
• Created: 09/May/18 15:52
• Updated: 22/Jan/20 00:22

Description

We should proactively work to flag dependencies with known CVEs so that we can then update them early in our development instead of near a release.

YETUS-441 is working to add a plugin for this, we should grab a copy early to make sure it works for us.

Rough outline:

1. install yetus locally
2. install the dependency-check cli (homebrew instructions on right hand margin)
3. Get a local copy of the OWASP datafile (dependency-check --updateonly --data /some/local/path/to/dir)
4. Run hbase_nightly_yetus.sh using matching environment variables from the “yetus general check” (currently line #126 in our nightly Jenkinsfile)
5. Grab the plugin definition and suppression file from from YETUS-441
6. put the plugin definition either in a directory of dev-support or into the hbase-personality.sh directly
7. Re-run hbase_nightly_yetus.sh to verify that the plugin results show up. (Probably this will involve adding new pointers for “where is the suppression file”, “where is the OWASP datafile” and pointing them somewhere locally.)

Once all of that is in place we’ll get the changes needed into a branch that we can test out. Over in YETUS-441 I’ll need to add a jenkins job that’ll handle periodically updating a copy of the datafile for the OWASP dependency checker. Presuming I have that in place by the time we have a nightly branch to check this out, then we’ll also need to update our nightly Jenkinsfile to fetch the data file from that job.

Issue Links

• is related to: YETUS-441 Add a precommit check for known CVEs from dependencies