[GRIFFIN-67] Simplify data quality env and deployment

• Type: Task
• Status: Closed
• Priority: Minor
• Resolution: Done
• Fix Version/s: 0.1.6-incubating
• Assignee: Lionel Liu
• Reporter: William Guo
• Created: 26/Oct/17 07:45
• Updated: 18/Sep/18 08:11
• Resolved: 09/Apr/18 01:50

Description

Hi, Guys

I try to run griffin measure in local environment following GitHub guide, but I really hit some issues which I take much time to solve. I think basic cause is multiple dependencies in the whole project. Roughly counting, I see spark/yarn/Hadoop/Scala/zookeeper/Kafka involving startup, actually it’s hard to describe configuration details in one guide. In addition, current Docker image is not enough, lack of zookeeper/Kafka components, which is too difficult to find runtime problems for new users. Meanwhile, the image is so huge about 2.7G that is big burden for network downloading traffic.

So, I have proposal below,

1. should strip complex dependencies and supply a basic measure sample image to try running

2. trim user guide and make easy run experience

thanks
Jin