[PIO-105] Batch Predictions

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.0-incubating
• Component/s: Core
• Target Version/s: 0.12.0-incubating
• External issue URL: https://github.com/apache/incubator-predictionio/pull/412
• Assignee: Mars Hall
• Reporter: Mars Hall
• Created: 14/Jul/17 23:59
• Updated: 05/Dec/17 22:44
• Resolved: 01/Aug/17 22:35

Description

Implement a new pio batchpredict command to enable massive, fast, batch predictions from a trained model. Read a multi-object JSON file as the input format, with one query object per line. Similarly, write results to a multi-object JSON file, with one prediction result + its original query per line.

Currently getting bulk predictions from PredictionIO is possible with either:

• a pio eval script, which will always train a fresh, unvalidated model before getting predictions
• a custom script that hits the queries.json HTTP API, which is a serious bottleneck when requesting hundreds-of-thousands or millions of predictions

Neither of these existing bulk-prediction hacks are adequate for the reasons mentioned.

It's time for this use-case to be a firstclass command

Pull request https://github.com/apache/incubator-predictionio/pull/412

Issue Links

• links to: GitHub Pull Request #412