[BEAM-3900] Introduce Euphoria Java 8 DSL

• Type: New Feature
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.9.0
• Component/s: dsl-euphoria
• Assignee: David Morávek
• Reporter: David Morávek
• Created: 21/Mar/18 15:22
• Updated: 16/May/20 14:04
• Resolved: 14/Dec/18 09:58

Description

This is the umbrella issue for integrating Euphoria API into Beam.

Issue Links

• is a parent of: BEAM-5393 Euphoria: Full windowing support
• links to: GitHub Pull Request #5447
• links to: GitHub Pull Request #5735
• links to: GitHub Pull Request #5783
• links to: GitHub Pull Request #6601
• links to: GitHub Pull Request #6709

Sub-Tasks

• Euphoria: Initial import from github: Resolved
• Euphoria: Integration with gradle build system: Resolved
• Join operator translator: Resolved
• Migration of ReduceByKey test suite: Resolved
• Flipping tests - some tests in Euphoria operator test suit are failing randomly: Resolved
• Add Euphoria Java 8 DSL to "Conbribute > Technical References > Ongoing Projects": Resolved
• Update Operator Testkit: Resolved
• General Cleanup: Resolved
• Add type hint information: Resolved
• Improve windowing API: Resolved
• Translation Euphoria Accumulators to Beam Metrics: Resolved
• Enable tests in `OperatorsTestSuite` where possible: Resolved
• Remove 'Executor' concept: Resolved
• Add posibility to define composite PTransform throught Euphoria API: Resolved
• Extend euphoria API to allow registaration of Beam Coders: Resolved
• Flipping tests: Resolved
• Better Kryo integration: Resolved
• Allow all operators to carry type descriptors of its element: Resolved
• Use type informations about output, key and values from operators to set coders: Resolved
• Use type informations about output, key and values from operators to set coders: Resolved
• Replace Euphoria's Pair with Beam's KV: Resolved
• Testsuite do not run all tests it should: Resolved
• Write Euphoria in Beam documentation: Resolved
• Fix failing `ReduceWindow` test: Resolved
• Enable FindBugs: Resolved
• Testsuite `BroadcastHashJoin` test are not enabled: Resolved
• Use chunked Kryo encoding instead of our own chunking: Resolved
• Add support for `TopPerKey` operator: Resolved
• Use chunked Kryo encoding instead of our own chunking: Resolved
• Clean TODOs and unused classes.: Resolved
• Missing windowing error message is not undestandable to Euphoria user: Resolved
• Euphoria to Beam translators should set coders to output `PCollections': Resolved
• Add support for accumulatrors to `SingleValueCollector`: Resolved
• `ListDataSource` should not enforce data items to be serializable: Resolved
• Support ReduceByKey - sorted values: Open
• Joins should support more than one output element from user joining UDF: Resolved
• SingleJvmAccumulatorProvider fix: Resolved
• Add option to run testkit tests using Spark runner on local cluster: Resolved
• Kryo NO-OP registrars can swap IDs between JVMs: Resolved
• Separate Kryo to module of its own: Resolved
• Refactor `EuphoriaCoderProvider` to `KryoCoderProvider`: Resolved
• Euphoria: Remove Dataset abstraction: Resolved
• Operator translation based on operator name: Resolved
• Refactor traslator providers: Resolved
• Remove unnecesarry Kryo dependecny from euphoria: Resolved