Description
New implementation for generation of code for SpecificRecord that improves decoding by over 10% and encoding over 30% (more improvements are on the way). This feature is behind a feature flag (org.apache.avro.specific.use_custom_coders) and (for now) turned off by default. See Getting Started (Java) for instructions.
(A bit more info: Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is slower for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords.)
Attachments
Attachments
Issue Links
- blocks
-
AVRO-2093 Extend "custom coders" to fully support union types
- Open
-
AVRO-2094 Extend "custom coders" to support logical types
- Open
-
AVRO-2091 Eliminate org.apache.avro.specific.use_custom_coder feature flag
- Open
-
AVRO-2092 Flip default value of org.apache.avro.specific.use_custom_coder to true
- Open
- is blocked by
-
AVRO-2235 Regenerate TestRecordWithLogicalTypes
- Resolved
- links to
Activity
Workflow | patch-available, re-open possible [ 13911495 ] | no-reopen-closed, patch-avail [ 13915311 ] |
Workflow | no-reopen-closed, patch-avail [ 13780678 ] | patch-available, re-open possible [ 13911495 ] |
Workflow | patch-available, re-open possible [ 13751968 ] | no-reopen-closed, patch-avail [ 13780678 ] |
Workflow | no-reopen-closed, patch-avail [ 13366991 ] | patch-available, re-open possible [ 13751968 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Fix Version/s | 1.9.0 [ 12333394 ] | |
Resolution | Fixed [ 1 ] | |
Status | Open [ 1 ] | Resolved [ 5 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Description |
New implementation for generation of code for SpecificRecord that improves decoding by over 10% and encoding over 30% (more improvements are on the way). This feature is behind a feature flag ({{org.apache.avro.specific.use_custom_coders}}) and (for now) turned off by default. See [Getting Started (Java)|https://avro.apache.org/docs/current/gettingstartedjava.html#Beta+feature:+Generating faster+code] for instructions.
(A bit more info: Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords.) |
New implementation for generation of code for SpecificRecord that improves decoding by over 10% and encoding over 30% (more improvements are on the way). This feature is behind a feature flag ({{org.apache.avro.specific.use_custom_coders}}) and (for now) turned off by default. See [Getting Started (Java)|https://avro.apache.org/docs/current/gettingstartedjava.html#Beta+feature:+Generating+faster+code] for instructions.
(A bit more info: Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords.) |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Description |
New implementation for generation of code for SpecificRecord that improves decoding by over 10% and encoding over 30% (more improvements are on the way). This feature is behind a feature flag ({{org.apache.avro.specific.use_custom_coders}}) and (for now) turned off by default. See [Getting Started (Java)|https://avro.apache.org/docs/current/gettingstartedjava.html#Beta+feature:+Faster+code+generation] for instructions.
(A bit more info: Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords.) |
New implementation for generation of code for SpecificRecord that improves decoding by over 10% and encoding over 30% (more improvements are on the way). This feature is behind a feature flag ({{org.apache.avro.specific.use_custom_coders}}) and (for now) turned off by default. See [Getting Started (Java)|https://avro.apache.org/docs/current/gettingstartedjava.html#Beta+feature:+Generating faster+code] for instructions.
(A bit more info: Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords.) |
Description |
Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords. We've run a public benchmark showing that the new code reduces serialization time by 2/3 and deserialization time by close to 50%. |
New implementation for generation of code for SpecificRecord that improves decoding by over 10% and encoding over 30% (more improvements are on the way). This feature is behind a feature flag ({{org.apache.avro.specific.use_custom_coders}}) and (for now) turned off by default. See [Getting Started (Java)|https://avro.apache.org/docs/current/gettingstartedjava.html#Beta+feature:+Faster+code+generation] for instructions.
(A bit more info: Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords.) |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Attachment | perf-data.txt [ 12944486 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #350 (Web Link)" [ 133428 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 110168 ] |
Assignee | Raymie Stata [ raymie ] |
Remote Link | This issue links to "GitHub Pull Request #256 (Web Link)" [ 93135 ] |
Attachment | customcoders.md [ 12890882 ] |