Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
currently MessageStores & EdgeStores expect ByteArrayVertexIdData objects. but this is too restrictive,
refactor giraph code to support multiple VertexId structs (for instance ByteBuf, OneMessageToMultipleIds, etc.)
Attachments
Attachments
- GIRAPH-907.patch
- 176 kB
- Pavan Kumar
Activity
Hudson
added a comment -
ABORTED: Integrated in Giraph-trunk-Commit #1444 (See https://builds.apache.org/job/Giraph-trunk-Commit/1444/)
GIRAPH-907: refactor giraph code to support multiple implementations of vertexId data (pavanka) (pavanka: http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=535a333b7776f7a229b76c7656e928d8a21d21bc)
- giraph-core/src/main/java/org/apache/giraph/comm/requests/WritableRequest.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteUtils.java
- CHANGELOG
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdMessageBytesIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/UnsafeArrayReads.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdIterator.java
- giraph-core/src/main/java/org/apache/giraph/comm/SendMessageCache.java
- giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/ByteArrayMessagesPerVertexStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdDataIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/ExtendedDataOutput.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/OneMessagePerVertexStore.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/IntByteArrayMessageStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdEdgeIterator.java
- giraph-core/src/main/java/org/apache/giraph/edge/primitives/IntEdgeStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructIterable.java
- giraph-core/src/main/java/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayIterator.java
- giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java
- giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerEdgesRequest.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdData.java
- giraph-core/src/main/java/org/apache/giraph/edge/SimpleEdgeStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayIterable.java
- giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/out_of_core/DiskBackedMessageStore.java
- giraph-core/src/main/java/org/apache/giraph/comm/SendVertexIdDataCache.java
- giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayDataOutput.java
- giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java
- giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayDataInput.java
- giraph-core/src/main/java/org/apache/giraph/utils/VerboseByteArrayMessageWrite.java
- giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/ResponseEncoder.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/LongByteArrayMessageStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdEdges.java
- giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteArrayIterable.java
- giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteStructIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessageIterator.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/IntFloatMessageStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/AbstractVertexIdData.java
- giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
- giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
- giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteArrayIterator.java
- giraph-core/src/main/java/org/apache/giraph/comm/SendMessageToAllCache.java
- giraph-core/src/main/java/org/apache/giraph/utils/UnsafeByteArrayInputStream.java
- giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteStructIterable.java
- giraph-core/src/test/java/org/apache/giraph/utils/MockUtils.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdEdgeIterator.java
- giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerDataRequest.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/MessagesIterable.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/LongDoubleMessageStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdDataIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdEdges.java
- giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessages.java
- giraph-core/src/main/java/org/apache/giraph/edge/primitives/LongEdgeStore.java
- giraph-core/src/main/java/org/apache/giraph/comm/messages/MessageStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/VerboseByteStructMessageWrite.java
- giraph-core/src/main/java/org/apache/giraph/utils/RequestUtils.java
- giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/UnsafeReads.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdData.java
- giraph-core/src/main/java/org/apache/giraph/comm/netty/InboundByteCounter.java
- giraph-core/src/main/java/org/apache/giraph/edge/AbstractEdgeStore.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdMessageIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessageBytesIterator.java
- giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerMessagesRequest.java
- giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java
- giraph-core/src/main/java/org/apache/giraph/comm/SendEdgeCache.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteStructIterator.java
- giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdMessages.java
Hudson
added a comment - ABORTED: Integrated in Giraph-trunk-Commit #1444 (See https://builds.apache.org/job/Giraph-trunk-Commit/1444/ )
GIRAPH-907 : refactor giraph code to support multiple implementations of vertexId data (pavanka) (pavanka: http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=535a333b7776f7a229b76c7656e928d8a21d21bc )
giraph-core/src/main/java/org/apache/giraph/comm/requests/WritableRequest.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteUtils.java
CHANGELOG
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdMessageBytesIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/UnsafeArrayReads.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdIterator.java
giraph-core/src/main/java/org/apache/giraph/comm/SendMessageCache.java
giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/ByteArrayMessagesPerVertexStore.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdDataIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/ExtendedDataOutput.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/OneMessagePerVertexStore.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/IntByteArrayMessageStore.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdEdgeIterator.java
giraph-core/src/main/java/org/apache/giraph/edge/primitives/IntEdgeStore.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructIterable.java
giraph-core/src/main/java/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayIterator.java
giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestServerHandler.java
giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerEdgesRequest.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdData.java
giraph-core/src/main/java/org/apache/giraph/edge/SimpleEdgeStore.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayIterable.java
giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/out_of_core/DiskBackedMessageStore.java
giraph-core/src/main/java/org/apache/giraph/comm/SendVertexIdDataCache.java
giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayDataOutput.java
giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestDecoder.java
giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java
giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayDataInput.java
giraph-core/src/main/java/org/apache/giraph/utils/VerboseByteArrayMessageWrite.java
giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/ResponseEncoder.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/LongByteArrayMessageStore.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdEdges.java
giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteArrayIterable.java
giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteStructIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessageIterator.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/IntFloatMessageStore.java
giraph-core/src/main/java/org/apache/giraph/utils/AbstractVertexIdData.java
giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteArrayIterator.java
giraph-core/src/main/java/org/apache/giraph/comm/SendMessageToAllCache.java
giraph-core/src/main/java/org/apache/giraph/utils/UnsafeByteArrayInputStream.java
giraph-core/src/main/java/org/apache/giraph/utils/RepresentativeByteStructIterable.java
giraph-core/src/test/java/org/apache/giraph/utils/MockUtils.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdEdgeIterator.java
giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerDataRequest.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/MessagesIterable.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/LongDoubleMessageStore.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdDataIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdEdges.java
giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessages.java
giraph-core/src/main/java/org/apache/giraph/edge/primitives/LongEdgeStore.java
giraph-core/src/main/java/org/apache/giraph/comm/messages/MessageStore.java
giraph-core/src/main/java/org/apache/giraph/utils/VerboseByteStructMessageWrite.java
giraph-core/src/main/java/org/apache/giraph/utils/RequestUtils.java
giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/UnsafeReads.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdData.java
giraph-core/src/main/java/org/apache/giraph/comm/netty/InboundByteCounter.java
giraph-core/src/main/java/org/apache/giraph/edge/AbstractEdgeStore.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdMessageIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessageBytesIterator.java
giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerMessagesRequest.java
giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/RequestEncoder.java
giraph-core/src/main/java/org/apache/giraph/comm/SendEdgeCache.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteStructIterator.java
giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayVertexIdMessages.java
https://reviews.apache.org/r/22157/
also posted on review board