[DROIDS-110] droids-norobots shouldn't have dependency on protocol implementation; it should be abstract Rules Engine

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.2.0
• Component/s: core, norobots
• Assignee: Unassigned
• Reporter: Fuad Efendi
• Created: 30/Nov/10 23:08
• Updated: 03/Dec/11 01:16

Description

Naturally enforced by DROIDS-109 requirements.

1. Move NoRobotsClient.java from droids-norobots into droids-core
2. Move ContentLoader.java from droids-norobots into droids-core
3. Refactor ContentLoader, ContentEntity, ManagedContentEntity, AdvancedManagedContentEntity
Having InputStream instead of byte[] doesn't seem right; and we need proper metadata.

Even for FileProtocol.FileContentEntity, why should we expect unlimited terabytes of data and use InputStream instead of bytearray and proper encoding in case of text? Most "robots" exist because of "search", and most simply limit data to 64kb - 128kb (although Amazon.com have 300kb raw web pages in average)