DfsBlockWorkerTItem, TOutput Constructor Naiad Help

[This is preliminary documentation and is subject to change.]

create a worker to read dfs files broken into blocks

Namespace: Microsoft.Research.Naiad.Frameworks.Storage.Dfs
Assembly: Microsoft.Research.Naiad.Storage (in Microsoft.Research.Naiad.Storage.dll) Version: 0.5.0.0 (0.5.0.0)
Syntax

C#
public DfsBlockWorker(
	int syncRequestLength,
	Action<TItem, Stream> syncToNextRecord,
	Func<TItem, Stream, IEnumerable<ArraySegment<TOutput>>> deserialize,
	HdfsClientBase client
)

Parameters

syncRequestLength
Type: SystemInt32
size of each dfs request when seeking past the end of the block for the start of the next record. If records are expected to be small this should also be small, to avoid prefetching and buffering a lot of the next block's data
syncToNextRecord
Type: SystemActionTItem, Stream
action to sync to the start of the next record. The first argument is the block item being read, which may contain metadata about sync markers. The second argument is the stream to scan.
deserialize
Type: SystemFuncTItem, Stream, IEnumerableArraySegmentTOutput
function to deserialize records in a stream
client
Type: HdfsClientBase
client used to read hdfs data
See Also

Reference