DfsBlockWorkerTItem, TOutput ClassNaiad Help

[This is preliminary documentation and is subject to change.]

IWorker implementation for the dfs worker that reads data from blocks. This is further specialized to different file formats by passing in functions for syncing to record boundaries and deserializing data
Inheritance Hierarchy

SystemObject
  Microsoft.Research.Naiad.Frameworks.Storage.DfsDfsBlockWorkerTItem, TOutput
    Microsoft.Research.Naiad.Frameworks.Storage.DfsDfsTextWorker

Namespace: Microsoft.Research.Naiad.Frameworks.Storage.Dfs
Assembly: Microsoft.Research.Naiad.Storage (in Microsoft.Research.Naiad.Storage.dll) Version: 0.5.0.0 (0.5.0.0)
Syntax

C#
public class DfsBlockWorker<TItem, TOutput> : IWorker<TItem, IPAddress[], TOutput>
where TItem : DfsBlock

Type Parameters

TItem
The work item passed by the matching coordinator, which inherits from DfsBlock but may contain metadata used in syncing to record boundaries or deserializing
TOutput
The type of deserialized records produced by the worker

The DfsBlockWorkerTItem, TOutput type exposes the following members.

Constructors

  NameDescription
Public methodDfsBlockWorkerTItem, TOutput
create a worker to read dfs files broken into blocks
Top
Methods

  NameDescription
Public methodDescribeWorker
the IWorker implementation used to identify this worker to the WebHdfs coordinator, so that it can be sent work items of blocks stored at the same machine
Public methodDoWork
find the range of valid records in the block (those that start within the block), deserialize them, and return them in batches
Top
Fields

  NameDescription
Protected fieldclient
the client used for reading Hdfs data
Top
See Also

Reference