[This is preliminary documentation and is subject to change.]
Namespaces
Namespace | Description |
---|---|
Microsoft.Research.Naiad |
The Microsoft.Research.Naiad namespace provides the basic classes that are used in all Naiad programs.
|
Microsoft.Research.Naiad.Dataflow |
The Dataflow namespace provides classes that enable the programmer to construct dataflow graphs manually.
|
Microsoft.Research.Naiad.Dataflow.Iteration |
The Dataflow.Iteration namespace provides classes that enable adding cycles to a dataflow graph.
|
Microsoft.Research.Naiad.Dataflow.PartitionBy |
The Dataflow.PartitionBy namespace provides extension methods that enable manual repartitioning of data between vertices.
|
Microsoft.Research.Naiad.Dataflow.StandardVertices |
The Dataflow.StandardVertices namespace provides base implementations of commonly-used dataflow vertex types, such as UnaryVertexTInput, TOutput, TTime and BinaryVertexTInput1, TInput2, TOutput, TTime.
|
Microsoft.Research.Naiad.Diagnostics |
The Diagnostics namespace provides classes that support Logging, tracing, and observing various events in the Naiad runtime.
|
Microsoft.Research.Naiad.Frameworks.Azure |
The Azure framework contains extension methods for Computation and StreamTRecord, TTime
that facilitate the use of Microsoft Azure Storage.
|
Microsoft.Research.Naiad.Frameworks.DifferentialDataflow |
The Differential Dataflow framework contains extension methods that support LINQ-style incremental and iterative operators.
|
Microsoft.Research.Naiad.Frameworks.GraphLINQ |
The GraphLINQ framework contains data types and extension methods that add optimized graph-specific
operators for streams.
|
Microsoft.Research.Naiad.Frameworks.Hdfs |
The Hdfs framework supports reading and writing Hdfs files using the Java-based native Hdfs protocol. It requires Java and
the Hdfs jar files to be installed locally, and will throw an exception if they are not installed. The environment JAVA_HOME
must be set to the location of the Java installation, and HADOOP_COMMON_HOME must be set to the location of the Hadoop jar files
including those for Hdfs
|
Microsoft.Research.Naiad.Frameworks.Lindi |
The Lindi framework contains extension methods that add LINQ-style operators for streams.
|
Microsoft.Research.Naiad.Frameworks.Storage |
The Storage framework includes base classes to support reading and writing a variety of files including Azure and Hdfs
|
Microsoft.Research.Naiad.Frameworks.Storage.Dfs |
The Dfs framework includes base classes to support reading and writing Hdfs files, and is intended to be extensible to other
block-based file systems
|
Microsoft.Research.Naiad.Frameworks.WebHdfs |
The WebHdfs framework supports reading and writing Hdfs files using the REST-based WebHdfs protocol.
|
Microsoft.Research.Naiad.Frameworks.WorkGenerator |
The WorkGenerator framework contains base classes for implementing a work queue of items to be distributed dynamically to a set
of Naiad workers. The queue is implemented as a Naiad loop; when new work arrives at a coordinator, it sends "wakeup" messages
to all the workers, which respond with "ready" messages. Each time the coordinator receives a ready message from a worker, it matches
an outstanding work item to that worker and sends it back. When the worker completes its work item, it responds with another ready message.
Work items are thus sent out as workers become ready, rather than requiring coordination across workers. The framework allows
workers to send identifying information that the coordinator can use for matching. For example the Hdfs and WebHdfs frameworks
instantiate work generators in which the workers identify themselves by the IP addresses of the computer they are running on,
allowing file blocks to be preferentially read on the worker computer that is hosting them.
|
Microsoft.Research.Naiad.Input |
The Input namespace provides classes and extension methods for ingesting data into a Naiad computation.
|
Microsoft.Research.Naiad.Runtime.Progress |
The Runtime.Progress namespace provides classes that support tracking the progress of a Naiad computation.
|
Microsoft.Research.Naiad.Serialization |
The Serialization namespace contains classes that support serialization of Naiad data for network and file I/O.
|
Microsoft.Research.Naiad.Utilities |
The Utilities namespace contains miscellaneous classes that support Naiad framework authors.
|