Naiad SDKNaiad Help

[This is preliminary documentation and is subject to change.]

Namespaces

NamespaceDescription
Microsoft.Research.Naiad
The Microsoft.Research.Naiad namespace provides the basic classes that are used in all Naiad programs.
Microsoft.Research.Naiad.Dataflow
The Dataflow namespace provides classes that enable the programmer to construct dataflow graphs manually.
Microsoft.Research.Naiad.Dataflow.Iteration
The Dataflow.Iteration namespace provides classes that enable adding cycles to a dataflow graph.
Microsoft.Research.Naiad.Dataflow.PartitionBy
The Dataflow.PartitionBy namespace provides extension methods that enable manual repartitioning of data between vertices.
Microsoft.Research.Naiad.Dataflow.StandardVertices
Microsoft.Research.Naiad.Diagnostics
The Diagnostics namespace provides classes that support Logging, tracing, and observing various events in the Naiad runtime.
Microsoft.Research.Naiad.Frameworks.Azure
Microsoft.Research.Naiad.Frameworks.DifferentialDataflow
The Differential Dataflow framework contains extension methods that support LINQ-style incremental and iterative operators.
Microsoft.Research.Naiad.Frameworks.GraphLINQ
The GraphLINQ framework contains data types and extension methods that add optimized graph-specific operators for streams.
Microsoft.Research.Naiad.Frameworks.Hdfs
The Hdfs framework supports reading and writing Hdfs files using the Java-based native Hdfs protocol. It requires Java and the Hdfs jar files to be installed locally, and will throw an exception if they are not installed. The environment JAVA_HOME must be set to the location of the Java installation, and HADOOP_COMMON_HOME must be set to the location of the Hadoop jar files including those for Hdfs
Microsoft.Research.Naiad.Frameworks.Lindi
The Lindi framework contains extension methods that add LINQ-style operators for streams.
Microsoft.Research.Naiad.Frameworks.Storage
The Storage framework includes base classes to support reading and writing a variety of files including Azure and Hdfs
Microsoft.Research.Naiad.Frameworks.Storage.Dfs
The Dfs framework includes base classes to support reading and writing Hdfs files, and is intended to be extensible to other block-based file systems
Microsoft.Research.Naiad.Frameworks.WebHdfs
The WebHdfs framework supports reading and writing Hdfs files using the REST-based WebHdfs protocol.
Microsoft.Research.Naiad.Frameworks.WorkGenerator
The WorkGenerator framework contains base classes for implementing a work queue of items to be distributed dynamically to a set of Naiad workers. The queue is implemented as a Naiad loop; when new work arrives at a coordinator, it sends "wakeup" messages to all the workers, which respond with "ready" messages. Each time the coordinator receives a ready message from a worker, it matches an outstanding work item to that worker and sends it back. When the worker completes its work item, it responds with another ready message. Work items are thus sent out as workers become ready, rather than requiring coordination across workers. The framework allows workers to send identifying information that the coordinator can use for matching. For example the Hdfs and WebHdfs frameworks instantiate work generators in which the workers identify themselves by the IP addresses of the computer they are running on, allowing file blocks to be preferentially read on the worker computer that is hosting them.
Microsoft.Research.Naiad.Input
The Input namespace provides classes and extension methods for ingesting data into a Naiad computation.
Microsoft.Research.Naiad.Runtime.Progress
The Runtime.Progress namespace provides classes that support tracking the progress of a Naiad computation.
Microsoft.Research.Naiad.Serialization
The Serialization namespace contains classes that support serialization of Naiad data for network and file I/O.
Microsoft.Research.Naiad.Utilities
The Utilities namespace contains miscellaneous classes that support Naiad framework authors.