Writing your first Naiad programNaiad Help

[This is preliminary documentation and is subject to change.]

Here we will dive into writing a first Naiad program: one that simply accepts strings read from the console and writes them back to the console.

Prerequisites

For concreteness, this tutorial will assume that you are using Visual Studio 2013, but similar steps apply for other development environments.

Create a new C# project with a Naiad reference

  1. In Visual Studio, create a new project of type "Console Application", and provide a descriptive name (perhaps "MyFirstNaiadProgram").

  2. Add a reference to your project (for example, using the Solution Explorer pane) to Microsoft.Research.Naiad.dll.

    To use NuGet, select "Manage NuGet Packages...", and install the "Naiad - Core" package.

    If you are using a locally-built copy of the Naiad sources, add a reference by browsing to the locally-built copy of Microsoft.Research.Naiad.dll

  3. Add the following statements at the top of Program.cs.

    using Microsoft.Research.Naiad;
    using Microsoft.Research.Naiad.Input;

    These namespaces contain important classes and methods for initializing and using Naiad, as well as providing input to Naiad computations.

At this point we have an empty program and are ready to start using Naiad. We'll next walk through the steps needed for a program to start using Naiad.

Writing a Naiad program to receive and display lines of text

  1. The first step in any Naiad program is to create a Computation. This can be done several ways, but the simplest is to use the NewComputation.FromArgs() method, which extracts Naiad-specific arguments from the command-line and returns a computation. The computation implements IDisposable, and should be used inside a using block.

  2. We next create a data source, an object to which our program can supply input, and from which a Naiad computation can accept input. A BatchedDataSource is one that accepts batches of inputs at a time. There is a DataSource interface allowing the programmer to define their own data sources, as required.

  3. The next step is to inform the computation of our data source, and get a handle to a Naiad dataflow stream representing the input.

  4. As this is a simple example, we will do nothing complicated with the stream, and only apply Naiad's mechanism for producing output. The Subscribe method on streams takes a callback on an enumeration of records, and applies the callback once for each batch of records it receives.

  5. Having fully defined our dataflow computation, we now activate the computation. Activation freezes dataflow construction and registers the computation with Naiad's workers, who are now able to start working.

  6. We now start reading lines of input from the console, spliting each line into a collection of words and handing them to our data source. Naiad introduces the strings into our dataflow computation, and they make their way to the subscribe callback.

  7. Once we have finished producing input for the computation we close the data source. This indicates to Naiad that no more data is expected, and it can start to cleanly shut down the parts of the computation with no further work.

  8. Before exiting the using block we call Join on the computation to block until all work has completed. It is an error to exit a using block (or otherwise dispose of a computation) without first calling Join.

The full program we have assembled reads as follows:

static void Main(string[] args)
{
  // 1. allocate a new dataflow computation.
  using (var computation = NewComputation.FromArgs(ref args))
  {
    // 2. define an object which accepts input strings.
    var source = new BatchedDataSource<string>();

    // 3. convert the data source into a Naiad stream of strings.
    var input = computation.NewInput(source);

    // 4.request a notification for each batch of strings received.
    var output = input.Subscribe(x =>
                 {
                   foreach (var line in x)
                     Console.WriteLine(line);
                 });

    // 5. start the computation, fixing the structure of the dataflow graph.
    computation.Activate();

    // 6. read inputs from the console as long as the user supplies them.
    for (var l = Console.ReadLine(); l.Length > 0; l = Console.ReadLine())
      source.OnNext(l.Split());

    // 7. signal that the source is now complete.
    source.OnCompleted();

    // 8. block until all work is finished.
    computation.Join();
  }
}