The root namespace for DryadLinq client programs
Classes
Class | Description | |
---|---|---|
AssociativeAttribute |
Indicates that a method is an associative aggregation method. The argument to the
constructor must be of type IAssociative. The computation of the method annotated
by this attribute can be decomposed to a sequence of calls to the Seed and
RecursiveAccumulate methods.
| |
CustomDryadLinqSerializerAttribute |
Provides a user-defined serialization method for a .NET type.
| |
DataProvider |
DataProvider provides an abstraction for different data backends. New data storage backends
could be added by subclassing this class.
| |
DecomposableAttribute |
Indicates that a method can be decomposed to multiple methods. The argument to the
constructor must be of type IDecomposable. The computation of the method annotated
by this attribute can be decomposed to a sequence of calls to the Seed, Accumulate,
RecursiveAccumulate methods and a FinalReduce.
| |
DryadLinqBinaryReader |
DryadLinqBinaryReader is the main interface for user provided custom serializers.
It is also used for DryadLINQ internal autoserialization to read primitive types.
| |
DryadLinqBinaryWriter |
DryadLinqBinaryWriter is the main interface for user provided custom serializers.
It is also used for DryadLINQ internal autoserialization to write primitive types.
| |
DryadLinqContext |
Represents the context necessary to prepare and execute a DryadLinq Query,
| |
DryadLinqException |
The exception that is thrown by DryadLINQ.
| |
DryadLinqExtension |
Defines some useful operators that are commonly used in applications. The
operators are defined using the basic DryadLINQ operators. This class
also shows how a user library can be defined.
| |
DryadLinqJobInfo |
Represents the current state of a set of DryadLINQ jobs that have already been
submitted for execution. A DryadLinqJobInfo object is returned after a job is
submitted for execution.
| |
DryadLinqLog |
DryadLINQ logging API.
| |
DryadLinqMetaData |
The metadata associated with a DryadLINQ dataset.
| |
DryadLinqQueryable |
This class extends LINQ with a set of new operators that are specific to DryadLINQ.
The new operators includes partitioning operators (HashPartition and RangePartition)
and the Apply operator that enables stateful transformations on datasets.
| |
DryadLinqStreamInfo |
Basic information of a dataset.
| |
DryadLinqYarnCluster |
The interface for a YARN native cluster.
| |
GenericAssociativeTAssoc, TAccumulate |
A helper class for calling IAssociative methods more efficiently. It is used in
auto-generated vertex code. A DryadLINQ user should not need to use this class directly.
| |
GenericDecomposableTDecomposable, TSource, TAccumulate, TResult |
A helper class for calling IDecomposable methods more efficiently. It is used in
auto-generated vertex code. A DryadLINQ user should not need to use this class directly.
| |
NullableAttribute |
The Nullable attribute specifies if a field is nullable. The information is used by DryadLINQ
serialization. DryadLINQ serialization by default treats all fields not nullable.
| |
ResourceAttribute |
The Resource attribute is used to specify the computation cost of a user defined
function (UDF). IsStateful asserts that the function is stateful; IsExpensive
asserts that the function is expensive to compute. The information is useful in
generating better execution plan. For example, expensive associative aggregation
functions enables the use of multiple aggregation layers.
|
Structures
Structure | Description | |
---|---|---|
ForkTupleT1, T2 |
Represents a pair of elements that may not have valid values.
| |
ForkTupleT1, T2, T3 |
Represents a tuple of three elements that may not have valid values.
| |
ForkValueT |
Represents an element that may not have valid value.
| |
LineRecord |
The DryadLINQ type to represent a line of text.
| |
PairT1, T2 |
Represents a key/value pair. Very similar to KeyValuePair, but adds a few more methods.
|
Interfaces
Interface | Description | |
---|---|---|
DryadLinqCluster |
Base interface for cluster types that the DryadLinqContext constructor can accept.
| |
IAssociativeTAccumulate |
Defines the DryadLINQ interface for associative accumulator.
| |
IDecomposableTSource, TAccumulate, TResult |
Defines the DryadLINQ interface for decomposable functions. It allows a function to be
decomposed into the composition of several functions that can be executed more efficiently.
| |
IDryadLinqSerializerT |
The interface for providing user-defined serialization for a .NET type.
If a class T implements DryadLinqSerializer{T}, DryadLinq will use the
read and write methods of the class to do serialization.
| |
IKeyedMultiQueryableT, K |
The interface to access a collection of IQueryable{T} instances. Each IQueryable{T}
contains only elements of the same key. The IQueryable{T}s are indexed by a set of keys.
| |
IMultiQueryable |
The base interface to access a collection of IQueryable instances. The
DryadLINQ Fork operator returns a value that implements this interface.
| |
IMultiQueryableT1, T2 |
The interface to access a collection of two IQueryable{T} instances.
| |
IMultiQueryableT1, T2, T3 |
The interface to access a collection of three IQueryable{T} instances.
|
Enumerations
Enumeration | Description | |
---|---|---|
CompressionScheme |
Specifies whether to compress the output of a DryadLINQ vertex.
| |
ExecutorKind |
The executor to run DryadLINQ jobs. The current release only supports Dryad.
| |
PlatformKind |
The service platforms where you can run DryadLINQ.
| |
QueryLoggingLevel |
Logging levels for DryadLinqQuery runtime.
|