[This is preliminary documentation and is subject to change.]
extension methods for working with Java-protocol Hdfs files
Inheritance Hierarchy
Microsoft.Research.Naiad.Frameworks.HdfsExtensionMethods
Namespace: Microsoft.Research.Naiad.Frameworks.Hdfs
Assembly: Microsoft.Research.Naiad.HdfsSupport (in Microsoft.Research.Naiad.HdfsSupport.dll) Version: 0.5.0.0 (0.5.0.0)
Syntax
C#
public static class ExtensionMethods
The ExtensionMethods type exposes the following members.
Methods
| Name | Description | |
|---|---|---|
  | FromHdfsTOutput, TTime |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized in a custom binary format. Concatenate all the records to the output, in an unspecified order.
|
| FromHdfsBinaryTOutput, TTime |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized in the default Naiad binary format. Concatenate all the records to the output, in an unspecified order.
|
| FromHdfsTextTTime |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized as lines of text. Concatenate all the lines of the files to the output, in an unspecified order.
|
| ReadHdfsBinaryCollectionTOutput |
Read a collection of HDFS files serialized in the default Naiad binary format. Concatenate all the records to the output,
in an unspecified order.
|
| ReadHdfsCollectionTOutput |
Read a collection of HDFS files serialized in a custom binary format. Concatenate all the records to the output,
in an unspecified order.
|
| ReadHdfsTextCollection |
Read a collection of HDFS files serialized as lines of text. Concatenate all the lines of the files to the output,
in an unspecified order.
|
| ToHdfsBinaryTOutput, TTime(StreamTOutput, TTime, Uri, Int32, Int64, Int64) |
method to write a stream of records to a collection of HDFS files using the default Naiad binary serializer,
partitioned by time as well as key. Within a given time and part, records are written in an undefined order
|
| ToHdfsBinaryTOutput, TWriter, TTime(StreamTOutput, TTime, FuncInt32, Int32, TTime, Int32, Uri, FuncStream, TWriter, ActionTWriter, ArraySegmentTOutput, Int32, Int64, Int64) |
general method to write a stream of records to a collection of HDFS files, partitioned by time as well as key.
Within a given time and part, records are written in an undefined order
|
| ToHdfsTextTTime |
write a sequence of strings as hdfs text files, partitioned by time as well as key.
Within a given time and part, records are written in an undefined order
|
| WriteHdfsBinaryTOutput(StreamTOutput, Epoch, Uri, Int32, Int64, Int64) |
method to write a stream of records to a collection of HDFS files using the default Naiad binary serializer.
The collection is active throughout the computation and is closed when the computation terminates: it concatenates
records from all epochs in an undefined order
|
| WriteHdfsBinaryTOutput, TWriter(StreamTOutput, Epoch, FuncInt32, Int32, Int32, Uri, FuncStream, TWriter, ActionTWriter, ArraySegmentTOutput, Int32, Int64, Int64) |
general method to write a stream of records to a collection of HDFS files. The collection is active
throughout the computation and is closed when the computation terminates: it concatenates records from all
epochs in an undefined order
|
| WriteHdfsText |
write a sequence of strings as hdfs text files. The collection is active throughout the computation and is
closed when the computation terminates: it concatenates records from all epochs in an undefined order
|
See Also