[This is preliminary documentation and is subject to change.]
extension methods for working with WebHdfs files
Inheritance Hierarchy
Microsoft.Research.Naiad.Frameworks.WebHdfsExtensionMethods
Namespace: Microsoft.Research.Naiad.Frameworks.WebHdfs
Assembly: Microsoft.Research.Naiad.WebHdfsSupport (in Microsoft.Research.Naiad.WebHdfsSupport.dll) Version: 0.5.0.0 (0.5.0.0)
Syntax
C#
public static class ExtensionMethods
The ExtensionMethods type exposes the following members.
Methods
| Name | Description | |
|---|---|---|
  | FromWebHdfsTOutput, TTime(StreamUri, TTime, FuncStream, IEnumerableArraySegmentTOutput) |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized in a custom binary format. Concatenate all the records to the output, in an unspecified order.
|
| FromWebHdfsTOutput, TTime(StreamUri, TTime, FuncStream, IEnumerableArraySegmentTOutput, String, Int32) |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized in a custom binary format. Concatenate all the records to the output, in an unspecified order.
|
| FromWebHdfsBinaryTOutput, TTime(StreamUri, TTime) |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized in the default Naiad binary format. Concatenate all the records to the output, in an unspecified order.
|
| FromWebHdfsBinaryTOutput, TTime(StreamUri, TTime, String, Int32) |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized in the default Naiad binary format. Concatenate all the records to the output, in an unspecified order.
|
| FromWebHdfsTextTTime(StreamUri, TTime) |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized as lines of text. Concatenate all the lines of the files to the output, in an unspecified order.
|
| FromWebHdfsTextTTime(StreamUri, TTime, String, Int32) |
Read a stream of path names (file or directory names) each of which corresponds to a collection of HDFS files
serialized as lines of text. Concatenate all the lines of the files to the output, in an unspecified order.
|
| ReadWebHdfsBinaryCollectionTOutput(Computation, Uri) |
Read a collection of HDFS files serialized in the default Naiad binary format. Concatenate all the records to the output,
in an unspecified order.
|
| ReadWebHdfsBinaryCollectionTOutput(Computation, Uri, String, Int32) |
Read a collection of HDFS files serialized in the default Naiad binary format. Concatenate all the records to the output,
in an unspecified order.
|
| ReadWebHdfsCollectionTOutput(Computation, Uri, FuncStream, IEnumerableArraySegmentTOutput) |
Read a collection of HDFS files serialized in a custom binary format. Concatenate all the records to the output,
in an unspecified order.
|
| ReadWebHdfsCollectionTOutput(Computation, Uri, FuncStream, IEnumerableArraySegmentTOutput, String, Int32) |
Read a collection of HDFS files serialized in a custom binary format. Concatenate all the records to the output,
in an unspecified order.
|
| ReadWebHdfsTextCollection(Computation, Uri) |
Read a collection of HDFS files serialized as lines of text. Concatenate all the lines of the files to the output,
in an unspecified order.
|
| ReadWebHdfsTextCollection(Computation, Uri, String, Int32) |
Read a collection of HDFS files serialized as lines of text. Concatenate all the lines of the files to the output,
in an unspecified order.
|
| ToWebHdfsBinaryTOutput, TTime(StreamTOutput, TTime, String, Int32, Uri, Int32, Int64, Int64) |
method to write a stream of records to a collection of HDFS files using the default Naiad binary serializer,
partitioned by time as well as key. Within a given time and part, records are written in an undefined order
|
| ToWebHdfsBinaryTOutput, TWriter, TTime(StreamTOutput, TTime, String, Int32, FuncInt32, Int32, TTime, Int32, Uri, FuncStream, TWriter, ActionTWriter, ArraySegmentTOutput, Int64, Int64) |
general method to write a stream of records to a collection of HDFS files, partitioned by time as well as key.
Within a given time and part, records are written in an undefined order
|
| ToWebHdfsTextTTime |
write a sequence of strings as hdfs text files, partitioned by time as well as key.
Within a given time and part, records are written in an undefined order
|
| WriteWebHdfsBinaryTOutput(StreamTOutput, Epoch, String, Int32, Uri, Int32, Int64, Int64) |
method to write a stream of records to a collection of HDFS files using the default Naiad binary serializer.
The collection is active throughout the computation and is closed when the computation terminates: it concatenates
records from all epochs in an undefined order
|
| WriteWebHdfsBinaryTOutput, TWriter(StreamTOutput, Epoch, String, Int32, FuncInt32, Int32, Int32, Uri, FuncStream, TWriter, ActionTWriter, ArraySegmentTOutput, Int64, Int64) |
general method to write a stream of records to a collection of HDFS files. The collection is active
throughout the computation and is closed when the computation terminates: it concatenates records from all
epochs in an undefined order
|
| WriteWebHdfsText |
write a sequence of strings as hdfs text files. The collection is active throughout the computation and is
closed when the computation terminates: it concatenates records from all epochs in an undefined order
|
See Also