org.apache.crunch.io
Class SequentialFileNamingScheme
java.lang.Object
org.apache.crunch.io.SequentialFileNamingScheme
- All Implemented Interfaces:
- FileNamingScheme
public class SequentialFileNamingScheme
- extends Object
- implements FileNamingScheme
Default FileNamingScheme
that uses an incrementing sequence number in
order to generate unique file names.
Method Summary |
String |
getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
org.apache.hadoop.fs.Path outputDirectory)
Get the output file name for a map task. |
String |
getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
org.apache.hadoop.fs.Path outputDirectory,
int partitionId)
Get the output file name for a reduce task. |
SequentialFileNamingScheme
public SequentialFileNamingScheme()
getMapOutputName
public String getMapOutputName(org.apache.hadoop.conf.Configuration configuration,
org.apache.hadoop.fs.Path outputDirectory)
throws IOException
- Description copied from interface:
FileNamingScheme
- Get the output file name for a map task. Note that the implementation is
responsible for avoiding naming collisions.
- Specified by:
getMapOutputName
in interface FileNamingScheme
- Parameters:
configuration
- The configuration of the job for which the map output
is being writtenoutputDirectory
- The directory where the output will be written
- Returns:
- The filename for the output of the map task
- Throws:
IOException
- if an exception occurs while accessing the output file
system
getReduceOutputName
public String getReduceOutputName(org.apache.hadoop.conf.Configuration configuration,
org.apache.hadoop.fs.Path outputDirectory,
int partitionId)
throws IOException
- Description copied from interface:
FileNamingScheme
- Get the output file name for a reduce task. Note that the implementation is
responsible for avoiding naming collisions.
- Specified by:
getReduceOutputName
in interface FileNamingScheme
- Parameters:
configuration
- The configuration of the job for which output is being
writtenoutputDirectory
- The directory where the file will be writtenpartitionId
- The partition of the reduce task being output
- Returns:
- The filename for the output of the reduce task
- Throws:
IOException
- if an exception occurs while accessing output file
system
Copyright © 2013 The Apache Software Foundation. All Rights Reserved.