org.apache.hadoop.chukwa.datacollection.writer.parquet

Class ChukwaParquetWriter

java.lang.Object

org.apache.hadoop.chukwa.datacollection.writer.PipelineableWriter

org.apache.hadoop.chukwa.datacollection.writer.parquet.ChukwaParquetWriter

All Implemented Interfaces:

ChukwaWriter

public class ChukwaParquetWriter
extends PipelineableWriter

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.chukwa.datacollection.writer.ChukwaWriter

ChukwaWriter.COMMIT_PENDING, ChukwaWriter.CommitStatus

Field Summary

Fields

Modifier and Type	Field and Description
static String	OUTPUT_DIR_OPT
protected String	outputDir

Fields inherited from class org.apache.hadoop.chukwa.datacollection.writer.PipelineableWriter

next

Fields inherited from interface org.apache.hadoop.chukwa.datacollection.writer.ChukwaWriter

COMMIT_FAIL, COMMIT_OK

Constructor Summary

Constructors

Constructor and Description
ChukwaParquetWriter()
ChukwaParquetWriter(org.apache.hadoop.conf.Configuration c)

Method Summary

Modifier and Type	Method and Description
ChukwaWriter.CommitStatus	add(List<Chunk> chunks) Called repeatedly with data that should be serialized.
void	close() Called once, indicating that the writer should close files and prepare to exit.
long	getDelayForFixedInterval(long currentTime, long rotateInterval, long offsetInterval) Calculates delay for scheduling the next rotation in case of FixedTimeRotatorScheme.
void	init(org.apache.hadoop.conf.Configuration c) Called once to initialize this writer.

Methods inherited from class org.apache.hadoop.chukwa.datacollection.writer.PipelineableWriter

setNextStage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_DIR_OPT

public static final String OUTPUT_DIR_OPT

See Also:

Constant Field Values

outputDir

protected String outputDir

Constructor Detail

ChukwaParquetWriter

public ChukwaParquetWriter()
                    throws WriterException

Throws:

WriterException

ChukwaParquetWriter

public ChukwaParquetWriter(org.apache.hadoop.conf.Configuration c)
                    throws WriterException

Throws:

WriterException

Method Detail

init

public void init(org.apache.hadoop.conf.Configuration c)
          throws WriterException

Description copied from interface: ChukwaWriter

Called once to initialize this writer.

Throws:

WriterException

close

public void close()
           throws WriterException

Description copied from interface: ChukwaWriter

Called once, indicating that the writer should close files and prepare to exit.

Throws:

WriterException

add

public ChukwaWriter.CommitStatus add(List<Chunk> chunks)
                              throws WriterException

Description copied from interface: ChukwaWriter

Called repeatedly with data that should be serialized. Subclasses may assume that init() will be called before any calls to add(), and that add() won't be called after close().

Specified by:

add in interface ChukwaWriter

Overrides:

add in class PipelineableWriter

Returns:

CommitStatus

Throws:

WriterException

getDelayForFixedInterval

public long getDelayForFixedInterval(long currentTime,
                                     long rotateInterval,
                                     long offsetInterval)

Calculates delay for scheduling the next rotation in case of FixedTimeRotatorScheme. This delay is the time difference between the currentTimestamp (t1) and the next time the collector should rotate the sequence files (t2). t2 is the time when the current rotateInterval ends plus an offset (as set by chukwaCollector.FixedTimeIntervalOffset). So, delay = t2 - t1

Parameters:

currentTime - - the current timestamp

rotateInterval - - chukwaCollector.rotateInterval

offsetInterval - - chukwaCollector.fixedTimeIntervalOffset

Returns:

delay for scheduling next rotation