org.apache.giraph.io.gora.utils

Class ExtraGoraInputFormat<K,T extends org.apache.gora.persistency.impl.PersistentBase>

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,T>

org.apache.giraph.io.gora.utils.ExtraGoraInputFormat<K,T>

Type Parameters:

K - KeyClass.

T - PersistentClass.

public class ExtraGoraInputFormat<K,T extends org.apache.gora.persistency.impl.PersistentBase>
extends org.apache.hadoop.mapreduce.InputFormat<K,T>

InputFormat to fetch the input from Gora data stores. The query to fetch the items from the datastore should be prepared and set via setQuery(Job, Query), before submitting the job. Hadoop jobs can be either configured through static setInput() methods, or from GoraMapper.

Field Summary

Fields

Modifier and Type	Field and Description
static String	QUERY_KEY String used to map partitioned queries into configuration object.

Constructor Summary

Constructors

Constructor and Description
ExtraGoraInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<K,T>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.gora.store.DataStore<K,T>	getDataStore()
org.apache.gora.query.Query<K,T>	getQuery()
org.apache.gora.query.Query<K,T>	getQuery(org.apache.hadoop.conf.Configuration conf) Gets the partitioned query from the conf object passed.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context) Gets splits.
void	setDataStore(org.apache.gora.store.DataStore<K,T> datStore)
static <K,V extends org.apache.gora.persistency.Persistent> void	setInput(org.apache.hadoop.mapreduce.Job job, org.apache.gora.query.Query<K,V> query, org.apache.gora.store.DataStore<K,V> dataStore, boolean reuseObjects) Sets the input parameters for the job
static <K,T extends org.apache.gora.persistency.Persistent> void	setQuery(org.apache.hadoop.conf.Configuration conf, org.apache.gora.query.Query<K,T> query) Sets the partitioned query inside the job object.
void	setQuery(org.apache.gora.query.Query<K,T> query)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

QUERY_KEY

public static final String QUERY_KEY

String used to map partitioned queries into configuration object.

See Also:

Constant Field Values

Constructor Detail

ExtraGoraInputFormat

public ExtraGoraInputFormat()

Method Detail

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<K,T> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                        org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                 throws IOException,
                                                                        InterruptedException

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<K,T extends org.apache.gora.persistency.impl.PersistentBase>

Parameters:

split - InputSplit to be used.

context - JobContext to be used.

Returns:

RecordReader record reader used inside Hadoop job.

Throws:

IOException

InterruptedException

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
                                                       throws IOException,
                                                              InterruptedException

Gets splits.

Specified by:

getSplits in class org.apache.hadoop.mapreduce.InputFormat<K,T extends org.apache.gora.persistency.impl.PersistentBase>

Parameters:

context - for the job.

Returns:

splits found

Throws:

IOException

InterruptedException

getDataStore

public org.apache.gora.store.DataStore<K,T> getDataStore()

Returns:

the dataStore

setDataStore

public void setDataStore(org.apache.gora.store.DataStore<K,T> datStore)

Parameters:

datStore - the dataStore to set

getQuery

public org.apache.gora.query.Query<K,T> getQuery()

Returns:

the query

setQuery

public void setQuery(org.apache.gora.query.Query<K,T> query)

Parameters:

query - the query to set

setQuery

public static <K,T extends org.apache.gora.persistency.Persistent> void setQuery(org.apache.hadoop.conf.Configuration conf,
                                                                                 org.apache.gora.query.Query<K,T> query)
                                                                          throws IOException

Sets the partitioned query inside the job object.

Type Parameters:

K - Key class

T - Persistent class

Parameters:

conf - Configuration used.

query - Query to be executed.

Throws:

IOException - Exception that be might thrown.

getQuery

public org.apache.gora.query.Query<K,T> getQuery(org.apache.hadoop.conf.Configuration conf)
                                          throws IOException

Gets the partitioned query from the conf object passed.

Parameters:

conf - Configuration object.

Returns:

passed inside the configuration object

Throws:

IOException - Exception that might be thrown.

setInput

public static <K,V extends org.apache.gora.persistency.Persistent> void setInput(org.apache.hadoop.mapreduce.Job job,
                                                                                 org.apache.gora.query.Query<K,V> query,
                                                                                 org.apache.gora.store.DataStore<K,V> dataStore,
                                                                                 boolean reuseObjects)
                                                                          throws IOException

Sets the input parameters for the job

Type Parameters:

K - Key class

V - Persistent class

Parameters:

job - the job to set the properties for

query - the query to get the inputs from

dataStore - the datastore as the input

reuseObjects - whether to reuse objects in serialization

Throws:

IOException