org.apache.lens.ml.algo.spark

Class HiveTableRDD

java.lang.Object

org.apache.lens.ml.algo.spark.HiveTableRDD

public final class HiveTableRDD
extends Object

Create a JavaRDD based on a Hive table using HCatInputFormat.

Method Summary

Modifier and Type	Method and Description
static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.WritableComparable,org.apache.hive.hcatalog.data.HCatRecord>	createHiveTableRDD(org.apache.spark.api.java.JavaSparkContext javaSparkContext, org.apache.hadoop.conf.Configuration conf, String db, String table, String partitionFilter) Creates the hive table rdd.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createHiveTableRDD

public static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.WritableComparable,org.apache.hive.hcatalog.data.HCatRecord> createHiveTableRDD(org.apache.spark.api.java.JavaSparkContext javaSparkContext,
                                                                                                                                                         org.apache.hadoop.conf.Configuration conf,
                                                                                                                                                         String db,
                                                                                                                                                         String table,
                                                                                                                                                         String partitionFilter)
                                                                                                                                                  throws IOException

Creates the hive table rdd.

Parameters:

javaSparkContext - the java spark context

conf - the conf

db - the db

table - the table

partitionFilter - the partition filter

Returns:

the java pair rdd

Throws:

IOException - Signals that an I/O exception has occurred.