Package org.apache.tika.eval.core.tokens

Class CommonTokenCountManager

java.lang.Object

org.apache.tika.eval.core.tokens.CommonTokenCountManager

public class CommonTokenCountManager
extends Object

Constructor Summary

Constructors

Constructor	Description
CommonTokenCountManager()
CommonTokenCountManager(Path commonTokensDir, String defaultLangCode)

Method Summary

Modifier and Type	Method	Description
void	close()
CommonTokenResult	countTokenOverlaps(String langCode, Map<String,org.apache.commons.lang3.mutable.MutableInt> tokens)	Deprecated.
Set<String>	getLangs()
org.apache.commons.lang3.tuple.Pair<String,LangModel>	getLangTokens(String lang)
Set<String>	getTokens(String lang)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

CommonTokenCountManager

public CommonTokenCountManager()

CommonTokenCountManager

public CommonTokenCountManager(Path commonTokensDir,
 String defaultLangCode)

Method Details

countTokenOverlaps

@Deprecated
public CommonTokenResult countTokenOverlaps(String langCode,
 Map<String,org.apache.commons.lang3.mutable.MutableInt> tokens)
                                     throws IOException

Deprecated.

Throws:

IOException

getTokens

public Set<String> getTokens(String lang)

getLangs

public Set<String> getLangs()

getLangTokens

public org.apache.commons.lang3.tuple.Pair<String,LangModel> getLangTokens(String lang)

Parameters:

lang -

Returns:

pair of actual language code used and a set of common tokens for that language

close

public void close()
           throws IOException

Throws:

IOException