public abstract class SpillableTupleMap extends java.util.HashMap<Tuple,java.util.Collection<Tuple>> implements Spillable
SpillableTupleList
instance. Only
values are spilled, keys are not spilled and too many keys can result in a OutOfMemoryError
.
The SpillableProps.MAP_THRESHOLD
value sets the number of total values that this map will
strive to keep in memory regardless of the number of keys. This is achieved by dynamically calculating the threshold
used by each child SpillableTupleList instance using
threshold = Min( list_threshold, map_threshold / current_num_keys )
.
To set the list threshold, see SpillableProps
fluent helper class.
This class is used by the HashJoin
pipe, to set properties specific to a given
join instance, see the Pipe.getConfigDef()
method.HadoopSpillableTupleMap
,
Serialized Formjava.util.AbstractMap.SimpleEntry<K,V>, java.util.AbstractMap.SimpleImmutableEntry<K,V>
Spillable.SpillListener, Spillable.SpillStrategy
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
MAP_CAPACITY
Deprecated.
|
static java.lang.String |
MAP_LOADFACTOR
Deprecated.
|
static java.lang.String |
MAP_THRESHOLD
Deprecated.
|
Constructor and Description |
---|
SpillableTupleMap(int initialCapacity,
float loadFactor,
int mapThreshold,
int initListThreshold) |
SpillableTupleMap(int mapThreshold,
int initListThreshold) |
Modifier and Type | Method and Description |
---|---|
protected abstract java.util.Collection<Tuple> |
createTupleCollection(Tuple object) |
java.util.Collection<Tuple> |
get(java.lang.Object object) |
Tuple |
getGrouping() |
int |
getInitListThreshold() |
static int |
getMapCapacity(FlowProcess flowProcess,
int defaultValue) |
static float |
getMapLoadFactor(FlowProcess flowProcess,
float defaultValue) |
protected int |
getMapThreshold() |
static int |
getMapThreshold(FlowProcess flowProcess,
int defaultValue) |
Spillable.SpillListener |
getSpillListener() |
void |
setGrouping(Tuple group) |
void |
setSpillListener(Spillable.SpillListener spillListener) |
void |
setSpillStrategy(Spillable.SpillStrategy spillStrategy) |
int |
spillCount()
The number of times this container has spilled data to disk.
|
clear, clone, containsKey, containsValue, entrySet, isEmpty, keySet, put, putAll, remove, size, values
@Deprecated public static final java.lang.String MAP_THRESHOLD
@Deprecated public static final java.lang.String MAP_CAPACITY
HashMap
,
Constant Field Values@Deprecated public static final java.lang.String MAP_LOADFACTOR
HashMap
,
Constant Field Valuespublic SpillableTupleMap(int mapThreshold, int initListThreshold)
public SpillableTupleMap(int initialCapacity, float loadFactor, int mapThreshold, int initListThreshold)
public static int getMapThreshold(FlowProcess flowProcess, int defaultValue)
public static int getMapCapacity(FlowProcess flowProcess, int defaultValue)
public static float getMapLoadFactor(FlowProcess flowProcess, float defaultValue)
protected int getMapThreshold()
public int getInitListThreshold()
public java.util.Collection<Tuple> get(java.lang.Object object)
protected abstract java.util.Collection<Tuple> createTupleCollection(Tuple object)
public void setGrouping(Tuple group)
setGrouping
in interface Spillable
public Tuple getGrouping()
getGrouping
in interface Spillable
public void setSpillStrategy(Spillable.SpillStrategy spillStrategy)
setSpillStrategy
in interface Spillable
public int spillCount()
Spillable
spillCount
in interface Spillable
public Spillable.SpillListener getSpillListener()
public void setSpillListener(Spillable.SpillListener spillListener)
setSpillListener
in interface Spillable