public class ProcessFlow<P> extends HadoopFlow
Flow
subclass that supports custom Riffle jobs.
Use this class to allow custom Riffle jobs to participate in the Cascade
scheduler. If
other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled
according to their dependencies (topologically).
Though this class sub-classes HadoopFlow
, it does not support all the methods available or features.
Currently FlowListener
s are supported but the
FlowListener.onThrowable(cascading.flow.Flow, Throwable)
event is not.BaseFlow.FlowHolder
flowStats, sinks, sources, stop, stopJobsOnExit, thread
CASCADING_FLOW_ID
Constructor and Description |
---|
ProcessFlow(java.util.Map<java.lang.Object,java.lang.Object> properties,
java.lang.String name,
P process)
Constructor ProcessFlow creates a new ProcessFlow instance.
|
ProcessFlow(java.lang.String name,
P process)
Constructor ProcessFlow creates a new ProcessFlow instance.
|
Modifier and Type | Method and Description |
---|---|
void |
cleanup() |
void |
complete()
Method complete starts the current Flow instance if it has not be previously started, then block until completion.
|
P |
getProcess()
Method getProcess returns the process of this ProcessFlow object.
|
void |
prepare()
Method prepare is used by a
Cascade to notify the given Flow it should initialize or clear any resources
necessary for Flow.start() to be called successfully. |
void |
setTapFromProcess()
Method setTapFromProcess build
Tap instance for the give process incoming and outgoing dependencies. |
void |
start()
Method start begins the execution of this Flow instance.
|
void |
stop()
Method stop stops all running jobs, killing any currently executing.
|
java.lang.String |
toString() |
getConfig, getConfigAsProperties, getConfigCopy, getFlowProcess, getMaxNumParallelSteps, getProperty, initConfig, initFromProperties, internalClean, internalShutdown, internalStart, isPreserveTemporaryFiles, newConfig, setConfigProperty, stepsAreLocal
addListener, addStepListener, areSinksStale, areSourcesNewer, createConfig, createFlowThread, deleteCheckpointsIfNotUpdate, deleteCheckpointsIfReplace, deleteSinks, deleteSinksIfNotUpdate, deleteSinksIfReplace, deleteTrapsIfNotUpdate, deleteTrapsIfReplace, fireOnCompleted, fireOnStarting, fireOnStopping, fireOnThrowable, getCascadeID, getCascadingServices, getCheckpointNames, getCheckpoints, getCheckpointsCollection, getClassPath, getFieldsFor, getFlowSession, getFlowSkipStrategy, getFlowStats, getFlowSteps, getFlowStepStrategy, getHolder, getID, getName, getPlatformInfo, getRunID, getSink, getSink, getSinkModified, getSinkNames, getSinks, getSinksCollection, getSource, getSourceNames, getSources, getSourcesCollection, getSpawnStrategy, getStats, getSubmitPriority, getTags, getTrapNames, getTraps, getTrapsCollection, handleExecutorShutdown, hasListeners, hasStepListeners, initialize, initializeNewJobsMap, initSteps, internalStopAllJobs, isSkipFlow, isStopJobsOnExit, logInfo, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, presentSinkFields, presentSourceFields, registerShutdownHook, removeListener, removeStepListener, resourceExists, retrieveSinkFields, retrieveSourceFields, setCascade, setCheckpoints, setFlowSkipStrategy, setFlowStepGraph, setFlowStepStrategy, setName, setSinks, setSources, setSpawnStrategy, setSubmitPriority, setTraps, updateSchemes, writeDOT, writeStepsDOT
@ConstructorProperties(value={"name","process"}) public ProcessFlow(java.lang.String name, P process)
name
- of type Stringprocess
- of type JobConf@ConstructorProperties(value={"properties","name","process"}) public ProcessFlow(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.String name, P process)
properties
- of type Mapname
- of type Stringprocess
- of type Ppublic void setTapFromProcess()
Tap
instance for the give process incoming and outgoing dependencies.
This method may be called repeatedly to re-configure the source and sink taps.public P getProcess()
public void prepare()
Flow
Cascade
to notify the given Flow it should initialize or clear any resources
necessary for Flow.start()
to be called successfully.
Specifically, this implementation calls BaseFlow.deleteSinksIfNotUpdate()
&& BaseFlow.deleteTrapsIfNotUpdate()
.public void start()
Flow
Flow.complete()
to block until this Flow completes.public void stop()
Flow
public void complete()
Flow
public void cleanup()