public class FileTap extends Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream> implements FileType<java.util.Properties>
Tap
sub-class that allows for direct local file access.
FileTap must be used with the LocalFlowConnector
to create
Flow
instances that run in "local" mode.Constructor and Description |
---|
FileTap(Scheme<java.util.Properties,java.io.InputStream,java.io.OutputStream,?,?> scheme,
java.lang.String path)
Constructor FileTap creates a new FileTap instance using the given
Scheme and file path . |
FileTap(Scheme<java.util.Properties,java.io.InputStream,java.io.OutputStream,?,?> scheme,
java.lang.String path,
SinkMode sinkMode)
|
Modifier and Type | Method and Description |
---|---|
boolean |
commitResource(java.util.Properties conf)
Method commitResource allows the underlying resource to be notified when all write processing is
successful so that any additional cleanup or processing may be completed.
|
boolean |
createResource(java.util.Properties conf)
Method createResource creates the underlying resource.
|
boolean |
deleteResource(java.util.Properties conf)
Method deleteResource deletes the resource represented by this instance.
|
java.lang.String[] |
getChildIdentifiers(java.util.Properties conf)
Method getChildIdentifiers returns an array of child identifiers if this resource is a directory.
|
java.lang.String[] |
getChildIdentifiers(java.util.Properties conf,
int depth,
boolean fullyQualified) |
java.lang.String |
getFullIdentifier(java.util.Properties conf)
Method getFullIdentifier returns a fully qualified resource identifier.
|
java.lang.String |
getIdentifier()
Method getIdentifier returns a String representing the resource this Tap instance represents.
|
long |
getModifiedTime(java.util.Properties conf)
Method getModifiedTime returns the date this resource was last modified.
|
long |
getSize(java.util.Properties conf)
Method getSize returns the size of the file referenced by this tap.
|
boolean |
isDirectory(java.util.Properties conf)
Method isDirectory returns true if the underlying resource represents a directory or folder instead
of an individual file.
|
TupleEntryIterator |
openForRead(FlowProcess<java.util.Properties> flowProcess,
java.io.InputStream input)
Method openForRead opens the resource represented by this Tap instance for reading.
|
TupleEntryCollector |
openForWrite(FlowProcess<java.util.Properties> flowProcess,
java.io.OutputStream output)
Method openForWrite opens the resource represented by this Tap instance for writing.
|
boolean |
resourceExists(java.util.Properties conf)
Method resourceExists returns true if the path represented by this instance exists.
|
createResource, deleteResource, equals, flowConfInit, getConfigDef, getFullIdentifier, getModifiedTime, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForWrite, outgoingScopeFor, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, rollbackResource, setScheme, sinkConfInit, sourceConfInit, taps, toString
public FileTap(Scheme<java.util.Properties,java.io.InputStream,java.io.OutputStream,?,?> scheme, java.lang.String path)
Scheme
and file path
.scheme
- of type LocalSchemepath
- of type Stringpublic java.lang.String getIdentifier()
Tap
getIdentifier
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
public java.lang.String getFullIdentifier(java.util.Properties conf)
Tap
getFullIdentifier
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
conf
- of type Configpublic TupleEntryIterator openForRead(FlowProcess<java.util.Properties> flowProcess, java.io.InputStream input) throws java.io.IOException
Tap
input
value may be null, if so, sub-classes must inquire with the underlying Scheme
via Scheme.sourceConfInit(cascading.flow.FlowProcess, Tap, Object)
to get the proper
input type and instantiate it before calling super.openForRead()
.
Note the returned iterator will return the same instance of TupleEntry
on every call,
thus a copy must be made of either the TupleEntry or the underlying Tuple
instance if they are to be
stored in a Collection.openForRead
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
flowProcess
- of type FlowProcessinput
- of type Inputjava.io.IOException
- when the resource cannot be openedpublic TupleEntryCollector openForWrite(FlowProcess<java.util.Properties> flowProcess, java.io.OutputStream output) throws java.io.IOException
Tap
SinkMode
setting. If SinkMode is
SinkMode.REPLACE
, this call may fail. See Tap.openForWrite(cascading.flow.FlowProcess)
.
output
value may be null, if so, sub-classes must inquire with the underlying Scheme
via Scheme.sinkConfInit(cascading.flow.FlowProcess, Tap, Object)
to get the proper
output type and instantiate it before calling super.openForWrite()
.openForWrite
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
flowProcess
- of type FlowProcessoutput
- of type Outputjava.io.IOException
- when the resource cannot be openedpublic long getSize(java.util.Properties conf) throws java.io.IOException
FileType
public boolean createResource(java.util.Properties conf) throws java.io.IOException
Tap
createResource
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
conf
- of type Configjava.io.IOException
- when there is an error making directoriespublic boolean deleteResource(java.util.Properties conf) throws java.io.IOException
Tap
deleteResource
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
conf
- of type Configjava.io.IOException
- when the resource cannot be deletedpublic boolean commitResource(java.util.Properties conf) throws java.io.IOException
Tap
Tap.rollbackResource(Object)
to handle cleanup in the face of failures.
This method is invoked once "client side" and not in the cluster, if any.
If other sink Tap instance in a given Flow fail on commitResource after called on this instance,
rollbackResource will not be called.
commitResource
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
conf
- of type Configjava.io.IOException
public boolean resourceExists(java.util.Properties conf) throws java.io.IOException
Tap
resourceExists
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
conf
- of type Configjava.io.IOException
- when the status cannot be determinedpublic long getModifiedTime(java.util.Properties conf) throws java.io.IOException
Tap
getModifiedTime
in class Tap<java.util.Properties,java.io.InputStream,java.io.OutputStream>
conf
- of type Configjava.io.IOException
public boolean isDirectory(java.util.Properties conf) throws java.io.IOException
FileType
isDirectory
in interface FileType<java.util.Properties>
conf
- of JobConfjava.io.IOException
public java.lang.String[] getChildIdentifiers(java.util.Properties conf) throws java.io.IOException
FileType
_log
).getChildIdentifiers
in interface FileType<java.util.Properties>
conf
- of JobConfjava.io.IOException
public java.lang.String[] getChildIdentifiers(java.util.Properties conf, int depth, boolean fullyQualified) throws java.io.IOException
getChildIdentifiers
in interface FileType<java.util.Properties>
java.io.IOException