Description:

"Tails" a file, ingesting data from the file as it is written to the file. The file is expected to be textual. Data is ingested only when a new line is encountered (carriage return or new-line character or combination). If the file to tail is periodically "rolled over", as is generally the case with log files, an optional Rolling Filename Pattern can be used to retrieve data from files that have rolled over, even if the rollover occurred while NiFi was not running (provided that the data still exists upon restart of NiFi). It is generally advisable to set the Run Schedule to a few seconds, rather than running with the default value of 0 secs, as this Processor will consume a lot of resources if scheduled very aggressively. At this time, this Processor does not support ingesting files that have been compressed when 'rolled over'.

Tags:

tail, file, log, text, source

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values.

NameDefault ValueAllowable ValuesDescription
File to TailFully-qualified filename of the file that should be tailed
Rolling Filename PatternIf the file to tail "rolls over" as would be the case with log files, this filename pattern will be used to identify files that have rolled over so that if NiFi is restarted, and the file has rolled over, it will be able to pick up where it left off. This pattern supports wildcard characters * and ? and will assume that the files that have rolled over live in the same directory as the file being tailed.
State File./conf/state/mock-processorSpecifies the file that should be used for storing state about what data has been ingested so that upon restart NiFi can resume from where it left off
Initial Start PositionBeginning of File
  • Beginning of Time Start with the oldest data that matches the Rolling Filename Pattern and then begin reading from the File to Tail
  • Beginning of File Start with the beginning of the File to Tail. Do not ingest any data that has already been rolled over
  • Current Time Start with the data at the end of the File to Tail. Do not ingest any data thas has already been rolled over or any data in the File to Tail that has already been written.
When the Processor first begins to tail data, this property specifies where the Processor should begin reading data. Once data has been ingested from the file, the Processor will continue from the last point from which it has received data.

Relationships:

NameDescription
successAll FlowFiles are routed to this Relationship.

Reads Attributes:

None specified.

Writes Attributes:

None specified.