geotrellis.spark.io.hadoop.formats

GdalInputFormat

class GdalInputFormat extends FileInputFormat[GdalRasterInfo, Tile]

Uses GDAL to attempt to read a raster file.

GDAL only supports reading files from local filesystem. In order for this InputFormat to work it must copy the entire input file to hadoop.tmp.dir, if it is not already on a local file system, and invoke GDAL JNI bindings.

If there the .so/.dylib files are not found in the class path, this will crash gloriously.

Also note that all the blocks need to be shuffled to a single machine for each file. If the file being ingested is much larger than HDFS block size this will be very inefficient.

Linear Supertypes
FileInputFormat[GdalRasterInfo, Tile], InputFormat[GdalRasterInfo, Tile], AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. GdalInputFormat
  2. FileInputFormat
  3. InputFormat
  4. AnyRef
  5. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GdalInputFormat()

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. def addInputPathRecursively(arg0: List[FileStatus], arg1: FileSystem, arg2: Path, arg3: PathFilter): Unit

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
    Annotations
    @throws( classOf[java.io.IOException] )
  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def computeSplitSize(arg0: Long, arg1: Long, arg2: Long): Long

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
  10. def createRecordReader(split: InputSplit, context: TaskAttemptContext): GdalRecordReader

    Definition Classes
    GdalInputFormat → InputFormat
  11. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. def getBlockIndex(arg0: Array[BlockLocation], arg1: Long): Int

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
  15. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  16. def getFormatMinSplitSize(): Long

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
  17. def getSplits(arg0: JobContext): List[InputSplit]

    Definition Classes
    FileInputFormat → InputFormat
    Annotations
    @throws( classOf[java.io.IOException] )
  18. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  19. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  20. def isSplitable(context: JobContext, fileName: Path): Boolean

    Definition Classes
    GdalInputFormat → FileInputFormat
  21. def listStatus(arg0: JobContext): List[FileStatus]

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
    Annotations
    @throws( classOf[java.io.IOException] )
  22. def makeSplit(arg0: Path, arg1: Long, arg2: Long, arg3: Array[String], arg4: Array[String]): FileSplit

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
  23. def makeSplit(arg0: Path, arg1: Long, arg2: Long, arg3: Array[String]): FileSplit

    Attributes
    protected[org.apache.hadoop.mapreduce.lib.input]
    Definition Classes
    FileInputFormat
  24. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  25. final def notify(): Unit

    Definition Classes
    AnyRef
  26. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  27. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  28. def toString(): String

    Definition Classes
    AnyRef → Any
  29. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  31. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from FileInputFormat[GdalRasterInfo, Tile]

Inherited from InputFormat[GdalRasterInfo, Tile]

Inherited from AnyRef

Inherited from Any

Ungrouped