abstract class InputTransformer[Input, Row] extends InputPreprocessor[Input, Row]
An abstract class InputTransformer that extends InputPreprocessor and defines methods for
transforming input data into rows, including error handling and extracting valid data rows.
This class provides a scaffold for implementing data preprocessing pipelines by utilizing
input-to-row transformations (mapTsToRows) and processing the results to extract valid rows
while disregarding or handling errors (processTriedRows). It facilitates robust and reusable
approaches in data processing workflows, making it easier to define reusable transformation
logic for specific input data types and row structures.
- Input
the type of input elements to be processed, for example, raw data records or strings.
- Row
the type of rows to be produced after transformation, typically structured representations like case classes or tuples.
- Alphabetic
- By Inheritance
- InputTransformer
- InputPreprocessor
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new InputTransformer()
Abstract Value Members
- abstract def processInput(xs: Iterator[Input]): Iterator[Row]
Processes an iterator of input elements, transforming them into an iterator of output rows.
Processes an iterator of input elements, transforming them into an iterator of output rows. This method applies a two-step transformation: first, it maps the input elements into
Try[Row]instances using themapTsToRowsmethod, then it processes theseTry[Row]instances withprocessTriedRowsto produce valid rows or handle errors.- xs
An
Iteratorof input elements of typeInput, representing the data to be transformed.- returns
An
Iteratorof output rows of typeRow, resulting from the transformation process.
- abstract def processTriedRows(rys: Iterator[Try[Row]]): Iterator[Row]
Processes an iterator of
Try[Row]values, extracting successful rows while filtering out failed ones.Processes an iterator of
Try[Row]values, extracting successful rows while filtering out failed ones.The method takes an iterator of
Try[Row]instances, which represent rows that may have been successfully created or could contain errors. The successful rows are extracted and returned as an iterator, while any failures are effectively ignored.- rys
an iterator of
Try[Row]elements where each element is aTryrepresenting the outcome of a row transformation.- returns
an iterator of successfully processed rows (
Row) extracted from the input iterator.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def mapTsToRows(xs: Iterator[Input])(multiline: Boolean)(g: ((Input, Int)) => Try[Row])(implicit ev: Joinable[(Input, Int)]): Iterator[Try[Row]]
Processes an input iterator of type
Inputand maps its elements into rows of typeRowwhile handling potential errors using a specified transformation function.Processes an input iterator of type
Inputand maps its elements into rows of typeRowwhile handling potential errors using a specified transformation function. The function can operate in two modes: - Multiline mode: input data can span multiple lines and will be joined using the providedJoinabletype class. - Single-line mode: each input element is processed independently.- xs
The input iterator containing elements of type
Input.- multiline
A Boolean flag indicating whether to process input in multiline mode.
true: Enables multiline processing using the providedJoinabletype class.false: Processes each input independently.
- g
A transformation function that takes a tuple
(Input, Int), representing an input element and its index, and returns aTry[Row]. The function should encapsulate the logic for mapping input to a row, including error handling.- ev
An implicit instance of the
Joinable[(Input, Int)]type class, which handles joining and validating input tuples during multiline mode.- returns
An iterator of
Try[Row], where each element represents either a successfully processed row or a failure.- In multiline mode, elements may span across multiple lines, joined using the logic provided by the
Joinabletype class. - In single-line mode, each input element is processed individually.
- In multiline mode, elements may span across multiple lines, joined using the logic provided by the
- Definition Classes
- InputPreprocessor
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)