abstract class InputTransformer[Input, Row] extends InputPreprocessor[Input, Row]

An abstract class InputTransformer that extends InputPreprocessor and defines methods for transforming input data into rows, including error handling and extracting valid data rows.

This class provides a scaffold for implementing data preprocessing pipelines by utilizing input-to-row transformations (mapTsToRows) and processing the results to extract valid rows while disregarding or handling errors (processTriedRows). It facilitates robust and reusable approaches in data processing workflows, making it easier to define reusable transformation logic for specific input data types and row structures.

Input

the type of input elements to be processed, for example, raw data records or strings.

Row

the type of rows to be produced after transformation, typically structured representations like case classes or tuples.

Linear Supertypes
InputPreprocessor[Input, Row], AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. InputTransformer
  2. InputPreprocessor
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new InputTransformer()

Abstract Value Members

  1. abstract def processInput(xs: Iterator[Input]): Iterator[Row]

    Processes an iterator of input elements, transforming them into an iterator of output rows.

    Processes an iterator of input elements, transforming them into an iterator of output rows. This method applies a two-step transformation: first, it maps the input elements into Try[Row] instances using the mapTsToRows method, then it processes these Try[Row] instances with processTriedRows to produce valid rows or handle errors.

    xs

    An Iterator of input elements of type Input, representing the data to be transformed.

    returns

    An Iterator of output rows of type Row, resulting from the transformation process.

  2. abstract def processTriedRows(rys: Iterator[Try[Row]]): Iterator[Row]

    Processes an iterator of Try[Row] values, extracting successful rows while filtering out failed ones.

    Processes an iterator of Try[Row] values, extracting successful rows while filtering out failed ones.

    The method takes an iterator of Try[Row] instances, which represent rows that may have been successfully created or could contain errors. The successful rows are extracted and returned as an iterator, while any failures are effectively ignored.

    rys

    an iterator of Try[Row] elements where each element is a Try representing the outcome of a row transformation.

    returns

    an iterator of successfully processed rows (Row) extracted from the input iterator.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  8. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  9. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  11. def mapTsToRows(xs: Iterator[Input])(multiline: Boolean)(g: ((Input, Int)) => Try[Row])(implicit ev: Joinable[(Input, Int)]): Iterator[Try[Row]]

    Processes an input iterator of type Input and maps its elements into rows of type Row while handling potential errors using a specified transformation function.

    Processes an input iterator of type Input and maps its elements into rows of type Row while handling potential errors using a specified transformation function. The function can operate in two modes: - Multiline mode: input data can span multiple lines and will be joined using the provided Joinable type class. - Single-line mode: each input element is processed independently.

    xs

    The input iterator containing elements of type Input.

    multiline

    A Boolean flag indicating whether to process input in multiline mode.

    • true: Enables multiline processing using the provided Joinable type class.
    • false: Processes each input independently.
    g

    A transformation function that takes a tuple (Input, Int), representing an input element and its index, and returns a Try[Row]. The function should encapsulate the logic for mapping input to a row, including error handling.

    ev

    An implicit instance of the Joinable[(Input, Int)] type class, which handles joining and validating input tuples during multiline mode.

    returns

    An iterator of Try[Row], where each element represents either a successfully processed row or a failure.

    • In multiline mode, elements may span across multiple lines, joined using the logic provided by the Joinable type class.
    • In single-line mode, each input element is processed individually.
    Definition Classes
    InputPreprocessor
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  16. def toString(): String
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from InputPreprocessor[Input, Row]

Inherited from AnyRef

Inherited from Any

Ungrouped