The IndexedInputToRowsAggregator class provides functionality to process and aggregate input data of type Input into structured rows of type Row, with support for error handling, validation, and conditional processing. It extends the abstract class InputAggregator.
The class leverages implicit Joinable behavior for Input and works with Try to handle potential failures during the transformation of input data into rows. The flexibility is further increased with options to apply a custom predicate to filter rows, enable forgiving behavior (ignoring failures), and process multi-line inputs.
Type parameters
Input
the type of input elements to be processed.
Row
the type of rows transformed from the input.
Value parameters
f
a function that takes a tuple of (Input, Int) and returns a Try[Row], used to transform each input element and its index into a Row.
forgiving
a flag indicating whether forgiving behavior is enabled (i.e., failures are logged and ignored rather than propagated).
multiline
a flag indicating whether multi-line input processing is enabled.
predicate
a function that takes a Try[Row] and returns a boolean, used to filter processed rows based on custom validation logic.
Implicit object that provides a JoinableTInt instance for the type Input.
Implicit object that provides a JoinableTInt instance for the type Input.
This object extends JoinableTInt for the type parameter Input and provides an implicit Joinable instance for Input. It defines the joining behavior for tuples of type (Input, Int) while utilizing the underlying Joinable instance for Input.
Processes an iterator of inputs and transforms them into a wrapped iterator of rows.
Processes an iterator of inputs and transforms them into a wrapped iterator of rows.
This method combines the transformation of inputs into Try[Row] through mapTsToRows and the subsequent processing of these tried rows through processTriedRows.
Value parameters
xs
an iterator of inputs of type Input to be processed.
Attributes
Returns
a Wrapper containing an iterator of rows of type Row, where the transformation and processing operations have been applied to the inputs.
Processes an iterator of Try wrapped rows, transforming it into a single Try that contains an iterator of successfully processed rows.
Processes an iterator of Try wrapped rows, transforming it into a single Try that contains an iterator of successfully processed rows.
NOTE this is not a simple iterator to iterator pipeline. We have to sequence the input to get the output.
Value parameters
rys
an iterator of Try[Row], where each Try represents the result of processing a single row which may either succeed or fail
Attributes
Returns
a Try[Iterator[Row]] that encapsulates either: - a success containing an iterator of all successfully processed rows, or - a failure if any of the individual rows failed during processing
Processes an input iterator of type Input and maps its elements into rows of type Row while handling potential errors using a specified transformation function. The function can operate in two modes:
Processes an input iterator of type Input and maps its elements into rows of type Row while handling potential errors using a specified transformation function. The function can operate in two modes:
Multiline mode: input data can span multiple lines and will be joined using the provided Joinable type class.
Single-line mode: each input element is processed independently.
Value parameters
ev
An implicit instance of the Joinable[(Input, Int)] type class, which handles joining and validating input tuples during multiline mode.
g
A transformation function that takes a tuple (Input, Int), representing an input element and its index, and returns a Try[Row]. The function should encapsulate the logic for mapping input to a row, including error handling.
multiline
A Boolean flag indicating whether to process input in multiline mode. - true: Enables multiline processing using the provided Joinable type class. - false: Processes each input independently.
xs
The input iterator containing elements of type Input.
Attributes
Returns
An iterator of Try[Row], where each element represents either a successfully processed row or a failure. - In multiline mode, elements may span across multiple lines, joined using the logic provided by the Joinable type class. - In single-line mode, each input element is processed individually.
Implicit object that provides a JoinableTInt instance for the type Input.
Implicit object that provides a JoinableTInt instance for the type Input.
This object extends JoinableTInt for the type parameter Input and provides an implicit Joinable instance for Input. It defines the joining behavior for tuples of type (Input, Int) while utilizing the underlying Joinable instance for Input.