com.phasmidsoftware.tableparser.core.table

Members list

Type members

Classlikes

trait Aggregation[X] extends Seq[X] => X

Attributes

Supertypes
trait Seq[X] => X
class Object
trait Matchable
class Any
object Analysis

The Analysis object provides factory methods to analyze tables from various sources.

The Analysis object provides factory methods to analyze tables from various sources.

Example usage:

 // Analyze a CSV-sourced RawTable
 val stats: ColumnStatistics = Analysis(rawTable)

 // Analyze a CSV file by Path
 implicit val parser: RawTableParser = RawTableParser()
 val stats: ColumnStatistics = Analysis.forCsv(Path.of("data.csv"))

 // Analyze a Parquet file (requires parquet module in scope)
 val stats: ColumnStatistics = Analysis.forParquet[YellowTaxiTrip](Path.of("data.parquet"))

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
Analysis.type
trait Analyzer

Unsealed trait for polymorphic analysis of table sources. Implement this trait to add support for new sources (CSV, Parquet, etc.).

Unsealed trait for polymorphic analysis of table sources. Implement this trait to add support for new sources (CSV, Parquet, etc.).

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class CsvAnalyzer
class ParquetAnalyzer[Row]
case class At(n: Int) extends Position

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait Position
class Object
trait Matchable
class Any
Show all
abstract class BaseRow(ws: Seq[String], hdr: Header) extends String => Try[String]

Abstract class to represent a (raw) row from a table.

Abstract class to represent a (raw) row from a table.

Value parameters

hdr

is the Header containing the column names.

ws

the (raw) Strings that make up the row.

Attributes

Supertypes
trait String => Try[String]
class Object
trait Matchable
class Any
Known subtypes
class RawRow
class Row
case class CellSelection(rectangles: Seq[(Span, Span)]) extends CsvFragment

A selection of cells (e.g. "cell=4,1-6,2"), each a rectangle: rows (from, to) and columns (from, to).

A selection of cells (e.g. "cell=4,1-6,2"), each a rectangle: rows (from, to) and columns (from, to).

Value parameters

rectangles

the specifications, each a Span of rows and a Span of columns.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait CsvFragment
class Object
trait Matchable
class Any
Show all
case class CellTransformation[X, Y](f: X => Y) extends Transformation[X, Y]

A case class representing a transformation for individual cell values in a table.

A case class representing a transformation for individual cell values in a table.

CellTransformation acts as a wrapper around a function f that maps values of type X to values of type Y. It extends the Transformation trait, which itself inherits from the Function1 trait, allowing this transformation to be used like a standard Scala function.

Type parameters

X

the input type of the function f.

Y

the output type of the function f.

Value parameters

f

the function that defines the transformation logic from type X to type Y.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait Transformation[X, Y]
trait X => Y
class Object
trait Matchable
class Any
Show all
case class Column(clazz: String, optional: Boolean, maybeStatistics: Option[MaybeStatistics])

A representation of the analysis of a column.

A representation of the analysis of a column.

Value parameters

clazz

a String denoting which class (maybe which variant of class) this column may be represented as.

maybeStatistics

an optional MaybeStatistics (eager or lazy); None if statistics are unavailable or deferred.

optional

if true then this column contains nulls (empty strings).

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Column

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Column.type
case class ColumnSelection(spans: Seq[Span]) extends CsvFragment

A selection of columns (e.g. "col=2-3").

A selection of columns (e.g. "col=2-3").

Value parameters

spans

the specifications.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait CsvFragment
class Object
trait Matchable
class Any
Show all
case class ColumnStatistics(rows: Int, columns: Int, columnMap: Map[String, Column])

Case class to represent the analysis of a table.

Case class to represent the analysis of a table.

Value parameters

columnMap

a map of column names to Column objects (the statistics of a column).

columns

the number of columns.

rows

the number of rows.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Provider trait for computing column statistics from external sources (e.g., Parquet files). Implement this in modules that add support for new sources.

Provider trait for computing column statistics from external sources (e.g., Parquet files). Implement this in modules that add support for new sources.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Content

The Content object provides a factory method for creating instances of the Content class. This utility is designed to facilitate the creation of Content objects from standard iterable collections by converting them into parallel collections for potential parallel processing.

The Content object provides a factory method for creating instances of the Content class. This utility is designed to facilitate the creation of Content objects from standard iterable collections by converting them into parallel collections for potential parallel processing.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Content.type
case class Content[+Row](xs: Vector[Row])

Class to represent the rows of a Table.

Class to represent the rows of a Table.

The rows are kept in a Vector, in order (as of 2.0; in 1.x, they were a ParIterable, in no particular order). For parallel processing, use par. The internal representation may change; thus xs is private.

In general, however, you should probably expect that storing rows inside Content will force an aggregation of all rows. There are alternatives (using InputProcessor) that allow you to process an Iterator[Input] and yield an Iterator[Row].

Type parameters

Row

the underlying Row type.

Value parameters

xs

a Vector[Row].

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
case class CsvAnalyzer(table: RawTable) extends Analyzer

Analyzer for CSV sources. Operates on an in-memory RawTable and walks rows to compute statistics.

Analyzer for CSV sources. Operates on an in-memory RawTable and walks rows to compute statistics.

Value parameters

table

the raw table to analyze.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait Analyzer
class Object
trait Matchable
class Any
Show all
object CsvAttributes

Companion object for the CsvAttributes case class. Provides utility methods and implicit instances for working with CSV attributes.

Companion object for the CsvAttributes case class. Provides utility methods and implicit instances for working with CSV attributes.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
case class CsvAttributes(delimiter: String, quote: String, newline: String = ...)

Case class for CSV attributes, especially for rendering as CSV. The defaults (a comma, a double quote and CRLF) write CSV as defined by RFC 4180 (https://www.rfc-editor.org/rfc/rfc4180).

Case class for CSV attributes, especially for rendering as CSV. The defaults (a comma, a double quote and CRLF) write CSV as defined by RFC 4180 (https://www.rfc-editor.org/rfc/rfc4180).

For the RowConfig which reads tables as these CsvAttributes write them, see rowConfig (and, the other way round, RowConfig.csvAttributes).

Value parameters

delimiter

the delimiter.

newline

the line ending of each record (defaults to CRLF, as in RFC 4180; use "\n" for LF).

quote

the quote character.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object CsvFragment

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
sealed trait CsvFragment

A fragment identifier of a CSV file, as defined by RFC 7111 (https://www.rfc-editor.org/rfc/rfc7111): a selection of rows (e.g. "row=5-7"), of columns (e.g. "col=2") or of cells (e.g. "cell=4,1-6,2"). Several specifications may be separated by ";" (e.g. "row=3;6").

A fragment identifier of a CSV file, as defined by RFC 7111 (https://www.rfc-editor.org/rfc/rfc7111): a selection of rows (e.g. "row=5-7"), of columns (e.g. "col=2") or of cells (e.g. "cell=4,1-6,2"). Several specifications may be separated by ";" (e.g. "row=3;6").

As in the RFC, rows and columns are numbered from 1, and the header line(s) of a CSV file are its first rows (so, for a file with one header line, row 2 is the first row of data). "*" means the last row (or column). A specification which selects backwards (e.g. "row=5-4"), or which is entirely beyond the data, selects nothing; a range which extends beyond the data is cut short.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes

Marker trait for dataset analyzer factories. Implemented by parquet module to enable Analysis.forParquetDataset.

Marker trait for dataset analyzer factories. Implemented by parquet module to enable Analysis.forParquetDataset.

Attributes

Supertypes
class Object
trait Matchable
class Any
trait HasKey[Row]

This typeclass pertains to a table row type T that has one particular column which is considered the primary key (row-id, identifier, whatever).

This typeclass pertains to a table row type T that has one particular column which is considered the primary key (row-id, identifier, whatever).

Type parameters

Row

the underlying type.

Attributes

Supertypes
class Object
trait Matchable
class Any
object HeadedTable

Companion object for HeadedTable. The apply methods provided arbitrarily use Vector as the collection for the rows of the table. CONSIDER using something else such as Array.

Companion object for HeadedTable. The apply methods provided arbitrarily use Vector as the collection for the rows of the table. CONSIDER using something else such as Array.

TESTME ?

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
case class HeadedTable[Row](content: Content[Row], header: Header) extends RenderableTable[Row]

Concrete case class implementing RenderableTable with a Header. The unit and apply methods are such that rows is in fact an Array[Row] (??).

Concrete case class implementing RenderableTable with a Header. The unit and apply methods are such that rows is in fact an Array[Row] (??).

NOTE: the existence or not of a Header in a RenderableTable only affects how the table is rendered. The parsing of a table always has a header of some sort.

Type parameters

Row

the underlying type of each Row

Value parameters

content

the rows of the table, stored as an Array.

header

the header.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class RenderableTable[Row]
trait TableRenderable[Row]
trait Table[Row]
trait Iterable[Row]
trait IterableFactoryDefaults[Row, Iterable]
trait IterableOps[Row, Iterable, Iterable[Row]]
trait IterableOnceOps[Row, Iterable, Iterable[Row]]
trait IterableOnce[Row]
class Object
trait Matchable
class Any
Show all
object Header

This companion object contains utility methods and constants for creating and manipulating instances of the Header class.

This companion object contains utility methods and constants for creating and manipulating instances of the Header class.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Header.type
case class Header(xs: Seq[String], xss: Seq[Seq[String]])

Case class to represent a header.

Case class to represent a header.

A Header is normally read from the first row(s) of a file (see TableParser.headerRowsToRead). Otherwise, a fixed Header can be given to a TableParser (e.g. via setHeader), made in one of these ways:

  • Header.create("a", "b", "c"): the given column names;
  • HeaderX: the names of the fields of the case class X;
  • Header(letters = true, n): n columns named as in a spreadsheet (A B ... Z AA AB ...), or by number (1 2 3 ...);
  • Header(wss): from several rows, the first of which gives the column names.

Value parameters

xs

the sequence of column names.

xss

a sequence of sequence of String representing any additional header lines.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Indexed

Companion object for the Indexed case class, providing utility methods to create and work with indexed collections.

Companion object for the Indexed case class, providing utility methods to create and work with indexed collections.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Indexed.type
case class Indexed[T](i: Int, t: T)

A wrapper class to index a T.

A wrapper class to index a T.

Type parameters

T

the underlying type.

Value parameters

i

the index (ordinal value).

t

the instance of T.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
case object Last extends Position

Attributes

Supertypes
trait Singleton
trait Product
trait Mirror
trait Serializable
trait Product
trait Equals
trait Position
class Object
trait Matchable
class Any
Show all
Self type
Last.type

Sealed trait representing statistics that may be eager or lazy. Eager statistics are computed upfront (from Parquet metadata). Lazy statistics are deferred as a thunk (computed on demand).

Sealed trait representing statistics that may be eager or lazy. Eager statistics are computed upfront (from Parquet metadata). Lazy statistics are deferred as a thunk (computed on demand).

Attributes

Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
sealed trait Position

A position in a CSV fragment: a row or column number (from 1), or the last one ("*").

A position in a CSV fragment: a row or column number (from 1), or the last one ("*").

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class At
object Last
case class RawRow(ws: Seq[String], header: Header) extends BaseRow

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class BaseRow
trait String => Try[String]
class Object
trait Matchable
class Any
Show all
case class RawRowProjection(columns: Seq[Int]) extends Transformation[RawRow, RawRow]

A case class RawRowProjection that transforms a RawRow into another RawRow by selecting specific columns.

A case class RawRowProjection that transforms a RawRow into another RawRow by selecting specific columns.

This transformation operates based on the specified indices of the columns to be kept. It selects the raw string values (ws) of the input RawRow corresponding to the provided indices and produces a new RawRow with the filtered data, while retaining the header from the input.

Value parameters

columns

A sequence of integers representing column indices to retain in the transformation.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait RawRow => RawRow
class Object
trait Matchable
class Any
Show all
case class RawRowTransformation(transformers: Map[Int, Transformation[String, String]]) extends Transformation[RawRow, RawRow]

Represents a transformation that can be applied to a RawRow, transforming its individual string elements based on a collection of column-specific transformation functions.

Represents a transformation that can be applied to a RawRow, transforming its individual string elements based on a collection of column-specific transformation functions.

This class is designed to process rows of data by applying transformations to specific columns, defined by their zero-based index. Any column index not explicitly configured with a transformation will default to an identity transformation, leaving the original value unchanged.

Value parameters

transformers

A map where the key is the column index (0-based) and the value is a Transformation[String, String] to be applied to the corresponding column's data. If a column index is not present in this map, an identity transformation is applied by default.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait RawRow => RawRow
class Object
trait Matchable
class Any
Show all
case class RawTableAggregation(aggregators: Map[String, Transformation[String, String]]) extends Transformation[RawTable, RawTable]

This case class represents a transformation that performs column-wise aggregation on a RawTable. Each aggregator specifies a mapping between a column name and its corresponding transformation function.

This case class represents a transformation that performs column-wise aggregation on a RawTable. Each aggregator specifies a mapping between a column name and its corresponding transformation function.

Value parameters

aggregators

A map where keys are column names (as String) and values are transformations (subclasses of Transformation) applied to the data in those columns. The transformation process is string-based (Transformation[String, String]), meaning that it takes string input from a column and produces a transformed string output.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait RawTable => RawTable
class Object
trait Matchable
class Any
Show all
case class RawTableProjection(columns: Seq[String]) extends Transformation[RawTable, RawTable]

A transformation that projects a subset of columns from a RawTable based on the specified column names.

A transformation that projects a subset of columns from a RawTable based on the specified column names.

The RawTableProjection class allows users to specify column names, and it produces a new RawTable containing only these columns. Internally, it determines the indices of the specified columns and applies the projection at the row level by retaining only the corresponding values.

Value parameters

columns

A sequence of column names to include in the resulting table.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait RawTable => RawTable
class Object
trait Matchable
class Any
Show all
case class RawTableTransformation(transformers: Map[String, Transformation[String, String]]) extends Transformation[RawTable, RawTable]

A transformation class that applies a specific set of transformations to the rows of a RawTable based on matching column names.

A transformation class that applies a specific set of transformations to the rows of a RawTable based on matching column names.

Value parameters

transformers

A map where the keys are column names (as String) and the values are Transformation[String, String] instances, which define how to transform data in the corresponding columns. This class implements the Transformation[RawTable, RawTable] trait, meaning it takes a RawTable as input and produces a transformed RawTable as output. The transformation process is based on matching column names from the transformers map with the column names in the table's header. If a match is found, the corresponding transformation is applied to the relevant column in each row of the table. ==Example Usage== To use this class, first construct a RawTableTransformation instance by providing a map of column names and their corresponding transformations:

val transformers: Map[String, Transformation[String, String]] = Map(
 "column1" -> CustomTransformation1,
 "column2" -> CustomTransformation2
)
val transformation = RawTableTransformation(transformers)
// Apply the transformation to a RawTable object
val transformedTable = transformation(rawTable)
               ==Implementation Details==
  • The apply method retrieves the table's maybeHeader object, which contains the column names.
  • For each column name in the transformers map, it fetches the column index using Header.getIndex.
  • The transformations are then compiled into a map associating column indices with their respective transformations.
  • Finally, the table's rows are transformed using the map method, which applies a RawRowTransformation to each row.

Attributes

See also
Supertypes
trait Serializable
trait Product
trait Equals
trait RawTable => RawTable
class Object
trait Matchable
class Any
Show all
abstract class RenderableTable[Row](rows: Content[Row], val maybeHeader: Option[Header]) extends Table[Row], TableRenderable[Row]

CONSIDER eliminating this base class or, perhaps, rename it.

CONSIDER eliminating this base class or, perhaps, rename it.

Type parameters

Row

the underlying type of each Row

Value parameters

maybeHeader

(optional) header

rows

the rows of the table

Attributes

Supertypes
trait TableRenderable[Row]
trait Table[Row]
trait Iterable[Row]
trait IterableFactoryDefaults[Row, Iterable]
trait IterableOps[Row, Iterable, Iterable[Row]]
trait IterableOnceOps[Row, Iterable, Iterable[Row]]
trait IterableOnce[Row]
class Object
trait Matchable
class Any
Show all
Known subtypes
class HeadedTable[Row]
class UnheadedTable[Row]
Self type
object Row

Companion object for the Row class, providing utilities for working with rows, including CSV rendering and generation.

Companion object for the Row class, providing utilities for working with rows, including CSV rendering and generation.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Row.type
case class Row(ws: Seq[String], hdr: Header, index: Int) extends BaseRow

Case class to represent a (raw) row from a table.

Case class to represent a (raw) row from a table.

Value parameters

hdr

is the Header containing the column names.

ws

the (raw) Strings that make up the row.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class BaseRow
trait String => Try[String]
class Object
trait Matchable
class Any
Show all
case class RowSelection(spans: Seq[Span]) extends CsvFragment

A selection of rows (e.g. "row=5-7;10").

A selection of rows (e.g. "row=5-7;10").

Value parameters

spans

the specifications.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
trait CsvFragment
class Object
trait Matchable
class Any
Show all
object RowSelection

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type

Marker trait for single-file analyzer factories. Implemented by parquet module to enable Analysis.forParquet.

Marker trait for single-file analyzer factories. Implemented by parquet module to enable Analysis.forParquet.

Attributes

Supertypes
class Object
trait Matchable
class Any
case class Span(from: Position, until: Position)

A range of positions: from (the first position wanted) up to, but not including, until (the first position not wanted), as for Scala's Ranges (e.g. 5 until 8). So Span(At(5), At(8)) is the positions 5, 6 and 7 (as "5-7" in RFC 7111); a Span whose until is not after its from is empty. As until, Last means after the last position (so Span(At(5), Last) is from 5 to the end, as "5-*").

A range of positions: from (the first position wanted) up to, but not including, until (the first position not wanted), as for Scala's Ranges (e.g. 5 until 8). So Span(At(5), At(8)) is the positions 5, 6 and 7 (as "5-7" in RFC 7111); a Span whose until is not after its from is empty. As until, Last means after the last position (so Span(At(5), Last) is from 5 to the end, as "5-*").

Value parameters

from

the first position wanted.

until

the first position not wanted.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Span

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Span.type
case class Statistics(mu: Double, sigma: Double, min: Double, max: Double)

Class to represent the statistics of a column.

Class to represent the statistics of a column.

Value parameters

max

the largest value.

min

the smallest value.

mu

the mean value.

sigma

the standard deviation.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Statistics

Companion object for the Statistics case class. Provides methods to create a Statistics instance from a sequence of doubles.

Companion object for the Statistics case class. Provides methods to create a Statistics instance from a sequence of doubles.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Statistics.type
object Table

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Table.type
trait Table[Row] extends Iterable[Row]

A Table of Content.

A Table of Content.

Type parameters

Row

the type of each row.

Attributes

Companion
object
Supertypes
trait Iterable[Row]
trait IterableFactoryDefaults[Row, Iterable]
trait IterableOps[Row, Iterable, Iterable[Row]]
trait IterableOnceOps[Row, Iterable, Iterable[Row]]
trait IterableOnce[Row]
class Object
trait Matchable
class Any
Show all
Known subtypes
class RenderableTable[Row]
class HeadedTable[Row]
class UnheadedTable[Row]
case class TableException(w: String) extends Exception

Table Exception.

Table Exception.

Value parameters

w

the message.

Attributes

Supertypes
trait Product
trait Equals
class Exception
class Throwable
trait Serializable
class Object
trait Matchable
class Any
Show all
trait Transformation[X, Y] extends X => Y

Trait which transforms an X into a Y. Basically, this is just a sub-class of Function1.

Trait which transforms an X into a Y. Basically, this is just a sub-class of Function1.

Type parameters

X

the input type.

Y

the output type.

Attributes

Supertypes
trait X => Y
class Object
trait Matchable
class Any
Known subtypes
case class UnheadedTable[Row](content: Content[Row]) extends RenderableTable[Row]

Concrete case class implementing RenderableTable without a Header.

Concrete case class implementing RenderableTable without a Header.

NOTE: the existence or not of a Header in a RenderableTable only affects how the table is rendered. The parsing of a table always has a header of some sort.

TESTME this: it is currently not used.

Type parameters

Row

the underlying type of each Row

Value parameters

content

the rows of the table.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class RenderableTable[Row]
trait TableRenderable[Row]
trait Table[Row]
trait Iterable[Row]
trait IterableFactoryDefaults[Row, Iterable]
trait IterableOps[Row, Iterable, Iterable[Row]]
trait IterableOnceOps[Row, Iterable, Iterable[Row]]
trait IterableOnce[Row]
class Object
trait Matchable
class Any
Show all
object UnheadedTable

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
trait Validity[T]

Attributes

Supertypes
class Object
trait Matchable
class Any

Types

Exports

Defined exports

Exported from MaybeStatistics$
Exported from MaybeStatistics$