com.phasmidsoftware.tableparser.core.table
Members list
Type members
Classlikes
Attributes
- Supertypes
-
trait Seq[X] => Xclass Objecttrait Matchableclass Any
The Analysis object provides factory methods to analyze tables from various sources.
The Analysis object provides factory methods to analyze tables from various sources.
Example usage:
// Analyze a CSV-sourced RawTable
val stats: ColumnStatistics = Analysis(rawTable)
// Analyze a CSV file by Path
implicit val parser: RawTableParser = RawTableParser()
val stats: ColumnStatistics = Analysis.forCsv(Path.of("data.csv"))
// Analyze a Parquet file (requires parquet module in scope)
val stats: ColumnStatistics = Analysis.forParquet[YellowTaxiTrip](Path.of("data.parquet"))
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
Analysis.type
Unsealed trait for polymorphic analysis of table sources. Implement this trait to add support for new sources (CSV, Parquet, etc.).
Unsealed trait for polymorphic analysis of table sources. Implement this trait to add support for new sources (CSV, Parquet, etc.).
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait Positionclass Objecttrait Matchableclass AnyShow all
Abstract class to represent a (raw) row from a table.
A selection of cells (e.g. "cell=4,1-6,2"), each a rectangle: rows (from, to) and columns (from, to).
A selection of cells (e.g. "cell=4,1-6,2"), each a rectangle: rows (from, to) and columns (from, to).
Value parameters
- rectangles
-
the specifications, each a Span of rows and a Span of columns.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait CsvFragmentclass Objecttrait Matchableclass AnyShow all
A case class representing a transformation for individual cell values in a table.
A case class representing a transformation for individual cell values in a table.
CellTransformation acts as a wrapper around a function f that maps values of type X to values of type Y. It extends the Transformation trait, which itself inherits from the Function1 trait, allowing this transformation to be used like a standard Scala function.
Type parameters
- X
-
the input type of the function
f. - Y
-
the output type of the function
f.
Value parameters
- f
-
the function that defines the transformation logic from type
Xto typeY.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait X => Yclass Objecttrait Matchableclass AnyShow all
A representation of the analysis of a column.
A representation of the analysis of a column.
Value parameters
- clazz
-
a String denoting which class (maybe which variant of class) this column may be represented as.
- maybeStatistics
-
an optional MaybeStatistics (eager or lazy); None if statistics are unavailable or deferred.
- optional
-
if true then this column contains nulls (empty strings).
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
A selection of columns (e.g. "col=2-3").
A selection of columns (e.g. "col=2-3").
Value parameters
- spans
-
the specifications.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait CsvFragmentclass Objecttrait Matchableclass AnyShow all
Case class to represent the analysis of a table.
Case class to represent the analysis of a table.
Value parameters
- columnMap
-
a map of column names to Column objects (the statistics of a column).
- columns
-
the number of columns.
- rows
-
the number of rows.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Provider trait for computing column statistics from external sources (e.g., Parquet files). Implement this in modules that add support for new sources.
Provider trait for computing column statistics from external sources (e.g., Parquet files). Implement this in modules that add support for new sources.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
The Content object provides a factory method for creating instances of the Content class. This utility is designed to facilitate the creation of Content objects from standard iterable collections by converting them into parallel collections for potential parallel processing.
The Content object provides a factory method for creating instances of the Content class. This utility is designed to facilitate the creation of Content objects from standard iterable collections by converting them into parallel collections for potential parallel processing.
Attributes
Class to represent the rows of a Table.
Class to represent the rows of a Table.
The rows are kept in a Vector, in order (as of 2.0; in 1.x, they were a ParIterable, in no particular order). For parallel processing, use par. The internal representation may change; thus xs is private.
In general, however, you should probably expect that storing rows inside Content will force an aggregation of all rows. There are alternatives (using InputProcessor) that allow you to process an Iterator[Input] and yield an Iterator[Row].
Type parameters
- Row
-
the underlying Row type.
Value parameters
- xs
-
a Vector[Row].
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Analyzer for CSV sources. Operates on an in-memory RawTable and walks rows to compute statistics.
Analyzer for CSV sources. Operates on an in-memory RawTable and walks rows to compute statistics.
Value parameters
- table
-
the raw table to analyze.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait Analyzerclass Objecttrait Matchableclass AnyShow all
Companion object for the CsvAttributes case class. Provides utility methods and implicit instances for working with CSV attributes.
Companion object for the CsvAttributes case class. Provides utility methods and implicit instances for working with CSV attributes.
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
CsvAttributes.type
Case class for CSV attributes, especially for rendering as CSV. The defaults (a comma, a double quote and CRLF) write CSV as defined by RFC 4180 (https://www.rfc-editor.org/rfc/rfc4180).
Case class for CSV attributes, especially for rendering as CSV. The defaults (a comma, a double quote and CRLF) write CSV as defined by RFC 4180 (https://www.rfc-editor.org/rfc/rfc4180).
For the RowConfig which reads tables as these CsvAttributes write them, see rowConfig (and, the other way round, RowConfig.csvAttributes).
Value parameters
- delimiter
-
the delimiter.
- newline
-
the line ending of each record (defaults to CRLF, as in RFC 4180; use "\n" for LF).
- quote
-
the quote character.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- trait
- Supertypes
-
trait Sumtrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
CsvFragment.type
A fragment identifier of a CSV file, as defined by RFC 7111 (https://www.rfc-editor.org/rfc/rfc7111): a selection of rows (e.g. "row=5-7"), of columns (e.g. "col=2") or of cells (e.g. "cell=4,1-6,2"). Several specifications may be separated by ";" (e.g. "row=3;6").
A fragment identifier of a CSV file, as defined by RFC 7111 (https://www.rfc-editor.org/rfc/rfc7111): a selection of rows (e.g. "row=5-7"), of columns (e.g. "col=2") or of cells (e.g. "cell=4,1-6,2"). Several specifications may be separated by ";" (e.g. "row=3;6").
As in the RFC, rows and columns are numbered from 1, and the header line(s) of a CSV file are its first rows (so, for a file with one header line, row 2 is the first row of data). "*" means the last row (or column). A specification which selects backwards (e.g. "row=5-4"), or which is entirely beyond the data, selects nothing; a range which extends beyond the data is cut short.
Attributes
- Companion
- object
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
Marker trait for dataset analyzer factories. Implemented by parquet module to enable Analysis.forParquetDataset.
Marker trait for dataset analyzer factories. Implemented by parquet module to enable Analysis.forParquetDataset.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
This typeclass pertains to a table row type T that has one particular column which is considered the primary key (row-id, identifier, whatever).
This typeclass pertains to a table row type T that has one particular column which is considered the primary key (row-id, identifier, whatever).
Type parameters
- Row
-
the underlying type.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
Companion object for HeadedTable. The apply methods provided arbitrarily use Vector as the collection for the rows of the table. CONSIDER using something else such as Array.
Companion object for HeadedTable. The apply methods provided arbitrarily use Vector as the collection for the rows of the table. CONSIDER using something else such as Array.
TESTME ?
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
HeadedTable.type
Concrete case class implementing RenderableTable with a Header. The unit and apply methods are such that rows is in fact an Array[Row] (??).
Concrete case class implementing RenderableTable with a Header. The unit and apply methods are such that rows is in fact an Array[Row] (??).
NOTE: the existence or not of a Header in a RenderableTable only affects how the table is rendered. The parsing of a table always has a header of some sort.
Type parameters
- Row
-
the underlying type of each Row
Value parameters
- content
-
the rows of the table, stored as an Array.
- header
-
the header.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass RenderableTable[Row]trait TableRenderable[Row]trait Table[Row]trait Iterable[Row]trait IterableFactoryDefaults[Row, Iterable]trait IterableOps[Row, Iterable, Iterable[Row]]trait IterableOnceOps[Row, Iterable, Iterable[Row]]trait IterableOnce[Row]class Objecttrait Matchableclass AnyShow all
This companion object contains utility methods and constants for creating and manipulating instances of the Header class.
Case class to represent a header.
Case class to represent a header.
A Header is normally read from the first row(s) of a file (see TableParser.headerRowsToRead). Otherwise, a fixed Header can be given to a TableParser (e.g. via setHeader), made in one of these ways:
- Header.create("a", "b", "c"): the given column names;
- HeaderX: the names of the fields of the case class X;
- Header(letters = true, n): n columns named as in a spreadsheet (A B ... Z AA AB ...), or by number (1 2 3 ...);
- Header(wss): from several rows, the first of which gives the column names.
Value parameters
- xs
-
the sequence of column names.
- xss
-
a sequence of sequence of String representing any additional header lines.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Companion object for the Indexed case class, providing utility methods to create and work with indexed collections.
A wrapper class to index a T.
A wrapper class to index a T.
Type parameters
- T
-
the underlying type.
Value parameters
- i
-
the index (ordinal value).
- t
-
the instance of T.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Sealed trait representing statistics that may be eager or lazy. Eager statistics are computed upfront (from Parquet metadata). Lazy statistics are deferred as a thunk (computed on demand).
Sealed trait representing statistics that may be eager or lazy. Eager statistics are computed upfront (from Parquet metadata). Lazy statistics are deferred as a thunk (computed on demand).
Attributes
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass BaseRowtrait String => Try[String]class Objecttrait Matchableclass AnyShow all
A case class RawRowProjection that transforms a RawRow into another RawRow by selecting specific columns.
A case class RawRowProjection that transforms a RawRow into another RawRow by selecting specific columns.
This transformation operates based on the specified indices of the columns to be kept. It selects the raw string values (ws) of the input RawRow corresponding to the provided indices and produces a new RawRow with the filtered data, while retaining the header from the input.
Value parameters
- columns
-
A sequence of integers representing column indices to retain in the transformation.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Represents a transformation that can be applied to a RawRow, transforming its individual string elements based on a collection of column-specific transformation functions.
Represents a transformation that can be applied to a RawRow, transforming its individual string elements based on a collection of column-specific transformation functions.
This class is designed to process rows of data by applying transformations to specific columns, defined by their zero-based index. Any column index not explicitly configured with a transformation will default to an identity transformation, leaving the original value unchanged.
Value parameters
- transformers
-
A map where the key is the column index (0-based) and the value is a
Transformation[String, String]to be applied to the corresponding column's data. If a column index is not present in this map, an identity transformation is applied by default.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
This case class represents a transformation that performs column-wise aggregation on a RawTable. Each aggregator specifies a mapping between a column name and its corresponding transformation function.
This case class represents a transformation that performs column-wise aggregation on a RawTable. Each aggregator specifies a mapping between a column name and its corresponding transformation function.
Value parameters
- aggregators
-
A map where keys are column names (as
String) and values are transformations (subclasses ofTransformation) applied to the data in those columns. The transformation process is string-based (Transformation[String, String]), meaning that it takes string input from a column and produces a transformed string output.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
A transformation that projects a subset of columns from a RawTable based on the specified column names.
A transformation that projects a subset of columns from a RawTable based on the specified column names.
The RawTableProjection class allows users to specify column names, and it produces a new RawTable containing only these columns. Internally, it determines the indices of the specified columns and applies the projection at the row level by retaining only the corresponding values.
Value parameters
- columns
-
A sequence of column names to include in the resulting table.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
A transformation class that applies a specific set of transformations to the rows of a RawTable based on matching column names.
A transformation class that applies a specific set of transformations to the rows of a RawTable based on matching column names.
Value parameters
- transformers
-
A map where the keys are column names (as
String) and the values areTransformation[String, String]instances, which define how to transform data in the corresponding columns. This class implements theTransformation[RawTable, RawTable]trait, meaning it takes aRawTableas input and produces a transformedRawTableas output. The transformation process is based on matching column names from thetransformersmap with the column names in the table's header. If a match is found, the corresponding transformation is applied to the relevant column in each row of the table. ==Example Usage== To use this class, first construct aRawTableTransformationinstance by providing a map of column names and their corresponding transformations:val transformers: Map[String, Transformation[String, String]] = Map( "column1" -> CustomTransformation1, "column2" -> CustomTransformation2 ) val transformation = RawTableTransformation(transformers) // Apply the transformation to a RawTable object val transformedTable = transformation(rawTable)==Implementation Details==- The
applymethod retrieves the table'smaybeHeaderobject, which contains the column names. - For each column name in the
transformersmap, it fetches the column index usingHeader.getIndex. - The transformations are then compiled into a map associating column indices with their respective transformations.
- Finally, the table's rows are transformed using the
mapmethod, which applies aRawRowTransformationto each row.
- The
Attributes
- See also
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
CONSIDER eliminating this base class or, perhaps, rename it.
CONSIDER eliminating this base class or, perhaps, rename it.
Type parameters
- Row
-
the underlying type of each Row
Value parameters
- maybeHeader
-
(optional) header
- rows
-
the rows of the table
Attributes
- Supertypes
-
trait TableRenderable[Row]trait Table[Row]trait Iterable[Row]trait IterableFactoryDefaults[Row, Iterable]trait IterableOps[Row, Iterable, Iterable[Row]]trait IterableOnceOps[Row, Iterable, Iterable[Row]]trait IterableOnce[Row]class Objecttrait Matchableclass AnyShow all
- Known subtypes
-
class HeadedTable[Row]class UnheadedTable[Row]
- Self type
-
RenderableTable[Row]
Companion object for the Row class, providing utilities for working with rows, including CSV rendering and generation.
Case class to represent a (raw) row from a table.
Case class to represent a (raw) row from a table.
Value parameters
- hdr
-
is the Header containing the column names.
- ws
-
the (raw) Strings that make up the row.
Attributes
A selection of rows (e.g. "row=5-7;10").
A selection of rows (e.g. "row=5-7;10").
Value parameters
- spans
-
the specifications.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait CsvFragmentclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
RowSelection.type
Marker trait for single-file analyzer factories. Implemented by parquet module to enable Analysis.forParquet.
Marker trait for single-file analyzer factories. Implemented by parquet module to enable Analysis.forParquet.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
A range of positions: from (the first position wanted) up to, but not including, until (the first position not wanted), as for Scala's Ranges (e.g. 5 until 8). So Span(At(5), At(8)) is the positions 5, 6 and 7 (as "5-7" in RFC 7111); a Span whose until is not after its from is empty. As until, Last means after the last position (so Span(At(5), Last) is from 5 to the end, as "5-*").
A range of positions: from (the first position wanted) up to, but not including, until (the first position not wanted), as for Scala's Ranges (e.g. 5 until 8). So Span(At(5), At(8)) is the positions 5, 6 and 7 (as "5-7" in RFC 7111); a Span whose until is not after its from is empty. As until, Last means after the last position (so Span(At(5), Last) is from 5 to the end, as "5-*").
Value parameters
- from
-
the first position wanted.
- until
-
the first position not wanted.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Class to represent the statistics of a column.
Class to represent the statistics of a column.
Value parameters
- max
-
the largest value.
- min
-
the smallest value.
- mu
-
the mean value.
- sigma
-
the standard deviation.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Companion object for the Statistics case class. Provides methods to create a Statistics instance from a sequence of doubles.
Companion object for the Statistics case class. Provides methods to create a Statistics instance from a sequence of doubles.
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
Statistics.type
A Table of Content.
A Table of Content.
Type parameters
- Row
-
the type of each row.
Attributes
- Companion
- object
- Supertypes
-
trait Iterable[Row]trait IterableFactoryDefaults[Row, Iterable]trait IterableOps[Row, Iterable, Iterable[Row]]trait IterableOnceOps[Row, Iterable, Iterable[Row]]trait IterableOnce[Row]class Objecttrait Matchableclass AnyShow all
- Known subtypes
Table Exception.
Table Exception.
Value parameters
- w
-
the message.
Attributes
- Supertypes
-
trait Producttrait Equalsclass Exceptionclass Throwabletrait Serializableclass Objecttrait Matchableclass AnyShow all
Trait which transforms an X into a Y. Basically, this is just a sub-class of Function1.
Trait which transforms an X into a Y. Basically, this is just a sub-class of Function1.
Type parameters
- X
-
the input type.
- Y
-
the output type.
Attributes
- Supertypes
-
trait X => Yclass Objecttrait Matchableclass Any
- Known subtypes
-
class RawRowProjectionclass RawRowTransformationclass RawTableAggregationclass RawTableProjectionclass RawTableTransformationShow all
Concrete case class implementing RenderableTable without a Header.
Concrete case class implementing RenderableTable without a Header.
NOTE: the existence or not of a Header in a RenderableTable only affects how the table is rendered. The parsing of a table always has a header of some sort.
TESTME this: it is currently not used.
Type parameters
- Row
-
the underlying type of each Row
Value parameters
- content
-
the rows of the table.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass RenderableTable[Row]trait TableRenderable[Row]trait Table[Row]trait Iterable[Row]trait IterableFactoryDefaults[Row, Iterable]trait IterableOps[Row, Iterable, Iterable[Row]]trait IterableOnceOps[Row, Iterable, Iterable[Row]]trait IterableOnce[Row]class Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
UnheadedTable.type
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any