Represents a singleton object that provides definitions or metadata for data-related functionalities.
This object is designed to encapsulate the structure and behavior associated with defining, managing, or interpreting data schemas or specifications.
It can be used to centralize data definitions, facilitate consistency across data-related processes, and support extensible data handling operations.
Attributes
- Companion
- trait
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
DataDefinition.type
Members list
Type members
Classlikes
A trait that defines a monoid for iterable collections of type T.
A trait that defines a monoid for iterable collections of type T.
Extends the Monoid trait, adding functionality for combining and initializing iterable collections. Provides a default implementation for the identity element (zero) and the associative binary operation (combine).
Type parameters
- T
-
the type of elements within the iterable collection.
Attributes
- Supertypes
Value members
Concrete methods
Creates a lazy DataDefinition from a sequence of key-value pairs, partitioning the data into the specified number of partitions. The provided sequence can contain duplicate keys, and the values associated with duplicate keys will be combined using the Monoid instance for the value type.
Creates a lazy DataDefinition from a sequence of key-value pairs, partitioning the data into the specified number of partitions. The provided sequence can contain duplicate keys, and the values associated with duplicate keys will be combined using the Monoid instance for the value type.
Type parameters
- K
-
The type of the keys in the key-value pairs.
- V
-
The type of the values in the key-value pairs. The values must form a monoid.
Value parameters
- kVs
-
A sequence of key-value pairs to define the initial data.
- partitions
-
The number of partitions to split the data into. Each partition will contain roughly an equal share of the data.
Attributes
- Returns
-
A
DataDefinitionrepresenting the lazily computed data with the specified partitions.
Creates a new DataDefinition from a sequence of key-value pairs, where the values conform to the Monoid typeclass, enabling combination and aggregation operations.
Creates a new DataDefinition from a sequence of key-value pairs, where the values conform to the Monoid typeclass, enabling combination and aggregation operations.
Type parameters
- K
-
the type of the keys in the sequence
- V
-
the type of the values in the sequence, which must have a
Monoidimplementation
Value parameters
- kVs
-
the sequence of key-value pairs to initialize the
DataDefinition
Attributes
- Returns
-
a lazily evaluated
DataDefinitionbased on the provided key-value sequence
Partitions the given map into a distributed DataDefinition with a specified number of partitions. The values in the map must adhere to the Monoid typeclass, allowing for combination and aggregation operations.
Partitions the given map into a distributed DataDefinition with a specified number of partitions. The values in the map must adhere to the Monoid typeclass, allowing for combination and aggregation operations.
Type parameters
- K
-
the type of the keys in the map
- V
-
the type of the values in the map, which must have a
Monoidimplementation
Value parameters
- kVm
-
the map containing key-value pairs to be partitioned
- partitions
-
the number of partitions for the resulting
DataDefinition
Attributes
- Returns
-
a lazily evaluated
DataDefinitionpartitioned into the specified number of parts
Creates a DataDefinition from a map of key-value pairs, where the value type has a Monoid defined.
Creates a DataDefinition from a map of key-value pairs, where the value type has a Monoid defined.
Type parameters
- K
-
the type of the keys in the map.
- V
-
the type of the values in the map, which must have a
Monoidinstance.
Value parameters
- kVm
-
a map containing key-value pairs, where the value type has an associated
Monoid.
Attributes
- Returns
-
a
DataDefinitioncontaining the provided key-value pairs.
Method to create a LazyDD from a Seq[V], an function V => K, and a specific number of partitions.
Method to create a LazyDD from a Seq[V], an function V => K, and a specific number of partitions.
Type parameters
- K
-
the key type.
- V
-
the underlying type of vs.
Value parameters
- f
-
the key generation function.
- partitions
-
the desired number of partitions.
- vs
-
a Seq[V].
Attributes
- Returns
-
a LazyDD.
The default (JVM-wide) DDContext: see DDContext.default.
The default (JVM-wide) DDContext: see DDContext.default.
Attributes
Method to create a LazyDD from a sequence of V (no keys) and using the default partitions value.
Method to create a LazyDD from a sequence of V (no keys) and using the default partitions value.
Type parameters
- K
-
the key type.
- V
-
the underlying type of vs.
Value parameters
- f
-
the key generation function.
- vs
-
a Seq[V].
Attributes
- Returns
-
a LazyDD.
Transforms a sequence of values into a sequence of tuples where the first element is a unit value and the second element is derived from the input sequence.
Transforms a sequence of values into a sequence of tuples where the first element is a unit value and the second element is derived from the input sequence.
Value parameters
- vs
-
A sequence of values of type V. The type V must have an implicit Monoid instance.
Attributes
- Returns
-
A sequence of tuples where each tuple contains a unit value as the first element and an element from the input sequence as the second element.
Terminates the ActorSystem backing this object's shared DDContext.
Terminates the ActorSystem backing this object's shared DDContext.
DataDefinition's actor context is a JVM-wide singleton, created lazily the first time this object is touched, and there was previously no way to explicitly tear it down. Any short-lived process (or test/benchmark) which embeds DataDefinition should call this once it's done, so the underlying ActorSystem doesn't keep the JVM alive indefinitely.
Attributes
- Returns
-
Unit
This lift method is used to lift a V=>W into a (K,V)=>(K,W) and is used in those situations where only the values of a key-value pair are to be transformed by the DataDefinition map method.
This lift method is used to lift a V=>W into a (K,V)=>(K,W) and is used in those situations where only the values of a key-value pair are to be transformed by the DataDefinition map method.
Type parameters
- K
-
the key type
- V
-
the incoming value type
- W
-
the outgoing value type
Value parameters
- f
-
a V=>W function
Attributes
- Returns
-
a (K,V) => (K,W) function