Majabigwaduce
API
Generated with
Majabigwaduce/com.phasmid.majabigwaduce/com.phasmid.majabigwaduce.dd

com.phasmid.majabigwaduce.dd

package com.phasmid.majabigwaduce.dd

Members list

Type members

Classlikes

abstract class BaseDD[K, V](using ec: ExecutionContext) extends DataDefinition[K, V]

Abstract base class which implements the generic DataDefinition[K, V].

Abstract base class which implements the generic DataDefinition[K, V].

Type parameters

K

the key type

V

the input value type

Value parameters

ec

an ExecutionContext

Attributes

Supertypes
trait DataDefinition[K, V]
trait () => Future[Map[K, V]]
class Object
trait Matchable
class Any
Known subtypes
class EagerDD[K, V]
class LazyDD[K, V, L, W]
case class DDContext(config: Config, system: ActorSystem[Nothing], timeout: Timeout)

The context in which DataDefinition instances will be evaluated

The context in which DataDefinition instances will be evaluated

Value parameters

config

the configuration

system

the actor system

timeout

the value of timeout

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object DDContext

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
DDContext.type
object DataDefinition

Represents a singleton object that provides definitions or metadata for data-related functionalities.

Represents a singleton object that provides definitions or metadata for data-related functionalities.

This object is designed to encapsulate the structure and behavior associated with defining, managing, or interpreting data schemas or specifications.

It can be used to centralize data definitions, facilitate consistency across data-related processes, and support extensible data handling operations.

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
sealed trait DataDefinition[K, V] extends () => Future[Map[K, V]]

Trait to represent a "data definition" (similar to RDD in Spark). Essentially, a DataDefinition[K, V] is a function which transforms Unit into a Future[Map[K,V]. Like RDD, it is lazy and can be partitioned. In order to yield a concrete value, i.e. an "action", there are three methods which may be called: apply, reduce(f), and count.

Trait to represent a "data definition" (similar to RDD in Spark). Essentially, a DataDefinition[K, V] is a function which transforms Unit into a Future[Map[K,V]. Like RDD, it is lazy and can be partitioned. In order to yield a concrete value, i.e. an "action", there are three methods which may be called: apply, reduce(f), and count.

Created by scalaprof on 10/31/16.

Type parameters

K

the key type

V

the value type

Attributes

Companion
object
Supertypes
trait () => Future[Map[K, V]]
class Object
trait Matchable
class Any
Known subtypes
class BaseDD[K, V]
class EagerDD[K, V]
class LazyDD[K, V, L, W]
case class DataDefinitionException(str: String) extends Exception

Exception to indicate issues related to data definition operations.

Exception to indicate issues related to data definition operations.

This exception is thrown when operations involving DataDefinition objects cannot be performed due to incompatible types, unsupported operations, or other invalid states. Examples include attempting to join Eager and non-Eager DataDefinition objects or operations not supported between Lazy and Base DataDefinition instances.

Value parameters

str

the error message describing the issue.

Attributes

Supertypes
trait Product
trait Equals
class Exception
class Throwable
trait Serializable
class Object
trait Matchable
class Any
Show all
case class EagerDD[K, V](kVs: Seq[(K, V)])(implicit ec: ExecutionContext) extends BaseDD[K, V], HasEvaluatedMap[K, V]

Case Class which implements DataDefinition[K, V] eagerly and which is based on a Map[K,V]. NOTE: this type serves as the appropriate result of evaluating a lazy DD. In that respect it is the equivalent of the array which is created when invoking collect on an RDD in Spark.

Case Class which implements DataDefinition[K, V] eagerly and which is based on a Map[K,V]. NOTE: this type serves as the appropriate result of evaluating a lazy DD. In that respect it is the equivalent of the array which is created when invoking collect on an RDD in Spark.

Type parameters

K

the key type

V

the input value type

Value parameters

ec

the (implicit) execution context

kVs

the actual data definition represented as a Map

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
trait HasEvaluatedMap[K, V]
class BaseDD[K, V]
trait DataDefinition[K, V]
trait () => Future[Map[K, V]]
class Object
trait Matchable
class Any
Show all
object EagerDD

A data structure that eagerly evaluates and stores key-value pairs.

A data structure that eagerly evaluates and stores key-value pairs.

EagerDD is designed for scenarios where key-value data needs to be stored and accessed in an immutable and thread-safe manner.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
EagerDD.type
sealed trait HasEvaluatedMap[K, V]

This trait is essentially a private trait: only to be used by this module.

This trait is essentially a private trait: only to be used by this module.

Type parameters

K

the key type

V

the value type

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class EagerDD[K, V]
object LazyDD

An object containing utility methods for performing operations on keyed collections and functions, particularly for supporting the LazyDD class.

An object containing utility methods for performing operations on keyed collections and functions, particularly for supporting the LazyDD class.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
LazyDD.type
case class LazyDD[K, V, L, W](kVs: Seq[(K, V)], f: ((K, V)) => (L, W))(partitions: Int = ...)(implicit evidence$1: Monoid[W], context: DDContext) extends BaseDD[L, W]

Case Class which implements DataDefinition[K, W] and which is based on a Map[K,V] and a function V => W.

Case Class which implements DataDefinition[K, W] and which is based on a Map[K,V] and a function V => W.

CONSIDER a property cutoff to determine whether it's appropriate to invoke the map-reduce machinery on kVs.

Type parameters

K

the key type

V

the input value type

W

the output value type

Value parameters

context

a DDContext

f

a function which will transform the key-value pairs

kVs

the map of key-value pairs which serve as the input to this LazyDD

partitions

the number of partitions to be used

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class BaseDD[L, W]
trait DataDefinition[L, W]
trait () => Future[Map[L, W]]
class Object
trait Matchable
class Any
Show all
Generated with