Majabigwaduce
API
Generated with
Majabigwaduce/com.phasmid.majabigwaduce/com.phasmid.majabigwaduce.core

com.phasmid.majabigwaduce.core

package com.phasmid.majabigwaduce.core

Members list

Type members

Classlikes

object Actors

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Actors.type
case class Actors(system: ActorSystem[_], config: Config) extends AutoCloseable

Case class to deal with the construction and configuration of actors.

Case class to deal with the construction and configuration of actors.

CONSIDER eliminating this class and calling spawn/systemActorOf directly. See Issue #13.

Value parameters

config

the configuration for this set of actors.

system

the actor system.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
trait AutoCloseable
class Object
trait Matchable
class Any
Show all
object CleanerCollector

Cleans a sequence of input data that may contain exceptions. The main purpose is to organize the valid results by their keys and separate out any encountered exceptions. Used by Mapper to build its response.

Cleans a sequence of input data that may contain exceptions. The main purpose is to organize the valid results by their keys and separate out any encountered exceptions. Used by Mapper to build its response.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
final case class CloseMapper[K1, V1, K2, W]() extends MapperCommand[K1, V1, K2, W]

Command to stop this Mapper.

Command to stop this Mapper.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait MapperCommand[K1, V1, K2, W]
class Object
trait Matchable
class Any
Show all
final case class CloseMaster[K1, V1, K2, V2]() extends MasterCommand[K1, V1, K2, V2]

Command to stop this Master -- its mapper and reducer children are stopped transitively.

Command to stop this Master -- its mapper and reducer children are stopped transitively.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait MasterCommand[K1, V1, K2, V2]
class Object
trait Matchable
class Any
Show all
final case class CloseReducer[K2, W, V2]() extends ReducerCommand[K2, W, V2]

Command to stop this Reducer.

Command to stop this Reducer.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait ReducerCommand[K2, W, V2]
class Object
trait Matchable
class Any
Show all
final case class ComputeMap[K1, V1, K2, V2](m: Map[K1, V1], replyTo: ActorRef[Try[Response[K2, V2]]]) extends MasterCommand[K1, V1, K2, V2]

Command carrying a Map[K1,V1] payload, replying to replyTo with a Try[Response[K2,V2]].

Command carrying a Map[K1,V1] payload, replying to replyTo with a Try[Response[K2,V2]].

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait MasterCommand[K1, V1, K2, V2]
class Object
trait Matchable
class Any
Show all
final case class ComputeSeq[K1, V1, K2, V2](s: Seq[(K1, V1)], replyTo: ActorRef[Try[Response[K2, V2]]]) extends MasterCommand[K1, V1, K2, V2]

Command carrying a Seq[(K1,V1)] payload, replying to replyTo with a Try[Response[K2,V2]].

Command carrying a Seq[(K1,V1)] payload, replying to replyTo with a Try[Response[K2,V2]].

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait MasterCommand[K1, V1, K2, V2]
class Object
trait Matchable
class Any
Show all
final case class DoMap[K1, V1, K2, W](kvs: KeyValuePairs[K1, V1], replyTo: ActorRef[MapperResponse[K2, W]]) extends MapperCommand[K1, V1, K2, W]

Command to map the given key-value pairs, replying to replyTo with a MapperResponse.

Command to map the given key-value pairs, replying to replyTo with a MapperResponse.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait MapperCommand[K1, V1, K2, W]
class Object
trait Matchable
class Any
Show all
final case class DoReduce[K2, W, V2](i: Intermediate[K2, W], replyTo: ActorRef[ReduceResult[K2, V2]]) extends ReducerCommand[K2, W, V2]

Command to reduce the sequence of W values associated with a key, replying to replyTo with a ReduceResult.

Command to reduce the sequence of W values associated with a key, replying to replyTo with a ReduceResult.

NOTE: named DoReduce (not Reduce) to avoid clashing with the unrelated Reduce[K,T,S] class in MapReduce.scala.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait ReducerCommand[K2, W, V2]
class Object
trait Matchable
class Any
Show all
object FP

This module contains functional programming methods which can also be found in the FP module of LaScala

This module contains functional programming methods which can also be found in the FP module of LaScala

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
FP.type
final case class FoldDef[K2, W, V2](g: (V2, W) => V2, z: () => V2) extends ReducerDef[K2, W, V2]

Definition of a Reducer_Fold (foldLeft with g from z()). NOTE z is kept as a function so that each reduction still starts from a fresh zero, exactly as Reducer_Fold's by-name parameter does.

Definition of a Reducer_Fold (foldLeft with g from z()). NOTE z is kept as a function so that each reduction still starts from a fresh zero, exactly as Reducer_Fold's by-name parameter does.

Attributes

Supertypes
trait Product
trait Equals
trait ReducerDef[K2, W, V2]
trait Serializable
class Object
trait Matchable
class Any
Show all
case class Intermediate[K2, W](k2: K2, ws: Seq[W])

Represents an intermediate result containing a key-value pair and a sequence of elements.

Represents an intermediate result containing a key-value pair and a sequence of elements.

Type parameters

K2

The type of the key.

W

The type of the elements in the sequence.

Value parameters

k2

The key of the intermediate result.

ws

A sequence of elements associated with the key.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object KeyValuePairs

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
case class KeyValuePairs[K, V](m: Seq[(K, V)])

Case class to package a map of key-value pairs for the purpose of sending to an actor.

Case class to package a map of key-value pairs for the purpose of sending to an actor.

Type parameters

K

the key type.

V

the value type.

Value parameters

m

a map (in sequential form).

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
trait LazyMonad[T, U] extends Iterable[U]

A trait that defines a lazy monad, which wraps an Iterable and provides functionality for applying transformations and filters in a lazy manner.

A trait that defines a lazy monad, which wraps an Iterable and provides functionality for applying transformations and filters in a lazy manner.

Type parameters

T

the type of elements in the wrapped iterable before transformation.

U

the type of elements in the wrapped iterable after transformation.

Attributes

Companion
object
Supertypes
trait Iterable[U]
trait IterableFactoryDefaults[U, Iterable]
trait IterableOps[U, Iterable, Iterable[U]]
trait IterableOnceOps[U, Iterable, Iterable[U]]
trait IterableOnce[U]
class Object
trait Matchable
class Any
Show all
Known subtypes
class LazySequence[T, U]
object LazyMonad

Companion object for the LazyMonad type. Provides factory methods to create instances of LazySequence, which is a concrete implementation of LazyMonad.

Companion object for the LazyMonad type. Provides factory methods to create instances of LazySequence, which is a concrete implementation of LazyMonad.

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
LazyMonad.type
case class LazySequence[T, U](ts: Iterable[T], f: T => U, p: T => Boolean) extends LazyMonad[T, U]

A case class that extends LazyMonad and provides a mechanism to lazily transform and filter an iterable collection. The transformation and filtering are performed using a mapping function and a predicate respectively.

A case class that extends LazyMonad and provides a mechanism to lazily transform and filter an iterable collection. The transformation and filtering are performed using a mapping function and a predicate respectively.

Type parameters

T

the type of the elements in the initial collection.

U

the type of the elements in the resulting collection after applying the mapping function.

Value parameters

f

the mapping function T => U to transform elements of the collection.

p

the predicate T => Boolean to pre-filter elements in the collection before applying the mapping function.

ts

the initial iterable collection of type T.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait LazyMonad[T, U]
trait Iterable[U]
trait IterableFactoryDefaults[U, Iterable]
trait IterableOps[U, Iterable, Iterable[U]]
trait IterableOnceOps[U, Iterable, Iterable[U]]
trait IterableOnce[U]
class Object
trait Matchable
class Any
Show all
object LocalWorkerPool extends WorkerPool

The default WorkerPool: spawns the workers as local children of the Master, so they are stopped when the Master stops. This is what Master did directly before 2.1.0.

The default WorkerPool: spawns the workers as local children of the Master, so they are stopped when the Master stops. This is what Master did directly before 2.1.0.

Attributes

Supertypes
trait WorkerPool
class Object
trait Matchable
class Any
Self type
trait MapReduce[T, K1, V1] extends Seq[T] => Future[Map[K1, V1]], AutoCloseable

MapReduce is a trait, with case classes, which implements a functional API for the map-reduce classes in this package.

MapReduce is a trait, with case classes, which implements a functional API for the map-reduce classes in this package.

Type parameters

K1

the key type of the returned map

T

the input type of the MapReduce function: T may be V1 for a first stage, or (K1,V1) for a subsequent stage.

V1

the value type of the returned map

Attributes

Supertypes
trait AutoCloseable
trait Seq[T] => Future[Map[K1, V1]]
class Object
trait Matchable
class Any
Known subtypes
class MapReduceComposed[T, K1, V1, K2, V2]
class MapReduce_Base[T, K1M, V1M, K, V]
class MapReduce_LoggingBase[T, K1M, V1M, K1, V1]
class MapReduceFirst[V0, K1, W, V1]
class MapReduceFirstFold[V0, K1, W, V1]
class MapReducePipe[K0, V0, K1, W, V1]
class MapReducePipeFold[K0, V0, K1, W, V1]
Show all
Self type
MapReduce[T, K1, V1]
object MapReduceActor

Shared lifecycle support for the actors in this package (Mapper, Reducer, Master).

Shared lifecycle support for the actors in this package (Mapper, Reducer, Master).

Akka Typed behaviors are composed, not inherited, so this replaces what was previously an abstract Actor base class. withLifecycle wraps a command handler with the same debug-level start/stop logging the old base class provided via preStart/postStop. Each actor's own command handler recognizes its own protocol's Close case and returns Behaviors.stopped -- there is no shared Close type across protocols, since each actor's command ADT is self-contained.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
case class MapReduceComposed[T, K1, V1, K2, V2](f1: MapReduce[T, K1, V1], f2: Seq[(K1, V1)] => Future[Map[K2, V2]])(implicit ec: ExecutionContext) extends MapReduce[T, K2, V2]

A composition MapReduce class which represents a MapReduce "pipeline" with two stages.

A composition MapReduce class which represents a MapReduce "pipeline" with two stages.

Type parameters

K1

intermediate key type

K2

output key type

T

the input type of the MapReduce function: T may be V1 for a first stage, or (K1,V1) for a subsequent stage.

V1

intermediate value type

V2

output value type

Value parameters

f1

an instance of MapReduce which will become the first of two stages of the resulting MapReduce instance .

f2

an ASync[Seq[(K1, V1)], Map[K2, V2] instance (functionally equivalent to MapReduce) which will become the second of the two stages.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait MapReduce[T, K2, V2]
trait AutoCloseable
trait Seq[T] => Future[Map[K2, V2]]
class Object
trait Matchable
class Any
Show all
case class MapReduceException(context: String, x: Throwable) extends Throwable

Represents an exception that occurs during MapReduce operations.

Represents an exception that occurs during MapReduce operations.

Value parameters

context

A string describing the context or scenario where the exception was raised.

x

The underlying cause of the exception, provided as a throwable.

Attributes

Companion
object
Supertypes
trait Product
trait Equals
class Throwable
trait Serializable
class Object
trait Matchable
class Any
Show all
object MapReduceException

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
case class MapReduceFirst[V0, K1, W, V1 >: W](f: V0 => Try[(K1, W)], g: (V1, W) => V1)(actors: Actors, timeout: Timeout) extends MapReduce_LoggingBase[V0, Unit, V0, K1, V1]

A first-stage MapReduce class where the result type V1 is a super-type of the intermediate type W

A first-stage MapReduce class where the result type V1 is a super-type of the intermediate type W

Type parameters

K1

output key type

V0

input value type

V1

output value type (super-type of W)

W

intermediate type

Value parameters

actors

an instance of Actors

f

the mapper function which takes a V0 instance and creates a key-value tuple of type (K1,W)

g

the reducer function which combines two values (an V1 and a W) into one V1

timeout

the value of timeout to be used

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class MapReduce_LoggingBase[V0, Unit, V0, K1, V1]
class MapReduce_Base[V0, Unit, V0, K1, V1]
trait MapReduce[V0, K1, V1]
trait AutoCloseable
trait Seq[V0] => Future[Map[K1, V1]]
class Object
trait Matchable
class Any
Show all
object MapReduceFirst

Companion object for the MapReduceFirst class that provides utility methods relevant to the MapReduceFirst implementation.

Companion object for the MapReduceFirst class that provides utility methods relevant to the MapReduceFirst implementation.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
object MapReduceFirstFold

Companion object for MapReduceFirstFold, providing factory methods and constants.

Companion object for MapReduceFirstFold, providing factory methods and constants.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
case class MapReduceFirstFold[V0, K1, W, V1](f: V0 => Try[(K1, W)], g: (V1, W) => V1)(actors: Actors, timeout: Timeout)(using evidence$1: Zero[V1]) extends MapReduce_LoggingBase[V0, Unit, V0, K1, V1]

A first-stage MapReduce class

A first-stage MapReduce class

Type parameters

K1

output key type.

V0

input value type.

V1

output value type (must support type class Init).

W

intermediate type.

Value parameters

actors

an instance of Actors.

f

the mapper function which takes a V0 instance and creates a key-value tuple of type (K1,W) (wrapped in Try, but see alternative constructor).

g

the reducer function which combines two values (an V1 and a W) into one V1.

timeout

the value of timeout to be used. CONSIDER why is config parameter set not implicit?

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class MapReduce_LoggingBase[V0, Unit, V0, K1, V1]
class MapReduce_Base[V0, Unit, V0, K1, V1]
trait MapReduce[V0, K1, V1]
trait AutoCloseable
trait Seq[V0] => Future[Map[K1, V1]]
class Object
trait Matchable
class Any
Show all
case class MapReducePipe[K0, V0, K1, W, V1 >: W](f: (K0, V0) => Try[(K1, W)], g: (V1, W) => V1, n: Int)(implicit actors: Actors, timeout: Timeout) extends MapReduce_LoggingBase[(K0, V0), K0, V0, K1, V1]

A later-stage MapReduce class where the result type V1 is a super-type of the intermediate type W

A later-stage MapReduce class where the result type V1 is a super-type of the intermediate type W

Type parameters

K0

input key type

K1

output key type

V0

input value type

V1

output value type (super-type of W)

W

intermediate type

Value parameters

actors

an instance of Actors

f

the mapper function which takes a K0,V0 pair and creates a key-value tuple of type (K1,W)

g

the reducer function which combines two values (an V1 and a W) into one V1

n

the stage number of this map-reduce stage.

timeout

the value of timeout to be used

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class MapReduce_LoggingBase[(K0, V0), K0, V0, K1, V1]
class MapReduce_Base[(K0, V0), K0, V0, K1, V1]
trait MapReduce[(K0, V0), K1, V1]
trait AutoCloseable
trait Seq[(K0, V0)] => Future[Map[K1, V1]]
class Object
trait Matchable
class Any
Show all
object MapReducePipe

Companion object for the MapReducePipe case class, providing utility functions for creating instances of the MapReducePipe.

Companion object for the MapReducePipe case class, providing utility functions for creating instances of the MapReducePipe.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
case class MapReducePipeFold[K0, V0, K1, W, V1](f: (K0, V0) => Try[(K1, W)], g: (V1, W) => V1, n: Int)(actors: Actors, timeout: Timeout)(using evidence$1: Zero[V1]) extends MapReduce_LoggingBase[(K0, V0), K0, V0, K1, V1]

A later-stage MapReduce class

A later-stage MapReduce class

Type parameters

K0

input key type

K1

output key type

V0

input value type

V1

output value type (must support type class Init)

W

intermediate type

Value parameters

actors

an instance of Actors.

f

the mapper function which takes a V0 instance and creates a key-value tuple of type (K1,W)

g

the reducer function which combines two values (an V1 and a W) into one V1

n

the stage number of this map-reduce stage.

timeout

the value of timeout to be used

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class MapReduce_LoggingBase[(K0, V0), K0, V0, K1, V1]
class MapReduce_Base[(K0, V0), K0, V0, K1, V1]
trait MapReduce[(K0, V0), K1, V1]
trait AutoCloseable
trait Seq[(K0, V0)] => Future[Map[K1, V1]]
class Object
trait Matchable
class Any
Show all
object MapReducePipeFold

Singleton object for creating and managing MapReducePipeFold instances.

Singleton object for creating and managing MapReducePipeFold instances.

This companion object provides utility methods to initialize MapReducePipeFold instances, given appropriate functions for mapping and reducing, stage numbers, actor system configurations, and timeout settings.

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
abstract class MapReduce_Base[T, K1M, V1M, K, V](actors: Actors)(using timeout: Timeout) extends MapReduce[T, K, V]

An abstract base class for MapReduce classes (other than MapReduceComposed).

An abstract base class for MapReduce classes (other than MapReduceComposed).

Type parameters

K

output key type

K1M

the key type of the master's own protocol (Unit for a first stage, or K for a subsequent stage).

T

the input type of the MapReduce function: T may be V1 for a first stage, or (K1,V1) for a subsequent stage.

V

output value type

V1M

the value type of the master's own protocol (T for a first stage, or the V1 half of T for a subsequent stage).

Attributes

Supertypes
trait MapReduce[T, K, V]
trait AutoCloseable
trait Seq[T] => Future[Map[K, V]]
class Object
trait Matchable
class Any
Show all
Known subtypes
class MapReduce_LoggingBase[T, K1M, V1M, K1, V1]
class MapReduceFirst[V0, K1, W, V1]
class MapReduceFirstFold[V0, K1, W, V1]
class MapReducePipe[K0, V0, K1, W, V1]
class MapReducePipeFold[K0, V0, K1, W, V1]
abstract class MapReduce_LoggingBase[T, K1M, V1M, K1, V1](actors: Actors)(timeout: Timeout) extends MapReduce_Base[T, K1M, V1M, K1, V1]

An abstract base class which extends MapReduce_Base and which implements the logException method with non-trivial logging.

An abstract base class which extends MapReduce_Base and which implements the logException method with non-trivial logging.

Type parameters

K1

intermediate key type

K1M

the key type of the master's own protocol (Unit for a first stage, or the outer K1 for a subsequent stage).

T

the input type of the MapReduce function: T may be V1 for a first stage, or (K1,V1) for a subsequent stage.

V1

intermediate value type

V1M

the value type of the master's own protocol.

Value parameters

actors

an instance of Actors

timeout

the value of timeout to be used

Attributes

Supertypes
class MapReduce_Base[T, K1M, V1M, K1, V1]
trait MapReduce[T, K1, V1]
trait AutoCloseable
trait Seq[T] => Future[Map[K1, V1]]
class Object
trait Matchable
class Any
Show all
Known subtypes
class MapReduceFirst[V0, K1, W, V1]
class MapReduceFirstFold[V0, K1, W, V1]
class MapReducePipe[K0, V0, K1, W, V1]
class MapReducePipeFold[K0, V0, K1, W, V1]
object Mapper

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
Mapper.type
sealed trait MapperCommand[K1, V1, K2, W]

The purpose of this mapper is to convert a sequence of objects into several sequences, each of which is associated with a key. It must be possible to do further processing (the reduce phase) on each of these resulting sequences independently (and, thus in parallel). Furthermore, the mapping function should try, when possible, to divide the input sequence into a number of more or less equally lengthy sequences.

The purpose of this mapper is to convert a sequence of objects into several sequences, each of which is associated with a key. It must be possible to do further processing (the reduce phase) on each of these resulting sequences independently (and, thus in parallel). Furthermore, the mapping function should try, when possible, to divide the input sequence into a number of more or less equally lengthy sequences.

The mapper is an actor whose behavior is parametrized by a function f which converts a (K1,V1) into a (K2,W). It handles a DoMap command carrying a KeyValuePairs[K1,V1] and a replyTo reference. It replies with a MapperResponse[K2,W], which carries both the successfully-mapped results and any exceptions encountered while mapping -- whether that constitutes an overall failure is a decision made by the caller (see Master.doMap), not by the Mapper itself.

KeyValuePairs is a convenience wrapper for incoming messages. It has the advantage of not suffering type erasure, and it also has a toString method which simply shows the number of pairs, not their values.

Note that logging the actual values received in the incoming message can be VERY verbose. It is therefore recommended practice (if required) to log the values as they pass through the mapper function (f) which is under the control of the application.

Type parameters

K1

(input) key type (may be Unit)

K2

(output) key type

V1

(input) value type

W

(output) value type

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class CloseMapper[K1, V1, K2, W]
class DoMap[K1, V1, K2, W]
final case class MapperDef[K1, V1, K2, W](f: (K1, V1) => Try[(K2, W)], forgiving: Boolean) extends Serializable

Describes how to build a mapper behavior. Unlike a Behavior, a definition holds only the mapper function and its mode, and is Serializable -- so a non-local WorkerPool can ship it to another node and build the behavior there (see doc/Clustering.md).

Describes how to build a mapper behavior. Unlike a Behavior, a definition holds only the mapper function and its mode, and is Serializable -- so a non-local WorkerPool can ship it to another node and build the behavior there (see doc/Clustering.md).

Value parameters

f

the mapper function.

forgiving

whether to use Mapper_Forgiving rather than Mapper.

Attributes

Supertypes
trait Product
trait Equals
trait Serializable
class Object
trait Matchable
class Any
Show all
case class MapperResponse[K2, W](result: Map[K2, Seq[W]], exceptions: Seq[Throwable])

The response sent back from a Mapper.

The response sent back from a Mapper.

Value parameters

exceptions

any exceptions encountered while mapping.

result

the successfully-mapped key-value pairs, grouped by key.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Mapper_Forgiving

Prior to the Typed migration, this was a distinct actor subclass overriding isStrict to retain exceptions instead of failing outright. Now that MapperResponse always carries both the successful results and any exceptions, strict-vs-forgiving is purely a decision made by the caller (see Master.doMap / Master.isForgiving), so this is an alias for Mapper -- kept as a separate name for source compatibility with existing call sites.

Prior to the Typed migration, this was a distinct actor subclass overriding isStrict to retain exceptions instead of failing outright. Now that MapperResponse always carries both the successful results and any exceptions, strict-vs-forgiving is purely a decision made by the caller (see Master.doMap / Master.isForgiving), so this is an alias for Mapper -- kept as a separate name for source compatibility with existing call sites.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
object Master

Value parameters

config

an instance of Config which defines a suitable configuration

f

the mapper function which takes a (K1,V1) pair and creates a key-value tuple of type (K2,W), wrapped in Try

g

the reducer function which combines two values (a V2 and a W) into one V2

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
Master.type
sealed trait MasterCommand[K1, V1, K2, V2]

The command protocol handled by a Master actor.

The command protocol handled by a Master actor.

A Master accepts its input in either of two shapes -- as a Map[K1,V1] or as a Seq[(K1,V1)] -- mirroring the two message shapes the original (classic-actor) Master.receive matched on. The "First" variants (Master_First / Master_First_Fold) reuse this same protocol with K1=Unit, exactly as the original MasterBaseFirst reused MasterBase[Unit,...] internally: their public factory functions unitize the incoming Seq[V1] into a Seq[(Unit,V1)] before it is sent here.

NOTE: unlike the classic actor (which replied with Status.Failure on a hard failure), a Typed reply can only ever be the declared type. So the reply type here is Try[Response[K2,V2]] rather than a bare Response[K2,V2] -- this preserves the original behavior of failing the caller's ask with the original exception, rather than letting it time out.

Type parameters

K1

key type: the message which this actor responds to is of type Map[K1,V1].

K2

key type: mapper groups things by this key and reducer processes said groups.

V1

input type: the message which this actor responds to is of type Seq[V1].

V2

output type: the message which is sent on completion to the sender is of type Response[K2,V2]

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class CloseMaster[K1, V1, K2, V2]
class ComputeMap[K1, V1, K2, V2]
class ComputeSeq[K1, V1, K2, V2]
object Master_First

Type parameters

K2

key type: mapper groups things by this key and reducer processes said groups.

V1

input type: the message which this actor responds to is of type Seq[V1].

V2

output type: the message which is sent on completion to the sender is of type Response[K2,V2]

W

transitional type -- used internally

Value parameters

config

an instance of Config which defines a suitable configuration

f

the mapper function which takes a V1 and creates a key-value tuple of type (K2,W), wrapped in Try

g

the reducer function which combines two values (a V2 and a W) into one V2

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
object Master_First_Fold

Type parameters

K2

key type: mapper groups things by this key and reducer processes said groups.

V1

input type: the message which this actor responds to is of type Seq[V1].

V2

output type: the message which is sent on completion to the sender is of type Response[K2,V2]

W

transitional type -- used internally

Value parameters

config

an instance of Config which defines a suitable configuration

f

the mapper function which takes a V1 and creates a key-value tuple of type (K2,W), wrapped in Try

g

the reducer function which combines two values (a V2 and a W) into one V2

z

the "zero" or "unit" (i.e. initializer) function which creates an "empty" V2.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
object Master_Fold

Type parameters

K1

key type: the message which this actor responds to is of type Map[K1,V1].

K2

key type: mapper groups things by this key and reducer processes said groups.

V1

input type: the message which this actor responds to is of type Seq[V1].

V2

output type: the message which is sent on completion to the sender is of type Response[K2,V2]

W

transitional type -- used internally

Value parameters

config

an instance of Config which defines a suitable configuration

f

the mapper function which takes a (K1,V1) pair and creates a key-value tuple of type (K2,W), wrapped in Try

g

the reducer function which combines two values (a V2 and a W) into one V2

z

the "zero" or "unit" (i.e. initializer) function which creates an "empty" V2.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
object Monoid

Provides Monoid instances for standard types and utility methods for working with monoids.

Provides Monoid instances for standard types and utility methods for working with monoids.

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Monoid.type
trait Monoid[X] extends Zero[X]

Type parameters

X

the type which we want to create a monoid value for. Created by scalaprof on 10/5/16.

Attributes

Companion
object
Supertypes
trait Zero[X]
trait Serializable
class Object
trait Matchable
class Any
Known subtypes
object MonoidDouble
object MonoidInt
object MonoidString
trait IterableMonoid[T]
case class Reduce[K, T, S](f: (S, T) => S)(using evidence$1: Zero[S]) extends Map[K, T] => S

A reduce function which can be composed (on the right) with a MapReduce object.

A reduce function which can be composed (on the right) with a MapReduce object.

Type parameters

S

the output (derived) type of this reduction

T

the input (free) type of this reduction

Value parameters

f

the function which will combine the current result with each element of an input set

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait Map[K, T] => S
class Object
trait Matchable
class Any
Show all
final case class ReduceDef[K2, W, V2 >: W](g: (V2, W) => V2) extends ReducerDef[K2, W, V2]

Definition of a plain Reducer (reduceLeft with g).

Definition of a plain Reducer (reduceLeft with g).

Attributes

Supertypes
trait Product
trait Equals
trait ReducerDef[K2, W, V2]
trait Serializable
class Object
trait Matchable
class Any
Show all
case class ReduceResult[K2, V2](k2: K2, result: Either[Throwable, V2])

The response sent back from a Reducer.

The response sent back from a Reducer.

Value parameters

k2

the key of the intermediate result which was reduced.

result

the aggregated value, or the Throwable resulting from a failed reduction.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Reducer

Type parameters

K2

key type

V2

the aggregation of W objects (in this form, must be super-type of W)

W

value type

Value parameters

g

a function which takes a V2 (the accumulator) and a W (the value) and combines them into a V2

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
Reducer.type
sealed trait ReducerCommand[K2, W, V2]

This actor performs the reduce operation on the received sequence of W objects, resulting in (ideally) a V2 object. It handles a Reduce command combining both the current K2 key and the sequence ws of W objects. The reply is a ReduceResult[K2,V2].

This actor performs the reduce operation on the received sequence of W objects, resulting in (ideally) a V2 object. It handles a Reduce command combining both the current K2 key and the sequence ws of W objects. The reply is a ReduceResult[K2,V2].

Note that logging the actual values received in the incoming message can be VERY verbose. It is therefore recommended practice to log the values as they pass through the reducer function (g) which is under the control of the application.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class CloseReducer[K2, W, V2]
class DoReduce[K2, W, V2]
sealed trait ReducerDef[K2, W, V2] extends Serializable

Describes how to build a reducer behavior: either a plain Reducer or a Reducer_Fold. Like MapperDef, it is Serializable so that it can be shipped to another node.

Describes how to build a reducer behavior: either a plain Reducer or a Reducer_Fold. Like MapperDef, it is Serializable so that it can be shipped to another node.

Attributes

Supertypes
trait Serializable
class Object
trait Matchable
class Any
Known subtypes
class FoldDef[K2, W, V2]
class ReduceDef[K2, W, V2]
object Reducer_Fold

Type parameters

K2

key type

V2

the aggregation of W objects

W

value type

Value parameters

g

a function which takes a V2 (the accumulator) and a W (the value) and combines them into a V2

z

a function which provides an initial value for V2 (this allows us to use Fold rather than Reduce methods)

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
object Response

Factory object for the Response case class.

Factory object for the Response case class.

Provides functionality to create a Response instance by processing a map of key-value pairs where the values are wrapped in an Either. The left side of the Either represents an error (Throwable), while the right side represents a valid value.

Type parameters

K

the key type.

V

the value type.

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Response.type
case class Response[K, V](left: Map[K, Throwable], right: Map[K, V])

Case class used to package a response from an actor.

Case class used to package a response from an actor.

Type parameters

K

the key type.

V

the value type.

Value parameters

left

a map of key-value pairs where the value is a Throwable.

right

a map of key-value pairs where the value is a value.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object TryUsing

The TryUsing object provides utility methods to manage resources safely and effectively using Scala's Using and Try. It encapsulates resource management in a functional way, ensuring proper release of resources. The methods in this object extend the functionality of Using.apply by offering flattening operations over nested Try.

The TryUsing object provides utility methods to manage resources safely and effectively using Scala's Using and Try. It encapsulates resource management in a functional way, ensuring proper release of resources. The methods in this object extend the functionality of Using.apply by offering flattening operations over nested Try.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
TryUsing.type
trait WorkerPool

Where a Master's mappers and reducers come from.

Where a Master's mappers and reducers come from.

acquire is called once, from the Master's setup (i.e. on the Master's own thread, so an implementation may use context directly). It returns a Future so that an implementation can obtain workers asynchronously -- e.g. by asking remote nodes to spawn them.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object WorkerPool

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
WorkerPool.type
final case class Workers[K1, V1, K2, W, V2](mappers: Seq[ActorRef[MapperCommand[K1, V1, K2, W]]], reducers: Seq[ActorRef[ReducerCommand[K2, W, V2]]])

The mapper and reducer actors which a Master drives.

The mapper and reducer actors which a Master drives.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Zero

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Zero.type
trait Zero[X] extends Serializable

Type-class Zero is used to add behavior of initialization (or zeroing) of X.

Type-class Zero is used to add behavior of initialization (or zeroing) of X.

NOTE Zero is Serializable because a fold's zero travels with its reducer when the reducer runs on another node of a cluster (see doc/Clustering.md).

Type parameters

X

the type which we want to create a zero value for.

Attributes

Companion
object
Supertypes
trait Serializable
class Object
trait Matchable
class Any
Known subtypes
trait Monoid[X]
object MonoidDouble
object MonoidInt
object MonoidString
trait IterableMonoid[T]
trait DoubleZero
object DoubleZero
trait IntZero
object IntZero
trait SeqZero[X]
object IntSeqZero
trait StringZero
object StringZero
trait VectorZero[X]
object IntVectorZero
Show all
Generated with