Trait to represent a "data definition" (similar to RDD in Spark). Essentially, a DataDefinition[K, V] is a function which transforms Unit into a Future[Map[K,V]. Like RDD, it is lazy and can be partitioned. In order to yield a concrete value, i.e. an "action", there are three methods which may be called: apply, reduce(f), and count.
Created by scalaprof on 10/31/16.
Type parameters
- K
-
the key type
- V
-
the value type
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
trait () => Future[Map[K, V]]class Objecttrait Matchableclass Any
- Known subtypes
Members list
Value members
Abstract methods
Clean up any residual resources from this DataDefinition
Clean up any residual resources from this DataDefinition
Attributes
Evaluate the number of elements in this DataDefinition
Evaluate the number of elements in this DataDefinition
Attributes
- Returns
-
the number of k-v pairs
Method to filter this DataDefinition according to a predicate which takes a k-v tuple.
Method to filter this DataDefinition according to a predicate which takes a k-v tuple.
Value parameters
- p
-
the predicate which will yield a Boolean for a given k-v tuple.
Attributes
- Returns
-
a new DataDefinition containing only those k-v pairs which satisfy the predicate p.
Method to group values by a new key type generated from the values, ignoring the current keys.
Method to group values by a new key type generated from the values, ignoring the current keys.
Type parameters
- L
-
the new key type, whose values are derived from the values of this DataDefinition
Attributes
- Returns
-
a DataDefinition based on L and Iterable[V]
Join method to perform inner join.
Join method to perform inner join.
Type parameters
- L
-
key type of other and also of the result
- W
-
value type of other
Value parameters
- other
-
the DataDefinition with which to join this
Attributes
- Returns
-
the inner join of this and other
Method to form a new DataDefinition where the resulting values derive from applying the function f to the original values
Method to form a new DataDefinition where the resulting values derive from applying the function f to the original values
Type parameters
- L
-
the underlying type of the keys of the resulting map
- W
-
the underlying type of the values of the resulting map
Value parameters
- f
-
the function to transform key-value pairs
Attributes
- Returns
-
a new DataDefinition
Method to evaluate this DataDefinition and reduce the dimensionality of the result by ignoring the keys and aggregating the values according to the function wv_w
Method to evaluate this DataDefinition and reduce the dimensionality of the result by ignoring the keys and aggregating the values according to the function wv_w
Type parameters
- W
-
the underlying type of the result
Value parameters
- wVWf
-
the aggregation function
Attributes
- Returns
-
a W value, wrapped in Future.
Inherited methods
Returns a string representation of the object.
Returns a string representation of the object.
The default representation is platform dependent.
Attributes
- Returns
-
a string representation of the object.
- Definition Classes
-
Function0 -> Any
- Inherited from:
- Function0
Inherited and Abstract methods
Applies the body of this function to the arguments.
Applies the body of this function to the arguments.
Attributes
- Returns
-
the result of function application.
- Inherited from:
- Function0