o

com.phasmidsoftware.tableparser.parquet

ParquetSchemaValidator

object ParquetSchemaValidator

Validates a Parquet MessageType schema against a target case class, prior to reading any rows.

Supports both flat case classes (all fields map directly to Parquet columns) and grouped case classes (some fields are sub-case-classes, each of which maps its own fields to Parquet columns via its own ColumnHelper).

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ParquetSchemaValidator
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  8. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  9. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  11. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  13. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  15. def toString(): String
    Definition Classes
    AnyRef → Any
  16. def validate[Row](schema: MessageType, helper: ColumnHelper[Row], groupedHelpers: Map[String, (ClassTag[_], ColumnHelper[_])] = Map.empty)(implicit arg0: ClassTag[Row]): Try[Unit]

    Validates a Parquet MessageType schema against the target case class Row.

    Validates a Parquet MessageType schema against the target case class Row.

    Checks: - Every primitive parameter of Row has a corresponding column in the Parquet schema (after ColumnHelper name mapping is applied). - Parameters whose type is a Product sub-class are validated recursively using the provided groupedHelpers map. - Every matched column has a supported Parquet type (via ParquetTypeMapper). - Any OPTIONAL Parquet column must map to an Option[T] parameter in Row.

    Columns present in the Parquet schema but absent from Row are silently ignored, consistent with existing CSV behaviour.

    Row

    the target case class type.

    schema

    the MessageType read from the Parquet file footer.

    helper

    the ColumnHelper providing parameter-to-column name mapping for Row.

    groupedHelpers

    a map from field name to (ClassTag, ColumnHelper) for grouped sub-case-class fields. Defaults to empty (flat schema behaviour).

    returns

    Success(()) if valid, Failure(ParquetParserException) if not.

  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from AnyRef

Inherited from Any

Ungrouped