object ParquetColumnStatisticsProvider extends ColumnStatisticsProvider
Provider for computing column statistics from Parquet files. Attempts to use Parquet metadata for eager statistics. Falls back to lazy column scan if metadata unavailable and useMetadataOnly=false.
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- ParquetColumnStatisticsProvider
- ColumnStatisticsProvider
- AnyRef
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def computeStatistics(path: Path, columnName: String, useMetadataOnly: Boolean = true): Option[Column]
Compute statistics for a single numeric column in a Parquet file.
Compute statistics for a single numeric column in a Parquet file. Prefers Parquet metadata (eager, no row scan). Falls back to lazy column materialization if metadata unavailable.
- path
the path to a .parquet file or dataset directory.
- columnName
the name of the column to analyze.
- useMetadataOnly
if true, return None if metadata unavailable; if false, return lazy thunk.
- returns
an optional Column with eager or lazy statistics.
- Definition Classes
- ParquetColumnStatisticsProvider → ColumnStatisticsProvider
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)