ML variants

Runtimes: shared conceptsDownload PDF (all four runtimes) (12.1 MB)

Spark and DuckDB each come as a base runtime and three ML variants. A variant is the base plus extra libraries.

Variant Adds
Base Spark or DuckDB, Groundzero helper libraries, scikit-learn 1.5.2, CPU PyTorch 2.14.0 and torchvision 0.29.0, document parsing
sklearn OpenCV (headless), XGBoost, LightGBM, CatBoost
pytorch sklearn extras, PyTorch 2.5.1 and torchvision 0.20.1 (GPU build), SHAP, LIME, Albumentations, Hugging Face Hub and Transformers, simsimd, fastai
tensorflow TensorFlow, OpenCV, XGBoost, LightGBM, CatBoost, SHAP, LIME, NLTK, spaCy, h5py

Notes:

  • The pytorch variant uses the GPU build of PyTorch, so GPU-sized runs can train on GPUs.
  • Variants appear in the runtime list with names such as gz-rte-duckdb-sklearn and a version tag such as 1.0.0.
  • The Standard runtime has no ML variants.