Runtimes: shared concepts
How the Standard, DuckDB, Spark and Lakehouse runtimes compare, how runs are launched and sized, and what every run receives.
Chapters
- IntroductionGroundzero offers four ways to run your data work: the Standard runtime, the DuckDB runtime, the Spark runtime and the Lakehouse SQL engine. This chapter covers what…
- Choosing a runtimeWork down this list and stop at the first match.
- How a run startsThe steps are the same everywhere you can start a run.
- What your code can use during a runEvery run gives your code a fixed set of values. You can read them from os.environ in notebooks and, where listed, in job code.
- Connections and secretsA connection stores how to reach a source (host, database, credentials or role) under a name. Your code refers to the connection by name and never contains credentials.
- The lakehouse (Iceberg)The Groundzero lakehouse stores tables in Apache Iceberg format on object storage, with one warehouse per site. The lakehouse console and its SQL editor run lakehouse…
- Custom runtimesA custom runtime is an existing runtime plus your own pip packages. Administrators create it on the Custom RTE page.
- ML variantsSpark and DuckDB each come as a base runtime and three ML variants. A variant is the base plus extra libraries.
- Jobs: lifecycle, status and logsQueued and Running are active. Success, Failed and Aborted are final.
- Glossary