Custom runtimes

Runtimes: shared conceptsDownload PDF (all four runtimes) (12.1 MB)

A custom runtime is an existing runtime plus your own pip packages. Administrators create it on the Custom RTE page.

  1. Upload a requirements.txt and choose the Base Image (the runtime to extend) and a name.
  2. Groundzero builds the runtime with the packages from requirements.txt.
  3. The build status shows as QUEUED, IN PROGRESS, then SUCCESS or FAILURE.
  4. When the build succeeds, the runtime appears in the runtime list.

Guidelines:

  • Extend the runtime you actually run on: a Spark base for Spark jobs, a DuckDB base for DuckDB jobs.
  • Pin versions. Do not change packages the base already includes (NumPy, pandas, scikit-learn) unless you need a different version.
  • Pure-Python packages install most reliably. A package that needs extra system libraries may fail to build. Check the FAILURE status and the package's requirements.
The Custom RTE page in the admin console lists each custom runtime and its build status.
The Custom RTE page in the admin console lists each custom runtime and its build status.
Add custom RTE: choose a base image, name the runtime and upload a requirements file.
Add custom RTE: choose a base image, name the runtime and upload a requirements file.