Skip to content

Configuration

Snowpack is configured through environment variables. Defaults are read from snowpack.config.CompactionConfig unless a component has a separate runtime default.

API

VariableConfig pathDefaultRequiredDescription
SNOWPACK_MAX_CONCURRENT_JOBSmax_workers2noMaximum concurrent in-process maintenance workers per API replica.
SNOWPACK_TABLE_CACHE_REFRESH_SECONDStable_cache_refresh_seconds300noTable inventory refresh cadence in seconds.

API / Worker

VariableConfig pathDefaultRequiredDescription
SNOWPACK_DRAIN_MODEdrain_modefalsenoWhen true/on, the API rejects new maintenance submissions and the worker avoids stale-claim recovery.
SNOWPACK_POSTGRES_HEALTH_CHECK_TIMEOUT_SECONDSpostgres.health_check_timeout_seconds2.0noMaximum seconds readiness waits for a pool connection and SELECT 1 probe.
SNOWPACK_POSTGRES_MAX_CONNECTIONSpostgres.max_connections20noMaximum PostgreSQL pool connections.
SNOWPACK_POSTGRES_MIN_CONNECTIONSpostgres.min_connections4noMinimum PostgreSQL pool connections.
SNOWPACK_SOCKET_TIMEOUT_MSspark.socket_timeout_ms7200000noSpark Thrift socket read timeout in milliseconds.
SNOWPACK_TABLE_CACHE_STALENESS_SECONDStable_cache_staleness_seconds2x SNOWPACK_TABLE_CACHE_REFRESH_SECONDSnoMaximum table-cache age before readiness fails.

API / Worker / CLI

VariableConfig pathDefaultRequiredDescription
SNOWPACK_CATALOGspark.cataloglakehouse_devnoIceberg catalog name used in Spark SQL statements.
SNOWPACK_SPARK_HOSTspark.hostkyuubi-maintenance.data-platform.us-east-1.test-dataops.fetchrewards.comnoSpark Thrift Server / Kyuubi hostname.
SNOWPACK_SPARK_PORTspark.port10009noSpark Thrift Server / Kyuubi port.

API / Worker / Health Sync

VariableConfig pathDefaultRequiredDescription
AWS_REGIONaws_regionus-east-1noAWS region used for Glue and S3 API calls.
SNOWPACK_GLUE_CATALOGglue_catalog_namelakehouse_devnoGlue catalog name used by PyIceberg for direct metadata access.
SNOWPACK_POLARIS_CATALOGpolaris.catalog_nameglue_devnoPolaris REST catalog name.
SNOWPACK_POLARIS_CLIENT_IDpolaris.client_idunsetnoOAuth2 client ID for Polaris REST catalog access.
SNOWPACK_POLARIS_CLIENT_SECRETpolaris.client_secretunsetyesOAuth2 client secret for Polaris REST catalog access.
SNOWPACK_POLARIS_URIpolaris.uriunsetnoPolaris REST catalog URI. Empty falls back to Glue.
SNOWPACK_POSTGRES_DATABASEpostgres.databasesnowpacknoPostgreSQL database name.
SNOWPACK_POSTGRES_HOSTpostgres.hostlocalhostnoPostgreSQL hostname.
SNOWPACK_POSTGRES_PASSWORDpostgres.passwordrequiredyesPostgreSQL password.
SNOWPACK_POSTGRES_PORTpostgres.port5432noPostgreSQL port.
SNOWPACK_POSTGRES_USERpostgres.usersnowpacknoPostgreSQL username.

Health Sync

VariableConfig pathDefaultRequiredDescription
SNOWPACK_HEALTH_SYNC_CONCURRENCYhealth_sync_concurrency10noMaximum concurrent PyIceberg table loads for health sync.
SNOWPACK_HEALTH_SYNC_DATABASEShealth_sync_databasesallnoComma-separated databases to sync health for. Empty means all databases.
SNOWPACK_MIMIR_ENDPOINTmimir_endpointunsetnoOTLP gRPC endpoint for Mimir metrics push. Empty disables push.

Orchestrator

VariableConfig pathDefaultRequiredDescription
SNOWPACK_API_URL-http://snowpack-api.snowpack.svc.cluster.local:443noSnowpack API base URL used for table health checks and job submission.
SNOWPACK_DRY_RUN-falsenoWhen true, logs orchestration decisions without submitting maintenance jobs.
SNOWPACK_HEALTH_CONCURRENCY-10noMaximum concurrent health check requests during orchestration.
SNOWPACK_MAX_SUBMIT-3noMaximum maintenance jobs submitted by one orchestrator run.
SNOWPACK_POLL_INTERVAL-30noSeconds between job status polls while the orchestrator waits for submitted jobs.
SNOWPACK_SLACK_WEBHOOK_URL-unsetnoSlack incoming webhook URL for orchestrator summaries. Optional.

Orchestrator, API

VariableConfig pathDefaultRequiredDescription
SNOWPACK_INCLUDE_DATABASESinclude_databasesunsetnoComma-separated database allowlist. The orchestrator maintains only these databases; the API also surfaces it on GET /tables so the UI greys out databases outside the allowlist.
SNOWPACK_MAINTENANCE_CADENCE_HOURS-6noGlobal minimum hours between automated maintenance runs for one table.
SNOWPACK_OPT_IN_MODE-falsenoOrchestrator enrollment mode. When false (opt-out, the default), every table in the database allowlist is maintained unless it sets snowpack.maintenance_enabled=false or compaction_skip=true. When true (opt-in), only tables with snowpack.maintenance_enabled=true are maintained.

Worker

VariableConfig pathDefaultRequiredDescription
OTEL_EXPORTER_OTLP_ENDPOINT-unsetnoOTLP endpoint used by worker pods for push metrics. Leave unset to disable.