config.py File Reference

Loads environment-based configuration for pipeline and services. More...

Go to the source code of this file.

Namespaces
namespace	scripts
namespace	scripts.config

Functions
	scripts.config.env (key, default=None)
	Retrieve an environment variable with an optional default.

Variables
	scripts.config.CLICKHOUSE_HOST = env("CLICKHOUSE_HOST", "localhost")
	scripts.config.CLICKHOUSE_TCP_PORT = int(env("CLICKHOUSE_TCP_PORT", 9000))
	scripts.config.CLICKHOUSE_HTTP_PORT = int(env("CLICKHOUSE_HTTP_PORT", 8123))
	scripts.config.CLICKHOUSE_USER = env("CLICKHOUSE_USER", "default")
	scripts.config.CLICKHOUSE_PASSWORD = env("CLICKHOUSE_PASSWORD", "")
	scripts.config.CLICKHOUSE_HOST_DOCKER = env("CLICKHOUSE_HOST_DOCKER", "clickhouse")
	scripts.config.CLICKHOUSE_TCP_PORT_DOCKER = int(env("CLICKHOUSE_TCP_PORT_DOCKER", 9000))
	scripts.config.CLICKHOUSE_HTTP_PORT_DOCKER = int(env("CLICKHOUSE_HTTP_PORT_DOCKER", 8123))
	scripts.config.DB_PATH = Path(env("DB_PATH", "db/db.parquet"))
	scripts.config.SAMPLE_PATH = Path(env("SAMPLE_PATH", "samples/db_sample.parquet"))

Detailed Description

Loads environment-based configuration for pipeline and services.

Description

Central configuration module that reads environment variables via dotenv and exposes all necessary settings for:

• ClickHouse connections
• Local and Docker-based port mapping
• Global Parquet file paths for pipeline storage

This module wraps os.getenv() via the env() helper, enabling fallback defaults and central management of required keys.

Usage

from scripts.config import CLICKHOUSE_HOST, DB_PATH, ... Parquet writers/readers will use DB_PATH and SAMPLE_PATH. CLI runners and ingestion tools will use ClickHouse config values.

Notes

• Loads from .env file in the project root (via python-dotenv)
• All paths are converted into Path() objects for consistency
• Ports are cast to int to prevent runtime casting bugs

Definition in file config.py.