Core Developer's Guide

This guide is for developers who want to contribute to DeltaFi's core codebase.

Prerequisites

• Docker Desktop with at least 4 cores and 8GB RAM allocated
• Git

Setup

1. Install DeltaFi

Follow the Quick Start to install DeltaFi. This installs the DeltaFi TUI (Terminal User Interface) - a command-line tool that manages the entire DeltaFi system. See Understanding the TUI for how it works, or read on for the basics.

When the installation wizard asks about your role, select Core Development. When asked about orchestration mode, select Compose to start (you can switch later with deltafi config).

This will:

• Set up a DeltaFi environment using Docker Compose
• Clone the DeltaFi repository into the repos/deltafi/ directory

2. Verify the Installation

After setup completes:

deltafi status

The DeltaFi UI will be available at http://local.deltafi.org.

Directory Structure

After installation, your directory will look like:

~/deltafi/                    # Install root (DELTAFI_INSTALL_DIR)
├── deltafi                   # TUI binary
├── deltafi.yaml              # Main configuration
├── VERSION
├── config/                   # Runtime configuration
├── data/                     # Runtime data
├── repos/                    # Source repositories
│   └── deltafi/              # Core repository (cloned automatically)
└── site/                     # Site-specific customization
    ├── compose.yaml          # Docker Compose overrides
    └── values.yaml           # Helm values overrides

Development Workflow

How Deployment Works

Understanding how code gets from your editor to running containers:

For Compose mode:

1. ./gradlew install builds Docker images locally and tags them (e.g., deltafi/deltafi-core:2.x.x)
2. Gradle calls deltafi up --force to restart services with the new images
3. Docker Compose pulls the locally-tagged images and restarts affected containers

For KinD mode:

1. ./gradlew install builds Docker images locally
2. Gradle loads images into the KinD cluster (deltafi kind load <image>)
3. Gradle calls deltafi up --force to restart pods with the new images

Key commands:

• deltafi up - Starts DeltaFi (or restarts with --force to pick up new images)
• deltafi down - Stops all DeltaFi services
• deltafi status - Shows what's running

Building and Deploying Changes

From the repos/deltafi/ directory:

# Build and deploy ALL components
./gradlew install

# Build and deploy just deltafi-core (faster iteration)
./gradlew :deltafi-core:install

# Build and deploy just deltafi-action-kit
./gradlew :deltafi-action-kit:install

# Build the TUI
./gradlew tui

Running Tests

# Run all tests
./gradlew test

# Run tests for a specific module
./gradlew :deltafi-core:test

# Run a specific test class
./gradlew :deltafi-core:test --tests "SomeTestClass"

Key Directories in the Repository

Directory	Description
deltafi-core/	Core platform (Java/Spring Boot)
deltafi-action-kit/	SDK for building actions (Java)
deltafi-core-actions/	Built-in actions
deltafi-common/	Shared types and utilities
tui/	Command-line interface (Go)
charts/	Helm charts for Kubernetes
compose/	Docker Compose configuration

Orchestration Modes

DeltaFi supports two orchestration modes:

Mode	Directory	Use Case
Compose	compose/	Docker Compose deployment (simpler, recommended for most development)
KinD	charts/	Kubernetes in Docker (for testing Helm charts and K8s-specific features)

To switch between modes:

deltafi config

Important: If you're making changes to orchestration files (compose/ or charts/), you generally need to make parallel changes in both and test in both modes. This applies to:

• Service definitions and configurations
• Environment variables
• Volume mounts
• Network settings

Site Customization

The site/ directory contains site-specific overrides that persist across upgrades. Different files are used depending on your orchestration mode:

File	Mode	Purpose
site/values.yaml	Compose	Override default configuration values
site/compose.yaml	Compose	Override Docker Compose settings
site/kind.values.yaml	KinD	Override Helm values for KinD
site/templates/	KinD/K8s	Custom Helm templates

values.yaml (Compose mode)

Override configuration values for Docker Compose deployments:

# site/values.yaml
deltafi:
  core_worker:
    enabled: true
    replicas: 4
  core_actions:
    replicas: 2
  auth:
    mode: basic  # basic, cert, or disabled
  api:
    workers: 16
  dirwatcher:
    enabled: true
    maxFileSize: 4294967296  # 4GB
ingress:
  domain: local.deltafi.org
  tls:
    enabled: true

compose.yaml (Compose mode)

Override Docker Compose settings directly:

# site/compose.yaml
services:
  deltafi-minio:
    ports:
      - 9000:9000  # Expose MinIO to host
  deltafi-core:
    environment:
      - JAVA_OPTS=-Xmx4g

kind.values.yaml (KinD mode)

Override Helm values for KinD deployments:

# site/kind.values.yaml
deltafi:
  core:
    resources:
      requests:
        memory: "2Gi"

Custom Helm Templates (KinD/K8s)

Add custom Kubernetes resources by placing Helm templates in site/templates/. These templates have full access to the DeltaFi chart's helpers and values. See templates_readme.md in the chart for examples.

Changing the Core Repository

During initial setup, the wizard prompts for the core repository URL. To change it later (e.g., switching to an internal fork):

1. Edit config/config.yaml and update the coreRepo value:

development:
  repoPath: /path/to/deltafi/repos
  coreRepo: git@gitlab.com:your-org/deltafi.git

2. Update the git remote in your existing clone:

cd repos/deltafi
git remote set-url origin git@gitlab.com:your-org/deltafi.git
git fetch origin

Submitting Changes

1. Create a Branch

cd repos/deltafi
git checkout -b feature/my-feature

2. Make Changes and Test

./gradlew install
./gradlew test

3. Add a Changelog Entry

bin/changelog -e

This creates a file in CHANGELOG/unreleased/<branch-name>.md.

4. Submit a Merge Request

Push your branch and create a merge request on GitLab targeting main.

Useful Commands

Command	Description
deltafi status	Check system health
deltafi up	Start DeltaFi
deltafi down	Stop DeltaFi
deltafi dashboard	View metrics
deltafi search	Search DeltaFiles

Reference

• TUI Reference - Command-line interface documentation
• Contributing - Merge request process
• Architecture - System design