Migrating to DeltaFi 3.0
In progress
DeltaFi 3.0 has not been released. This guide is assembled on the 2.x baseline as each change lands, so the entries below cover only the work that has already merged. Coming later lists what is still ahead.
3.0 is a fresh install
There is no in-place upgrade path from 2.x to 3.0. A 3.0 system is deployed fresh, and DeltaFiles in flight on a 2.x system are not carried across. Plan the cutover as a drain-and-redeploy: stop ingress, let the running DeltaFiles finish, export the configuration you want to keep, then install 3.0 and import it.
That policy removes the data-migration half of the work, but not the configuration half. Plugins built against the 2.x SDKs, Helm values and environment overrides that name internal services, and GraphQL or REST clients all still need the changes below.
How to use this guide
Every entry has the same two halves:
- Prepare on 2.x — what you can change today, on a current 2.x release, without losing any function. A plugin, configuration file, or client that has done this work runs unchanged on both 2.x and 3.0, so the cutover carries no code changes with it.
- Changes in 3.0 — what is removed or behaves differently once 3.0 ships. Until then the 2.x surface keeps working, usually with a deprecation warning at compile time.
Do the Prepare on 2.x half of every entry before you cut over. What remains after that is the work that cannot be done early, and it should be a short list.
Coming later
These changes have not reached the 2.x baseline yet, so there is nothing to prepare for them. They are listed so operators and plugin developers can see what is still ahead.
| Change | Audience | Issue |
|---|---|---|
dgs result queues renamed to core across core and every action kit | Operators | #678 |
deltafi-action-kit renamed deltafi-springboot-action-kit | Plugin developers | #679 |
| Technology names decoupled from services, environment variables, values, and CLI | Operators | #680 |
| Ingress service, exception layer, and REST path renamed to DataSource terminology | Operators, API consumers | #681 |
createdAt/updatedAt normalized to created/modified on users, roles, and user snapshots | API consumers | #682 |
| FlowFile egress defaults to V3 | Operators | #673 |
| Core and analytics backward-compat shims removed | Operators | #671 |
For plugin developers
Action unit tests move to DeltaFiTestRunner
The test framework that predates DeltaFiTestRunner is deprecated in full: the ActionTest / TestCaseBase hierarchy, its TransformActionTest and EgressActionTest subclasses with their *TestCase builders, the IOContent / Child / Enrichment input holders, the ActionResultAssertions static factory, and a handful of methods on the typed asserters.
Prepare on 2.x: rewrite each action test to build its own input and call the action directly, with DeltaFiTestRunner supplying content storage and an ActionContext, and assert through the typed *ResultAssert classes. Everything the rewrite needs already ships in deltafi-action-kit-test on 2.x.
Changes in 3.0: the classes and methods below are deleted. A test still using them will not compile.
Replacements:
| Deprecated | Replacement |
|---|---|
ActionTest, TestCaseBase | Call the action directly; take content and context from DeltaFiTestRunner |
TransformActionTest, TransformActionTestCase | DeltaFiTestRunner plus TransformResultAssert / TransformResultsAssert |
EgressActionTest, EgressActionTestCase | DeltaFiTestRunner plus EgressResultAssert |
IOContent, Child, Enrichment | DeltaFiTestRunner.saveContent(...) and saveContentFromResource(...) |
ActionResultAssertions.assertTransformResult(result) | TransformResultAssert.assertThat(result) |
ActionResultAssertions.assertTransformResults(result) | TransformResultsAssert.assertThat(result) |
ActionResultAssertions.assertErrorResult(result) | ErrorResultAssert.assertThat(result) |
ActionResultAssertions.assertFilterResult(result) | FilterResultAssert.assertThat(result) |
ActionResultAssertions.assertEgressResult(result) | EgressResultAssert.assertThat(result) |
ContentResultAssert.hasContentMatching(predicate) | hasContentMatchingAt(index, predicate) |
ContentResultAssert.contentLoadBytesEquals(list) | hasContentMatchingAt(index, name, mediaType, bytes), once per content |
ContentResultAssert.contentLoadStringEquals(list), contentLoadStringEquals(list, charset) | hasContentMatchingAt(index, name, mediaType, string), once per content |
ContentResultAssert.contentIsNotNull() | Assert on the content itself — hasContentCount(count), or a hasContentMatchingAt overload |
MetadataResultAssert.addedMetadataEquals(map) | addedMetadata(map) |
MetadataResultAssert.deletedKeyEquals(keys...) | deletedMetadataKeys(List), or one deletedMetadataKey(key) per key |
TransformResultsAssert.hasChildResultAt(index, predicate) | Any other hasChildResultAt overload |
ResultAssert.getResult() | Keep the result in a local variable and assert on it directly |
Subclassing becomes a direct call
The deprecated base classes handed the test a context() method and an execute entry point. DeltaFiTestRunner replaces both: runner.actionContext() is the context, and the test calls the action's own method.
Before:
import org.deltafi.test.action.transform.TransformActionTest;
import static org.deltafi.test.asserters.ActionResultAssertions.*;
class JoltTransformActionTest extends TransformActionTest {
JoltTransformAction action = new JoltTransformAction();
@Test
void testJoltTransformByIndex() {
JoltParameters params = new JoltParameters();
params.setJoltSpec("[{\"operation\": \"shift\", \"spec\": {\"original\": \"new\"}}]");
params.setContentIndexes(List.of(0));
TransformInput input = createInput();
ResultType result = action.transform(context(), params, input);
assertTransformResult(result)
.hasContentCount(1)
.hasContentMatchingAt(0, this::checkContent);
}
private TransformInput createInput() {
List<ActionContent> content = List.of(ActionContent.saveContent(
context(), "{\"original\": \"value\"}", "example.json", "application/json"));
return TransformInput.builder().content(content).build();
}
}After:
import org.deltafi.test.asserters.TransformResultAssert;
import org.deltafi.test.content.DeltaFiTestRunner;
class JoltTransformTest {
JoltTransform action = new JoltTransform();
DeltaFiTestRunner runner = DeltaFiTestRunner.setup();
@Test
void testJoltTransformByIndex() {
JoltParameters params = new JoltParameters();
params.setJoltSpec("[{\"operation\": \"shift\", \"spec\": {\"original\": \"new\"}}]");
params.setContentIndexes(List.of(0));
TransformInput input = createInput();
ResultType result = action.transform(runner.actionContext(), params, input);
TransformResultAssert.assertThat(result)
.hasContentCount(1)
.hasContentMatchingAt(0, this::checkContent);
}
private TransformInput createInput() {
return TransformInput.builder()
.content(List.of(runner.saveContent(
"{\"original\": \"value\"}", "example.json", "application/json")))
.build();
}
}The class extends nothing, so it also drops the @ExtendWith(MockitoExtension.class), @InjectMocks, and @Component annotations the base class required.
Test cases become explicit calls
A TransformActionTestCase described the input, the parameters, and the expected result in one builder, and execute compared the whole action event. Split that into an input, a call, and typed assertions on the parts the test actually cares about.
Before:
class ModifyMetadataTransformActionTest extends TransformActionTest {
@InjectMocks
ModifyMetadataTransformAction action;
@Test
void testTransform() {
execute(TransformActionTestCase.builder()
.action(action)
.parameters(Map.of("addOrModifyMetadata", Map.of("key1", "value1", "key2", "value2"),
"copyMetadata", Map.of("origKey1", "key3, key4 ,key5"),
"deleteMetadataKeys", List.of("origKey2", "origKey3", "key5")))
.inputs(List.of(IOContent.builder().name("test.txt").contentType("*/*").build()))
.sourceMetadata(Map.of("origKey1", "origVal1", "origKey2", "origVal2"))
.expectTransformResult(List.of(IOContent.builder().name("test.txt").contentType("*/*").build()))
.resultMetadata(Map.of("key1", "value1", "key2", "value2", "key3", "origVal1", "key4", "origVal1"))
.resultDeleteMetadataKeys(List.of("origKey2"))
.build());
}
}After:
class ModifyMetadataTest {
ModifyMetadata action = new ModifyMetadata();
DeltaFiTestRunner runner = DeltaFiTestRunner.setup();
@Test
void testTransform() {
ModifyMetadataParameters params = new ModifyMetadataParameters();
params.setAddOrModifyMetadata(new HashMap<>(Map.of("key1", "value1", "key2", "value2")));
params.setCopyMetadata(Map.of("origKey1", "key3, key4 ,key5"));
params.setDeleteMetadataKeys(List.of("origKey2", "origKey3", "key5"));
TransformInput input = TransformInput.builder()
.metadata(Map.of("origKey1", "origVal1", "origKey2", "origVal2"))
.build();
ResultType result = action.transform(runner.actionContext(), params, input);
TransformResultAssert.assertThat(result)
.addedMetadata(Map.of("key1", "value1", "key2", "value2", "key3", "origVal1", "key4", "origVal1"))
.deletedMetadataKey("origKey2");
}
}Parameters become the action's own typed parameter class instead of a Map, so a renamed or retyped parameter fails at compile time rather than inside an assertion.
Tests that read their input from src/test/resources keep doing so: DeltaFiTestRunner.setup("my-folder") sets the resource folder, and runner.saveContentFromResource("input.json") saves one or more of those files as content.
See Action Unit Testing for the full DeltaFiTestRunner and asserter reference.
For operators
Entries land here as each operator-visible change reaches the 2.x baseline. See Coming later for what is ahead.
For API consumers
Entries land here as each GraphQL and REST change reaches the 2.x baseline. See Coming later for what is ahead.
For contributors
Entries land here as each convention change reaches the 2.x baseline. See Coming later for what is ahead.