Skip to main content

Handling Event Schema Changes: Implementing Upcasters to Fix Deserialization Errors

 You have successfully implemented Event Sourcing using the Axon Framework. Your event store is immutable, capturing every state change your application has ever seen. It works perfectly until your domain logic evolves.

You add a new mandatory field to an existing event class. You deploy the new version. Suddenly, your replay fails, or your projection groups stall. The logs are flooded with serialization exceptions:

com.fasterxml.jackson.databind.exc.MismatchedInputException: Missing required creator property 'currency'

This is the "Schema Evolution" problem. Since you cannot modify the historical events in your append-only store, you must bridge the gap between the old data structure and your new Java class definition.

The solution is not to relax your serialization rules, but to implement Upcasters.

The Root Cause: The Immutability Paradox

In a CRUD-based system, schema changes are handled via database migration scripts (e.g., Flyway or Liquibase). You simply run ALTER TABLE orders ADD COLUMN currency VARCHAR(3) DEFAULT 'USD' and update all existing rows.

In Event Sourcing, this is impossible.

  1. Immutability: Events represent historical facts. You cannot change history. You cannot run an UPDATE statement on a serialized blob inside the Axon Server or RDBMS event store.
  2. Strict Deserialization: Your modern application code expects the OrderCreatedEvent to represent the current business rules (V2), but the database contains JSON/XML representing the rules from two years ago (V1).

When the JacksonSerializer (or XStream) attempts to hydrate the V1 JSON into the V2 Java object, it fails because the payload lacks the data required by the new constructor or fields.

We resolve this by intercepting the raw data stream before it reaches the deserializer using an Upcaster.

The Solution: Implementing a SingleEventUpcaster

We will fix this by creating an Upcaster that transforms the serialized form of the event.

Scenario

Imagine an E-commerce system. Version 1 (Old): The OrderCreatedEvent only tracked an ID and an amount. The currency was implicitly "USD". Version 2 (New): We are going global. The OrderCreatedEvent now strictly requires a currency field.

Step 1: The Domain Event Evolution

Here is the transition in our Java code.

The Original Event (V1 - Deprecated conceptually, but exists in DB):

{
  "orderId": "12345",
  "amount": 100.00
}

The New Event Class (V2):

package com.example.ecommerce.coreapi.events;

import java.math.BigDecimal;

public class OrderCreatedEvent {
    private final String orderId;
    private final BigDecimal amount;
    private final String currency; // New mandatory field

    // Constructor annotated for Jackson
    public OrderCreatedEvent(String orderId, BigDecimal amount, String currency) {
        this.orderId = orderId;
        this.amount = amount;
        this.currency = currency;
    }

    // Getters...
}

If we try to load the V1 JSON into this V2 class, it crashes because currency is null or missing.

Step 2: Writing the Upcaster

We will implement a SingleEventUpcaster. This class sits in the Axon deserialization chain. It takes the intermediate representation (usually JsonNode if using Jackson), manipulates it, and passes it downstream.

We will inject a default value of "USD" into old events.

package com.example.ecommerce.upcasting;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import org.axonframework.serialization.SimpleSerializedType;
import org.axonframework.serialization.SerializedType;
import org.axonframework.serialization.upcasting.event.IntermediateEventRepresentation;
import org.axonframework.serialization.upcasting.event.SingleEventUpcaster;

public class OrderCreatedEventUpcaster extends SingleEventUpcaster {

    private static final SerializedType TARGET_TYPE = 
        new SimpleSerializedType("com.example.ecommerce.coreapi.events.OrderCreatedEvent", null);

    @Override
    protected boolean canUpcast(IntermediateEventRepresentation representation) {
        // We only want to upcast "OrderCreatedEvent" where the revision is null (V1)
        return representation.getType().getName().equals(TARGET_TYPE.getName())
                && representation.getType().getRevision() == null;
    }

    @Override
    protected IntermediateEventRepresentation doUpcast(IntermediateEventRepresentation representation) {
        return representation.upcast(
            // The new type definition (we can add a revision number "2.0" if we want, or keep it null)
            TARGET_TYPE, 
            // The expected input type for manipulation (JsonNode)
            JsonNode.class, 
            // The transformation logic
            this::addCurrencyField
        );
    }

    private JsonNode addCurrencyField(JsonNode eventData) {
        if (eventData instanceof ObjectNode) {
            ObjectNode objectNode = (ObjectNode) eventData;
            // Set default currency to USD for all historical events
            objectNode.put("currency", "USD"); 
            return objectNode;
        }
        throw new IllegalArgumentException("Event data is not a valid JSON Object");
    }
}

Step 3: Registering the Upcaster

Axon needs to know this upcaster exists. In a Spring Boot environment, simply defining it as a @Bean is usually sufficient, as Axon auto-configures the EventUpcasterChain.

package com.example.ecommerce.config;

import com.example.ecommerce.upcasting.OrderCreatedEventUpcaster;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AxonConfig {

    @Bean
    public OrderCreatedEventUpcaster orderCreatedEventUpcaster() {
        return new OrderCreatedEventUpcaster();
    }
}

Deep Dive: How the Upcaster Pipeline Works

Understanding the pipeline is critical for performance tuning and debugging.

When your application requests an event stream (e.g., during a projection rebuild or a Saga initialization), the data flow is as follows:

  1. Fetch: The raw bytes are retrieved from the Event Store (Axon Server/Postgres).
  2. Upcast: Before deserialization, the raw bytes are wrapped in an IntermediateEventRepresentation.
  3. Chain Execution: The EventUpcasterChain iterates through all registered upcasters.
  4. Match: Your canUpcast method checks the event type. If it matches, doUpcast is triggered.
  5. Transform: The JSON structure is modified in memory.
  6. Deserialize: The modified JSON is finally passed to the Serializer (Jackson), which maps it to the Java Pojo (V2).

Key Takeaway: The database is never updated. The upcasting happens lazily in memory every time the event is read. This preserves the immutable audit trail while allowing the application code to evolve.

Handling Revisions and Complexity

In the example above, we checked for revision == null. In production systems, explicit revisioning is safer.

Using Explicit Revisions inside @Revision

Ideally, decorate your Event classes with the @Revision annotation.

V1 Class (Historical): No annotation (implicitly revision null).

V2 Class (Current):

@Revision("2.0")
public class OrderCreatedEvent { ... }

Updated Upcaster Logic:

@Override
protected boolean canUpcast(IntermediateEventRepresentation representation) {
    SerializedType type = representation.getType();
    // Upcast if type is OrderCreatedEvent AND revision is null
    return "com.example.ecommerce.coreapi.events.OrderCreatedEvent".equals(type.getName()) 
           && type.getRevision() == null;
}

@Override
protected IntermediateEventRepresentation doUpcast(IntermediateEventRepresentation representation) {
    // Transform to revision "2.0"
    SerializedType newType = new SimpleSerializedType(
        "com.example.ecommerce.coreapi.events.OrderCreatedEvent", "2.0"
    );
    
    return representation.upcast(newType, JsonNode.class, this::addCurrencyField);
}

Common Pitfalls

1. Renaming Classes

If you rename the Java class from OrderCreatedEvent to OrderPlacedEvent, the serialized type in the database will still point to the old name.

You must create an upcaster that changes the Type Name, not just the content.

return representation.upcast(
    new SimpleSerializedType("com.new.package.OrderPlacedEvent", "2.0"),
    JsonNode.class,
    json -> json // No content change, just type name change
);

2. Chaining Upcasters

If you go from V1 -> V2 -> V3, do not write a single massive upcaster that handles V1 -> V3. Write one for V1 -> V2 and another for V2 -> V3. Axon will automatically chain them. This keeps your code modular and testable.

3. Performance Overhead

Upcasting incurs a CPU cost during serialization. If you have millions of events and heavy upcasting logic, replay times will increase. If the schema change is drastic and permanent, consider "Snapshotting" your aggregates to reduce the number of events that need to be read (and subsequently upcasted) during loading.

Conclusion

Schema changes in Event Sourcing are inevitable. While serialization errors can be intimidating, the Upcaster pattern provides a clean, robust mechanism to handle backward compatibility.

By manipulating the serialized form of the event before it reaches your Java classes, you maintain the sanctity of your immutable event log while allowing your domain model to adapt to new business requirements. Always ensure your upcasters are registered as beans and rely on specific revision checks to prevent unintended data corruption.

Labels: