When a user updates a feed’s schema, existing data stored in the source domain table must be updated to match the new shape. Previously, Amperity would force a truncation of the domain table and purge the existing data.
With this product update, feed updates trigger a schema migration workflow to resolve any mismatches between the existing domain data and the new feed configuration. This workflow also occurs on sandbox promotion and pull.
Supported changes
Below is the list of feed updates that are supported by the schema migration workflow.
- Adding a new field
- Removing a field
- Unchecking the “Include” checkbox
- Changing a field's data type. Note: The schema migration job uses the Apache Spark cast function. Expected behavior can be found in Spark’s documentation.
- Renaming a field
- Changing the name of an incoming field
- Modifying a generated field
Unsupported changes
Below are feed updates that cannot run a schema migration job. In these scenarios, a truncation will be forced.
- Changing the fields used in the primary key
- Making an update to a field used as part of the primary key
Updated feed activation options
