🚨 Action Required: Stop BPM Archival Service in Production

Neutrinos Platform
1

:police_car_light: Action Required: Stop BPM Archival Service in Production

We have identified a critical issue in the BPM archival process that can potentially lead to data corruption.

Required Action

  • Immediately stop the BPM archival service in the production environment
  • Do not restart the service until further communication

Status

  • Issue is under active investigation
  • Updates and next steps will be shared once the fix is validated

Please treat this as high priority.

— Alpha

2 Likes
2

Hi Team,

Our investigation indicates that the underlying data inconsistency existed prior to the backup operation in Development Environment. At the time the backup was taken, certain task records were already missing, even though the backup process itself completed successfully.

During archival, the system performs a validation check that allows task events to be deleted only when a corresponding valid task exists. Since the task records were already absent, the related task events were retained. When the data was subsequently restored, these retained task events resulted in duplicate ID conflicts.

At this stage, there is no indication that the backup mechanism itself introduced corruption; rather, the issue surfaced due to pre-existing data inconsistencies combined with the current archival validation logic.

We are working on:

  • Implementing a patch to handle scenarios where tasks have been deleted prior to backup or archival, even though such data corruption cases are considered edge cases

We plan to release this patch by Wednesday (14-01-2026), taking this scenario into consideration as well.

Regards,
Alpha Team

1 Like
3

:wrench: Alpha Archival Hotfix

Release Date: January 5, 2026
Release Channel: LTS & CURRENT

We’ve released a hotfix for the Alpha Platform to address an issue in Alpha Archival.

:lady_beetle: Bug Fixes

  • Fixed an issue where orphan task-related records (such as taskevent and content) were included in backups even after tasks were deleted.
    This caused restore failures due to key conflicts.

  • Fixed an issue where the remaining process count in audit did not decrease correctly because orphan audit/task records were still being counted after processes were deleted.

  • The updated_on field was not being updated when an audit record was updated. This has now been fixed to correctly reflect the latest audit update time.
    Result:
    Only valid task and process records are now considered during archival and audit processing, ensuring reliable restores and accurate remaining process counts.

  • The last audit row was incorrectly copying processInstanceIds and childProcessInstanceIds from the previous batch. This issue has been resolved.

  • Backup and Restore APIs now support a maxProcessCount parameter, allowing consumers to limit the maximum number of process instances processed in a request.

  • An issue with date filtering in the Audits APIs has been fixed to ensure correct and consistent behavior when applying from and to filters.

  • An issue causing timeouts during calls to the IDS Introspect API has been fixed to improve stability and reliability.

  • Resolved an issue where backup execution failed when data contained handlebar template-like ({{ }}) patterns. SQL execution has been made safer to prevent data-dependent parsing errors.

Docker Images

Service Name Image Repository Image Version
alpha-archival neutrinos.azurecr.io/alpha/alpha-archival 26.1.0.0.35-35f9c19517-release-26.01.16.0.0

:telephone_receiver: Support & Resources

1 Like