Clarity - Outbound Call Issues

Incident Report for Cloudli Communications

Postmortem

Incident Report

Services Affected: Clarity Outbound Calling

Start Date/Time: 9:16AM (EST), November 24, 2025

End Date/Time: 10:10AM (EST), November 24, 2025

Summary of Events

Clarity’s dynamic call path system failed on a small number of customer facing servers resulting in down stream systems failing to process outbound calls to the PSTN.

Monitoring and Alerting systems notified our L3-NOC staff whom immediately engaged our incident response procedures and began vaildating the issue. Upon confirmation that the issue was isolated to customers still on the Clarity platform we began isolating systems affected and confirmed that the dynamic call path system was failing.

Our L3-NOC engaged development to review a proposed patch and approved the patch to stabilize the platform. We then immediately began the patching process completing this for all servers (both impacted and non-impacted) focusing on the affected systems. This deployment began at 9:50AM (EST) and was completed for impacted servers at 10:00AM (EST) we immediately began seeing outbound calls flowing for affected servers at that time. We continued to patch non-impacted servers completing at 10:10AM (EST) at which we moved the incident into a monitoring status.

Mitigation Measures

In order to mitigate this issue we have modified a system process that performs a health check on Sunday evenings to include a vaildation that the dynamic call path system is in a healthy replication state. This will ensure that this edge case does not occur again.

Final Remarks

At Cloudli, we take any interruption of service very seriously and are continuously evaluating new processes and mitigation measures that can be proactively implemented to ensure service continuity.

When service interruptions do occur, our incident management procedure prioritizes prompt and clear notification and timely status and resolution updates to our customers and partners.

We thank you for your continued support. Please feel free to reach out if you would like to discuss the particulars of this incident report further.

Posted Nov 24, 2025 - 13:13 EST

Resolved

This incident has been fully resolved.
Posted Nov 24, 2025 - 12:35 EST

Monitoring

As of 10:10 (EST) service is restored.

The issue was directly related to the Clarity dynamic call path systems and their subsequent backend dependencies. We will provide a written update after two(2) hours of monitoring the system to ensure full resolution. The written update will be provided after the period of monitoring is complete.
Posted Nov 24, 2025 - 10:21 EST

Identified

The issue has been identified and is related to some HA Pairs in Clarity. We are in the process of correcting the HA Pairs that are failing. We will continue to porovide updates as they become available.
Posted Nov 24, 2025 - 10:09 EST

Investigating

We are currently experiencing outbound call issues in Clarity.
Posted Nov 24, 2025 - 10:01 EST
This incident affected: Clarity (Outbound Calling).