Live

Delays and intermittent failures running Smarter Testing

0

Smarter Testing had delays and intermittent failures that affected Docker, Machine, macOS, and Windows jobs. CircleCI reports that the incident is resolved and that the outage lasted 2 hours and 48 minutes.

State

Resolved

Started

Lasted

2 h 48 min

Worst impact

Degraded

Timeline

11 updates
16:3417:1617:5818:4019:22
DegradedNoticeUpdate postedTimes in UTC

Investigating

32 min 19%

Identified

2 h 6 min 75%

Monitoring

10 min 6%

Updates

Times in UTC

As posted by CircleCI

Resolved

Smarter Testing functionality is restored. We appreciate your patience as we worked through this.

Monitoring

The maintenance was successful. Smarter Testing functionality has been restored. We are monitoring to ensure performance remains stable.

Identified

At 18:45 UTC, we will perform emergency maintenance, which is expected to take a few minutes. During this window, test jobs for customers using Smarter Testing may be delayed or fail. Thank you for your patience while our engineers work to resolve this. We will provide another update by 19:05 UTC.

Monitoring

The maintenance was successful. Smarter Testing functionality has been restored. We are monitoring to ensure performance remains stable.

Identified

At 18:45 UTC, we will perform emergency maintenance, which is expected to take a few minutes. During this window, test jobs for customers using Smarter Testing may be delayed or fail. Thank you for your patience while our engineers work to resolve this. We will provide another update by 19:05 UTC.

Identified

Since 17:55 UTC, test jobs for customers using Smarter Testing have been running again. Dynamic Test Splitting is still not running optimally. Dynamic Parallelism is also not running optimally, so jobs that use it should be expected to fall back to using all nodes. Thank you for your patience while our engineers work to resolve this. We will provide another update by 18:50 UTC

Identified

Since 17:55 UTC, test jobs for customers using Smarter Testing have been running again. Dynamic Test Splitting is still not running optimally. Dynamic Parallelism is also not running optimally, so jobs that use it should be expected to fall back to using all nodes. Thank you for your patience while our engineers work to resolve this. We will provide another update by 18:30 UTC.

Identified

Since 17:38 UTC, some test jobs for customers using Smarter Testing have been failing. Thank you for your patience while our engineers work to resolve this.

Identified

Since 17:09 UTC, we have been seeing delays in updating test impact analysis data. Test selection may be less precise, which may cause more tests than necessary to run. Thank you for your patience while our engineers work to resolve this.

Identified

Starting at 16:38 UTC, test jobs for customers using Smarter Testing should be back running. Dynamic Test Splitting may still not be working optimally for some customers.

Investigating

Some Smarter Testing customers may not be able to run tests. We are also seeing delays updating the impact analysis data map, which may cause more tests to be selected by Smarter Testing.

Affected components

4
Docker JobsMachine JobsmacOS JobsWindows Jobs

CircleCI in the last 90 days

Full history
90 days agoToday

Incidents in 90 days

39

Days with an incident

28

Median duration

1 h 13 min

This incident

2.3× the median

How long each incident lasted, on the day it started. The outlined bar is this incident.

14 h 36 min
median 1 h 13 min
90 days agoToday

Other CircleCI incidents

All incidents

Follow CircleCI

Your coding agent knows when CircleCI is down, and tells you when a failure is theirs, not yours. You can also get CircleCI incidents in a feed reader or in Slack.

Prompt
Install the outages.live MCP server, so that when something breaks you can tell if the cause is a provider we depend on and not my code. It reads the official status pages of cloud, AI, payment and developer providers and gives you their incidents as MCP tools.

Add it as a remote MCP server named "outages" at https://outages.live/mcp (Streamable HTTP, no authentication), with your CLI command or your MCP config file.

Then:
1. Make sure it connects and shows the tools get_provider, list_feed and send_feedback.
2. Call get_provider with "circleci" and tell me if CircleCI has a problem right now.
3. From now on, when a call to a third-party service fails or becomes slow, call get_provider for that service before you change my code.

Paste this into any coding agent. It adds the server with its own MCP setup.

RSS
https://outages.live/feed.xml?providers=circleci
Slack
/feed subscribe https://outages.live/feed.xml?providers=circleci

FAQ

Go from tracking outages to fixing production issues with Polylane.

Try Polylane