On August 30, 2023, between 4:07 and 5:30 UTC, some customers were unable to login to Atlassian's Cloud products using id.atlassian.com. Logged-in users were also unable to switch accounts, change passwords, or log out. Users with existing sessions were not impacted.
Between 5:32 and 6:00 UTC, traffic was incrementally restored to a previous build, mitigating the impact for users.
The total time to resolution was one hour and 53 minutes.
Users were not able to login using Atlassian's shared account management system (id.atlassian.com).
This affected users who were trying to login to the following products: Jira, Confluence, Trello, Opsgenie, mobile apps and ecosystem apps.
Aside from the inability to login, there was no impact on other Atlassian products or features.
Multiple Set-Cookie headers were unintentionally modified so that only the last Set-Cookie header remained in the response to user's browsers. The issue was caused by a change to Network Extensions within the Edge Network. As a result, users that needed a new session could not login. Upon login, the users were redirected to login again and no session was created for them.
We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue was not detected in Atlassian's staging environment. End-to-end tests did not cover the use case of multiple Set-Cookie headers in the single response and therefore this bug went unnoticed.
We are prioritizing the following improvement actions to avoid repeating this type of incident:
Furthermore, we typically deploy our changes progressively by cloud region to avoid broad impact, but in this case, the change was not deemed risky and was deployed to all regions. To minimize the impact of breaking changes to our environments, we will implement additional preventative measures:
We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability.
Atlassian Customer Support