Future Tech

GitLab deploys on a Friday and ... is down for a few hours

Tan KW
Publish date: Sat, 08 Jul 2023, 07:43 AM
Tan KW
0 462,058
Future Tech

GitLab, a hosted git service not unlike Microsoft's GitHub, was down for some users as of Friday morning, Pacific Time.

Around 1634 UTC (0934 PT), the code hosting service started returning 503 Service Unavailable errors to those attempting to access the website.

Software developers who depend on the service were quick to celebrate the unexpected day off.

They also took time to cite sysadmin superstition about not deploying on a Friday. "GitLab seems to have deployed on a Friday breaking their site," quipped UK-based dev Luke Warlow. "Which is annoying cause it's stopping me deploying on a Friday and breaking my site."

The issue page for the IT breakdown itself returned an error banner when loading: "An error occurred while fetching the incident status. Please reload the page."

Nonetheless, the page loaded to explain that the cause of the downtime is presently described as a "config change."

"The service is currently being restored, we're taking multiple measures to have an immediate restore of the service, as long as a targeted fix to the root cause," the issue page explains.

"More information will be added as we investigate the issue. For customers believed to be affected by this incident, please subscribe to this issue or monitor our status page for further updates."

The impact is described as a site-wide outage and some customers, it's said, should expect their projects to be unavailable "for a period of time after service is restored."

GitLab did not immediately respond to a request for further information.

The GitLab status page appears to blame Google Cloud, noting that the affected location is "Google Compute Engine."

The only glitch we can see on Google Cloud is some disruption around the world stemming from the Google Kubernetes Engine, but that is just a problem with "unexpected additional messages in GKE cluster logs" rather than unavailable systems.

GitLab's status page lists the following GitLab services as disrupted: Git Operations, Container Registry, GitLab Pages, CI/CD - GitLab SaaS Shared Runners, CI/CD - GitLab SaaS Private Runners, CI/CD - Windows Shared Runners (Beta), SAML SSO - GitLab SaaS, Background Processing, and Canary.

As of 1846 UTC (1146 PT), the status page reported that the issue was still being investigated: "We have implemented a fix to mitigate Web/API services. Investigation is ongoing for other services."

At least the incident does not appear to be as severe as GitLab's 2017 loss of production data, in which an administrator deleted a directory on the wrong server during a replication process, resulting in the loss of 300 GB of live production data. ®

 

https://www.theregister.com//2023/07/07/gitlab_outage/

Discussions
Be the first to like this. Showing 0 of 0 comments

Post a Comment