Updated Despite what Oracle supremo Larry Ellison might have you believe, Big Red's clouds do in fact go down.

Yesterday some Oracle engineers' Valentine's Day celebrations were perhaps put on hold after the NetSuite enterprise resource planning (ERP) suite experienced an outage at its datacenter in Boston, USA. The downtime, which began around 1215 ET Tuesday, stretched on before services were restored around 1146 today.

The Register spoke with one New York-based military equipment manufacturer via email, who told us the outage took out everything but the phones, with customer service, finance, shipping/receiving, and shop floor all grinding to a halt.

According to complaints shared via Reddit, the outage effectively paralyzed businesses relying on the platform's Boston-based servers. We've seen downtime reports from organizations across America: Tennessee, Wisconsin, Chicago, Texas, North Carolina, Minnesota, Vermont, Nevada, Washington, Louisiana, Delaware, Georgia, and Arizona, at least.

We do note that NetSuite has multiple regions in the United States and some of the world, which weren't directly affected by this particular outage. On February 13, US Ashburn 2 went down for about an hour. Oracle right now claims Netsuite had a 99.97 percent availability over the past 12 months.

While NetSuite, which Oracle acquired in 2016 for $9.3 billion, is now back up and running in that Boston datacenter, the tech goliath notes customers may experience errors on their first attempt logging back in. Notably missing from the status report were any warnings about lost data.

However, Oracle reportedly told one Redditor that approximately 30 minutes of data was lost during the recovery process, as the "restoration point was about 30 minutes prior to the outage."

The IT giant is said to be working with customers to identify transactions that were created in the 30-minute window.

According to Oracle's NetSuite status page, the cause of the outage was the result of an unspecified event in the Boston area that "forced the local team to disconnect power until the issue is resolved."

It's not clear exactly what the event that necessitated cutting power to the servers was. We've reached out to Oracle and the colocation provider believed to house the mega-corp's infrastructure for comment and they declined to go on the record.

What we do know is the problem at the Boston datacenter was bad enough that the database supplier began restoring accounts to a disaster recovery site around 2226 ET Tuesday.

The outage flies in the face of Ellison's rather hyperbolic claims made during the company's Q2 earnings call in December. "Our cloud is very secure and extremely reliable," he told financial analysts. "It doesn't go down."

But even at the time, the billionaire's comments overlooked an outage just months earlier. Amid Britain's heatwave this summer, both Google and Oracle cloud datacenters were taken offline by cooling failures after temperatures creeped about 40.3˚C (104.5˚F). ®

Updated to add

Cyxtera, the colocation provider housing Oracle's NetSuite Boston servers, has handed us the following statement, blaming what sounds like a fire in electrical equipment at that datacenter site:

Thus, it now appears NetSuite's Boston outage was caused by the power being cut to servers after firefighters showed up to the smoking datacenter facility.

https://www.theregister.com//2023/02/15/oracle_netsuite_down/