Updated The Xen Project - creator and manager of the open-source Xen hypervisor and associated tools - has warned its community of potential problems flowing from the imminent closure of the colocation facility it uses.

“We regret to inform you that the Xen Project is currently experiencing unexpected changes due to the sudden shutdown of our colocated data center facility by Synoptek,” opens a Wednesday post penned by community manager Kelly Choi.

“This incident is beyond our control and will impact the continuity of OSSTest (the gating Xen Project CI loop), which we heavily rely on as a project,” Choi added.

OSSTest is a custom tool the project has used for over a decade to check its code.

Choi therefore bemoaned the “inconvenience this may cause to our users and stakeholders” and assured them “we are working diligently to consider different solutions as swiftly as possible.”

But probably not swiftly enough to avoid problems, as Choi warned the instance of OSSTest housed at Synoptek “will likely be disrupted with an anticipated shutdown date of October 31st, 2024.”

One might think the migration of something like OSSTest from one colo to another would not be too difficult for a hypervisor maker, in that the test application should surely be virtualized so that its guests can be reasonably run anywhere. But judging from the documentation, the system involves automated testing of the bare-metal hypervisor on a decent variety of hosts, to ensure it'll work as it's intended to be used. Thus a migration would involve shifting or replacing a pool of hardware and networking in a colo move.

“The potential consequence of this shutdown may mean availability of OSSTest will be impacted, therefore affecting the number of bugs being caught,” Choi added. “Contributors may experience quality issues with code, or face slower response times as our technical community spends more time resolving issues.”

Choi’s post explains the project is considering the following options:

None of those options appear palatable, as Choi writes the project is not sure its hardware would survive a move. Nor is the org confident it could acquire new hardware and replicate OSSTest, its custom automated testing tech.

“As a last resort, abandoning OSSTest has been discussed,” Choi revealed. “The project has no replacement for test coverage similar to what OSSTest provides. Even if Gitlab CI expanded test coverage similar to what OSSTest currently covers, some features are unlikely to be replicated: Automatic bisections on regressions and testing of external projects (Linux kernel, QEMU, OVMF…)."

“We apologize for the inconvenience caused by this unexpected announcement and appreciate your understanding and patience as we navigate this challenging situation,” Choi wrote before concluding: “Your trust and continued support are invaluable to us, and we remain committed to developing a powerful and reliable open-source hypervisor.”

Suffice to say this situation is sad news for the Xen Project, as Choi’s post points to a lack of human and financial resources at a time the project’s influence has waned as hyperscalers chose other hypervisors and Citrix - once Xen’s champion - largely dropped the ball on its own products using the tech. While the likes of XCP-NG have given the platform new impetus, in these cloudy times standalone and independent hypervisors are less visible and/or less important.

The Xen Project has tried to keep itself relevant by prioritizing embedded platforms in industries such as automotive, and a port to the RISC-V architecture.

We’ve asked Synoptek, an IT service provider that offers colo services, for comment - if only because in the current climate it’s unusual for a datacenter to close. We’ll update this story if we receive substantial information. ®

Updated at 0230 UTC, July 12

Xen Project insiders have told The Register the situation is “far from critical” and that the demise of the colocation facility may even help by giving impetus to years-long efforts to shift CI and test routines efforts to Gitlab.

“So strictly speaking, Xen Project isn't in peril at all, the old test platform might be, but frankly it's not really a problem,” we're now told. “The colo is just for a part of automated tests, it doesn't host anything else than a few old machines.”

The org’s post we reported painted the colo’s imminent demise as a likely source of disruption that could make bug-finding harder, and that donations may be needed to fund a replacement. That said, we're happy to revise our coverage - that Xen is in a pickle not a peril - in light of new information.

https://www.theregister.com//2024/07/11/xen_project_colo_testing/