Google DeepMind and UC Berkeley's research into a robot cooking up new materials predicted by AI algorithms is being called into question by a group of chemists.

The initial study, published in November in Nature, garnered attention for deploying a robotic lab system, A-Lab, to automatically synthesize novel compounds predicted by Google DeepMind's model GNoMe.

The software generated millions of recipes for new inorganic crystalline compounds that could have the potential to be useful in future electronics. Over 17 days, a robotic arm reportedly made more than 40 new materials, 35 of which had been predicted by GNoMe. It was able to mix and heat various powders to create materials whose structures were probed using X-ray diffraction.

A machine learning algorithm inspected the experimental patterns and compared them to predicted models to confirm whether the compound was made successfully. The experiment was billed as an important demonstration that showed how AI-powered robots could help drive scientific discovery.

However the results are now being disputed. In a separate paper, seven researchers from Princeton University and University College London believe that A-Lab didn't manage to make a single novel inorganic material.

"Unfortunately, we found that the central claim of the A-Lab paper, namely that a large number of previously unknown materials were synthesized, does not hold," they wrote in their analysis released on ChemRxiv [PDF]. When they combed through the X-ray diffraction data for each material, they found that most of them have been misclassified.

X-ray diffraction patterns allow scientists to calculate the position of the atoms inside. Different materials will make varying diffraction patterns. Scientists closely inspect the peaks and troughs in the data and compare them to existing patterns to interpret each material's structure.

The data from the A-Lab paper, however, shows that most of the 35 patterns for novel materials predicted by GNoMe resemble a mix of already known compounds, while three of them aren't new at all. The errors stem from trying to use AI to computationally determine whether a new material had been made or not, Robert Palgrave, Professor of Inorganic and Materials Chemistry at UCL, told The Register.

Researchers from Google DeepMind and UC Berkeley reportedly determined that if each sample made by the robot had a purity level over 50 percent, and if its structure differed from a list of known compounds contained in the ​​Inorganic Crystal Structure Database (ICSD), it should be declared as novel. But that process is unreliable, Palgrave and his colleagues claim.

"On the computational side, they couldn't deal with something called 'compositional disorder,' which is a very important feature of inorganic materials. A crystal is an ordered arrangement of atoms. But even within that order there can be disorder. Imagine you have a set of children's building blocks, all the same size and shape, and they are arranged in a perfectly ordered pattern on the floor. The blocks are like atoms in a crystal," he told us.

"But now imagine that there are two colors of block, red and blue. We have an ordered pattern of colors, say alternating red, blue, red, blue etc. You might end up with a chess board type arrangement. But it is also possible for the colors to be mixed up randomly. In this case the blocks themselves are ordered, but the colors are disordered."

The chemists believe the initial experiment had not taken compositional disorder into account, and assumed that the atoms in each compound made by A-Lab are ordered when they are actually disordered and already exist in the ICSD. "On the experimental side, they tried to use AI to interpret their experimental data, but it really didn't do a good job. I think AI can certainly do this kind of analysis. I have no idea why they failed, but the outputs are worse than even a novice human would achieve," Palgrave added.

Many of the outputs were poor fits to the diffraction patterns predicted by models, and they cannot be reliably used as proof of a compound's structure or purity, the group said. The results don't necessarily cast doubt on the GNoME algorithm per se. In fact, Palgrave and his colleagues believe that if some of the inorganic crystal structures predicted by GNoME managed to be successfully synthesized, it would result in a novel material.

Yet the compounds made by A-Lab aren't new, meaning none of GNoME's new materials appear to have been produced yet, they believe. "My own view is that [the paper] should be retracted as the main claim of discovery of new materials is wrong," he told us.

A representative from Google DeepMind declined to comment on the record.

Gerbrand Ceder, a lead author of the original A-Lab paper and Professor of Materials Science and Engineering at UC Berkeley, told The Register in a statement: "The work of Dr Palgrave is not peer reviewed and we believe it has multiple errors in it. We will comment on it in due course, but will do that through the peer reviewed literature." ®

https://www.theregister.com//2024/01/31/ai_chemistry_research_disputed/