New AI Training Method is 13 Times Faster, 10 Times More Efficient

New AI Training Method is 13 Times Faster, 10 Times More Efficient

Researchers at Google’s Deepmind have discovered a faster, more efficient method for training artificial intelligence (AI) models, and they claim the new technique offers 13 times faster performance and 10 times more power efficiency compared to existing technologies.

This discovery was pioneered by Google’s AI research lab, which is working to create JEST (joint example selection), an advanced AI training method that can speed up the process while reducing computational resources and time required.

This new approach to training models is timely, as concerns about the environmental impact and power consumption of AI data companies continue to loom large.

Artificial Intelligence and Mother Nature

The AI ​​industry uses a lot of processing power, which requires a lot of energy. By 2023, AI operations will reach an alarming record of around 4.3 GW, almost equal to the power consumption of Cyprus in 2011.

Now, a request on ChatGPT costs 10 times more power than a simple Google search. Experts also predict that AI will cover 25% of the US power grid by 2030, up from just 4% today.

Extensive energy usage requires an equally extensive volume of water to dissipate the heat generated in these systems. Therefore, Microsoft has contributed to the reduction in water supply after a 34% increase in water consumption from 2021 to 2022 following increased AI workloads. Many people also accused ChatGPT of using half a liter of water for every 5 to 50 requests.

But All Hope Is Not Lost

However, Deepmind’s JEST method allows Google to significantly reduce the number of iterations and computational power required to train AI models, which could lower overall energy consumption.

The new method differs from existing training techniques in that it uses complementary data sets rather than individual data points to boost an AI model’s machine learning, according to a technical paper published by the researchers.

“We showed that selecting datasets together is a more effective method of learning than selecting examples independently,” the paper stated.

JEST works by first building a smaller AI model to rate the quality of information from curated datasets, ranking the groups in terms of quality, and applying the findings to larger, lower-quality datasets. In this way, the smaller JEST model initially determines the most appropriate groups for training, and the larger model is then trained based on the results of the smaller model.

For AI training to be successful, datasets must be of the highest quality to optimize training efficiency. This feature makes the method difficult to replicate, as professional research expertise is needed to create the initial training data on which the entire technique is based.

According to Google Deepmind’s research, JEST “outperforms state-of-the-art models with up to 13x fewer iterations and 10x fewer computations.”

“A reference model trained on a small set of organized data can effectively guide the organization of a much larger data set, allowing training of a model that significantly exceeds the quality of the reference model on many downstream tasks,” the paper stated.

The report backed up the claims using experiments that showed notable improvements in power efficiency and learning speed when JEST was applied to train an AI model using a basic web language image (WebLI) dataset.