close
close

Smaller Language Models for Mobile Devices

Smaller Language Models for Mobile Devices

While big language AI models continue to grab headlines, small language models are where the action is. At least, that seems to be what Meta is betting on, according to a recently published paper by a team of research scientists.

Large language models like ChatGPT, Gemini, and Llama can use billions or even trillions of parameters to achieve their results. The size of these models makes them too large to run on mobile devices. Therefore, Meta scientists noted in their research that there is a growing need for efficient large language models on mobile devices; this need is driven by increasing cloud costs and latency concerns.

In their research, the scientists explained how they created high-quality large language models with less than a billion parameters, arguing that this is a good size for mobile deployment.

Contrary to popular belief that the amount of data and parameters plays a major role in determining model quality, scientists have achieved similar results to Meta’s Llama LLM program in some areas with the small language model.

“There’s a common paradigm that bigger is better, but this shows that it’s really about how the parameters are used,” said CEO Nick DeGiacomo. BucephalusAn AI-powered e-commerce supply chain platform based in New York City.

“This paves the way for more widespread adoption of on-device AI,” he told TechNewsWorld.

A Critical Step

Meta’s research is significant because it challenges the current norm of cloud-based AI, where data is typically processed in remote data centers, says Darian Shimy, Meta’s CEO and founder. FutureFundA venture capital firm based in San Francisco.

“By bringing AI processing to the device itself, Meta flips the script; it has the potential to reduce the carbon footprint associated with data transmission and processing in large, energy-hungry data centers, and makes device-based AI a major player in the technology ecosystem,” he told TechNewsWorld.

“This research is the first comprehensive and publicly shared effort of this magnitude,” added Yashin Manraj, CEO Pvotal TechnologiesAn end-to-end security software developer based in Eagle Point, Oregon.

“It’s a critical first step toward achieving an SLM-LLM-compliant approach where developers can find the right balance between cloud and on-device data processing,” he told TechNewsWorld. “It lays the groundwork for the promise of AI-enabled applications to reach the level of support, automation, and assistance that has been marketed in recent years but lacks the engineering capacity to support these visions.”

Metascientists have also taken a significant step in reducing the size of a language model. “They propose to shrink a model by an order of magnitude and make it more accessible for wearables, hearables, and mobile phones,” said Nishant Neekhra, Skyworks SolutionsA semiconductor company in Westlake Village, California.

“They’re enabling a whole new set of applications for AI, while also enabling new ways for AI to interact in the real world,” he told TechNewsWorld. “By shrinking, they’re also solving a big growth challenge affecting LLMs, which is their ability to be deployed on edge devices.”

High Impact on Health Care

One area where uvula models can make a significant impact is medicine.

“The research promises to unlock the potential of generative AI for applications involving mobile devices, which are widely used for remote monitoring and biometric assessments in today’s healthcare landscape.” Danielle KelvasDr. is a physician consultant working at IT Medical, a global medical software development company.

By showing that effective SLMs can have fewer than a billion parameters and still perform comparable to larger models on certain tasks, researchers are opening the door to widespread adoption of AI in everyday health monitoring and personalized patient care.

Kelvas explained that using SLMs can increase patient privacy by ensuring sensitive health data can be securely processed on one device. They can also facilitate real-time health monitoring and intervention, which is critical for patients with chronic conditions or those requiring ongoing care.

He added that the models could also reduce technological and financial barriers to the use of AI in healthcare and make advanced health monitoring technologies more accessible to the public.

Reflecting Industry Trends

Meta’s focus on small AI models for mobile devices reflects a broader industry trend to optimize AI for efficiency and accessibility, it explained Caridad Munoz“This shift not only addresses practical challenges, but also aligns with growing concerns about the environmental impact of large-scale AI operations,” ., a professor of new media technology at the City University of New York, told TechNewsWorld.

“By supporting smaller and more efficient models, Meta is setting a precedent for sustainable and inclusive AI development,” added Muñoz.

Small language models also fit into the edge computing trend, which focuses on bringing AI capabilities closer to users. “Large language models from OpenAI, Anthropic, and others often go overboard — ‘if you only have a hammer, everything looks like a nail,’” DeGiacomo said.

“Custom, tuned models can be more efficient and cost-effective for certain tasks,” he noted. “Many mobile applications don’t require cutting-edge AI. You don’t need a supercomputer to send a text message.”

“This approach allows the device to address the relationship between questions that can be answered using SLM and specialized use cases, similar to the relationship between GPs and specialist physicians,” he added.

Profound Impact on Global Connectivity

Shimy argued that SLMs could have profound effects on global connectivity.

“As on-device AI becomes more capable, the need for constant internet connectivity decreases, which could significantly change the technology landscape in regions where internet access is inconsistent or costly,” he observed. “This could democratize access to advanced technologies and make cutting-edge AI tools available across a range of global markets.”

While Meta is leading the development of SLMs, Manraj noted that developing countries are aggressively pursuing the situation to keep AI development costs in check. “China, Russia, and Iran seem to have shown great interest in the ability to defer computations on local devices, especially when cutting-edge AI hardware chips are embargoed or not easily accessible,” he said.

“We don’t expect this to be an overnight or radical change,” he predicted, “because complex, multilingual queries will require cloud-based LLMs to deliver the latest technology to end users. However, this shift to allowing a ‘last mile’ model on the device can help reduce the burden on LLMs to handle smaller tasks, reduce feedback loops, and provide local data enrichment.”

“Ultimately,” he continued, “the end user is clearly going to be the winner, because it’s going to enable next-generation capabilities on their devices and a more promising overhaul of front-end applications and the way people interact with the world.”

“While the usual suspects are driving innovation in this sector with promising potential impact on everyone’s daily lives,” he added, “SLMs can also be a Trojan Horse, providing a new level of sophistication in intruding into our daily lives by having models that can collect data and metadata at an unprecedented level. We hope that with appropriate security measures, we can direct these efforts to a productive conclusion.”