[ExI] LLMs can run on cellphones

[BillK]

I asked Perplexity AI.
Currently, RAM limitations mean only smaller LLMs can run on cellphones.
But cellphones will soon have more RAM.
BillK

Perplexity AI:

Yes—many *small LLMs* can run entirely on modern smartphones, offline and
without sending prompts to a cloud service. The practical sweet spot is
roughly *1B–4B parameter models*, often quantized to 4-bit precision;
flagship phones can sometimes handle 7–8B models.
Models that work well
Model Typical size Good for
Gemma 3 / Gemma 3n 1B–4B General chat, summarization, mobile-first use
Llama 3.2 1B / 3B Lightweight chat and mobile deployment
Microsoft Phi-4 Mini 3.8B Strong reasoning/efficiency for its size
Qwen 3 / Qwen 3.5 small ~0.8B–4B Multilingual tasks and compact chat
SmolLM 2 1.7B Very lightweight local assistant

These are not equivalent to the largest cloud models on complex reasoning,
coding, or long-context work, but they are useful for drafting,
summarizing, answering questions, extracting structured data, and basic
offline
How to run one

   -

   *Easiest apps:* PocketPal AI, MLC Chat, and Google AI Edge Gallery
   support downloading and running compact models on Android and/or iOS.
   -

   *Built-in options:* Recent Android flagships may include Gemini Nano,
   while newer iPhones can use Apple’s on-device Foundation Models; these are
   usually integrated into system/app features rather than general-purpose
   chat apps.
   -

   *More technical route:* On Android, Termux plus llama.cpp can run GGUF
   model files locally; MLC LLM can also use phone GPUs.

Practical limits

RAM is the main constraint: 1B–2B models generally suit phones with 6–8 GB
RAM, while 3B–4B models are more comfortable on flagships with 8–12 GB.
Expect slower generation, higher battery use, and a need to keep several
gigabytes free for the model file.

For a first test, install *PocketPal AI* and try *Gemma 3 1B*, *Llama 3.2
1B*, or *Qwen 3 1.7B*; if your phone has ample RAM, move up to *Phi-4 Mini*
or *Gemma 3 4B*.

--------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20261020/fdaeb1e5/attachment-0001.htm>