Running a fraud-detection LLM entirely on the phone

My MSc research project was accepted for presentation at RANLP 2025: Efficient Financial Fraud Detection on Mobile Devices Using Lightweight Large Language Models. It tackles a simple question. Can a phone run an LLM good enough to catch scam messages, without sending anything to the cloud?

The problem

Digital transactions grow every day, and so do fraudulent SMS and emails. Traditional detection models are often cloud-dependent, which brings problems with efficiency, scale and especially privacy. Large language models understand language well, but they are usually too heavy for a phone.

The approach

• Fine-tune compact LLMs, under 1B parameters, for one task: spotting fraud in unstructured text.
• Use LoRA and QLoRA so fine-tuning fits modest hardware.
• Convert to ONNX with Hugging Face Optimum, then quantise to INT8 and 4-bit.
• Run the models with ONNX Runtime inside a React Native app, fully offline.

Making it run on a phone

Getting a trained model onto a phone took more work than training it.

• Download only what’s needed. The app reads a model list from GitHub and lets you pick a model and a quantisation level. It then fetches just the config, the tokenizer and that one .onnx file from Hugging Face, with resumable downloads and per-file progress.
• Tokenising on Hermes. The JavaScript tokenizer expected a browser, so I patched it to load only local files, rewrote Unicode regexes that Hermes doesn’t support, and added TextEncoder and Buffer polyfills.
• Answer in a few tokens. The model only needs to say “Yes” or “No”, so generation stops after five tokens, using greedy decoding with a KV cache in ONNX Runtime. The app shows how long each check took.

The results

The fine-tuned, quantised Llama-160M-Chat-v1 reached 99.47% accuracy with a 168 MB footprint. The fine-tuned, quantised Qwen1.5-0.5B-Chat reached 99.50% at 797 MB. Optimised LLMs can deliver accurate, real-time fraud detection on a phone without compromising privacy.

Let’s make AI safer and smarter, right in our pockets.

The app, FraudShield, is live on the App Store and Google Play, and the paper is in the ACL Anthology. Thanks to my supervisor, Deshan Sumanathilaka, for his guidance from the first idea to deployment.