Microsoft Unveils Tiny AI Model: Phi-3 Mini

  Pexels

Microsoft has released the first iteration of its ultra-small Phi-3 models, the Phi-3 Mini. This ‘small language model’ (SLM) is compact enough to run smoothly on a smartphone, a significant improvement from its predecessor, Phi-2.

Phi-3 Mini: Compact but Capable

Despite its small size, the Phi-3 Mini boasts 3.8 billion parameters and has been trained on a very compact dataset. For comparison, Meta’s open-source Llama models, widely used by AI developers, consist of a minimum of 7 billion parameters.

Despite being smaller in size, the Phi-3 Mini has surprisingly good performance. It can’t match the performance of OpenAI’s leading GPT-4, but it delivers outputs comparable to larger language models (LLMs) that are up to 10 times its size.

The Training Plan

The Phi-3 Mini was trained using a plan inspired by how children learn from bedtime stories. This involved books with relatively simple words and sentence structures covering a variety of topics. Other LLMs were used to create ‘children’s books’ with only a list of three thousand words. This process was used in training the Phi-3 Mini, according to the Corporate Vice President of Azure AI Platform at Microsoft.

Applications and Future Plans

Microsoft’s new AI model is especially suitable for use with companies’ own applications. Moreover, the Phi-3 Mini can run on conventional processors, eliminating the need for an AI computer. Even smartphones could potentially run this model.

Microsoft plans to release further versions of the Phi-3 LLM, including the Phi-3 Small (7 billion parameters) and Phi-3 Medium (14 billion parameters). The release dates for these models are currently unknown.

The Phi-3 Mini is now available via Azure, as well as on the AI platforms of Hugging Face and Ollama.