The Tiny Model Agentic Revolution

The reason GenAI is confusing isn't the technology. It's the dubious marketing claims made by large vendors who want to lock you into their ecosystem while promising a path to success for you.

They can't build you a solution, so they try to sell you a platform. Copilots, AgentForce... these don't solve AI challenges. Rather, they make the promise that you can have AI without doing the work of building the AI application layer for your company.

Yet, we don't see these solutions being deployed. And the the fear of foundation models consuming application layer efforts is rapidly proving unfounded. In fact, the vendors building the application layer are the ones winning.

OpenAI: General Chat

Perplexity: search and research

Cursor: software development

Buying a large vendors low-code/no-code system, with a tacked on vector database and the promise that your data is your competitive advantage, is a route to failure.

AI is fantastic, but not sufficient to go into business operations without building a robust application layer. And it looks like this application layer is going to be built agentically, on many tiny models.

The hard truth: there only moat IS the application layer

Any what will the application layer look like?

Smart Experiences (a reasoning based application layer) built on top of agentic reasoning built with tiny models and reinforcement learning.

DeepSeek R1 and the surrounding innovation in reinforcement learning made this obvious.

Released on January 20, 2025, DeepSeek introduced reinforcement learning and distillation to the industry consciousness and released models small enough to fit on your phone, run a browser, or run on an old computer without a GPU.

Distillation: training a smaller model by having it interact with a larger "teacher" model.

This would be the equivalent of a PhD professor training undergraduate and master students.

Reinforcement learning: a method of training a model by giving it feedback on its answers through a reward signal.

This would be the equivalent of joining a new organization and learning the ropes of a new specific job.

The promise of this is specialized LLMs that are much cheaper to run, while also being more accurate and effective.

Just like we hire diverse skills into an company, AI solutions will be best served by a collection of specialized AIs to the task.

Post DeepSeek we've seen some remarkable results from small teams:

https://xyzlabs.substack.com/p/berkeley-team-recreates-deepseeks

https://novasky-ai.github.io/posts/sky-t1/

Large language models are "accidentally" good at what they do, but 96% of their neural network is fluff.

THIS is why large language models can understand the world

The above is an 18 minute video that seems academic, but is worth the watch. It will help you home your intuition of LLM and agentic strategy.

Conclusion

You need to build open systems to leverage the best emerging AI. And if compute is the barrier to competitive AI, then you need to build open systems that can use low cost compute and smaller models. Most importantly, you need to build and ship the AI application layer into your organization today. You ability to deploy industry leading AI application layers will be your competitive advantage.