I recently tried to fine-tune a small language model (LLM) on my laptop, but it was a frustrating experience. Despite being touted as a cost-effective solution, I found that training an LLM can be an expensive affair – not just in terms of money, but also time.
The reality is that LLMs still haven’t matured enough to select the right tools with simple one-shot or few-shot prompting. I’ve had an easier time teaching my pet dog to pick the right tool than teaching my LLM to do the same!
While it’s relatively easy to leverage a large LLM with 70 billion parameters for better tool-calling capability, it’s a ridiculous waste of money that would quickly become untenable for businesses. Financial operations (FinOps) is a serious business in the real world, and I believe there’s a big scope of optimization in this area by leveraging the right-sized LLM and infrastructure to host it.
By doing so, we can get the best bang for our bucks invested in Agentic AI. It’s time to rethink our approach to AI development and focus on finding more efficient and cost-effective solutions.