I was thrilled to launch a new application that leverages a financial search API, allowing agents to query financial data in natural language. But, my excitement was short-lived. I discovered that the new OpenAI open source model (120b) has a major limitation when it comes to agentic workflows – it’s terrible at tool calls.
I’ve been testing it with the Vercel AI SDK through the AI gateway, and it seems incapable of simple tool calls that other models like 4o-mini can handle with ease. I’m not the only one experiencing this issue; others have echoed similar frustrations. This limitation could be a major roadblock for developers relying on tool calls in their applications.
Is this a problem with the AI SDK, or is it a more fundamental issue with the OpenAI OSS 120b model? I’d love to hear from others who have encountered this problem and find out if there are any workarounds.
The potential of natural language querying is immense, but limitations like this hold us back from realizing its full potential.