As machine learning practitioners, we’re constantly exploring new opportunities to provide value to our clients. Two potential services have caught my attention: private LLM infrastructure deployment and custom model fine-tuning. But before diving in, I’d love to get a reality check from the community.
**Private LLM Infrastructure Deployment**
Helping enterprises deploy local LLM servers to avoid sending sensitive data to OpenAI/Anthropic APIs. This approach offers a one-time setup plus ongoing support. The question is, are enterprises concerned enough about data privacy to pay for on-prem solutions?
**Custom Model Fine-Tuning**
Training smaller, specialized models on company-specific data for better performance at lower cost than general-purpose models. But how hard is it realistically to fine-tune models that outperform GPT-4 on narrow tasks?
**The Big Questions**
– Are enterprises willing to pay for on-prem solutions to protect their data privacy?
– How challenging is it to fine-tune models that outperform general-purpose models?
– Which space is more crowded with existing players?
If you have any real-world experience with either approach, I’d love to hear about it. Let’s discuss the market dynamics and opportunities in the comments below.
—
*Further reading: [LLM Deployment Strategies](https://towardsdatascience.com/deploying-llms-in-production-5-strategies-83a9a21f1f4c)*