As a developer building a demand forecasting tool for small to medium-sized enterprises (SMEs), I’ve encountered a common challenge: limited historical data. This constraint makes it difficult to accurately predict demand, leading to stockouts and lost sales. In this post, I’ll explore the best machine learning approach to predict demand for SMEs with limited historical data. To set the context, my demand forecasting tool aims to optimize inventory and avoid stockouts by providing a React dashboard with time-series plots, ‘days until stockout’ estimates, and reorder quantity suggestions. The constraints are: Multiple products across stores with irregular and sometimes seasonal demand Limited historical data per SKU (some only a few months) SMEs can’t afford heavy infrastructure, so the solution must run on a modest Python stack (scikit-learn, Prophet, etc.) Forecast horizon: 7 days ahead I’ve already tested naive and moving average baselines, but now I want to move towards more robust models. The key questions are: Should I use a global model across all SKUs or train one per SKU? What are the preferred models for this setup (Gradient Boosting, Prophet, SARIMAX, hybrid)? How can I effectively engineer features with sparse time series data? To tackle these questions, I’ll need to consider the characteristics of my data and the requirements of my project. For instance, using a global model across all SKUs might not capture the unique demand patterns of each product. On the other hand, training a model per SKU could lead to overfitting with limited data. In terms of models, Gradient Boosting and Prophet are popular choices for time series forecasting. However, SARIMAX might be more suitable for handling seasonal demand patterns. A hybrid approach could also be explored, combining the strengths of different models. Feature engineering is critical in machine learning, and sparse time series data presents a unique challenge. One approach is to use techniques like mean encoding, where the mean of a feature is calculated for each SKU and used as a new feature. Another approach is to use domain knowledge to create features that are relevant to the business, such as holidays or weather patterns. In conclusion, predicting demand for SMEs with limited historical data requires a thoughtful approach to machine learning. By considering the characteristics of the data, the requirements of the project, and the strengths of different models, we can develop a robust demand forecasting tool that helps SMEs optimize their inventory and avoid stockouts.
