production

LLMs are pretty cool, but they are massive! If you want to run those for yourself you need a hefty GPU and quite a lot of engineering. But the world of machine learning is so much bigger then LLMs. In organizations all over the world, there are models forecasting time-series, predicting prices, creating embeddings, classifying categories and what not. If you have several prediction/classification steps that combine into one end- result, you could consider training one bigger model that does all of the things. [Read More]

I always push on people to make their ML project reach production. Even if it is not that good yet and even if you could eke out a bit more performance. I’ve been inspired by the dev-ops and lean movements and I hope you will be too. ML products have many ways to improve, you can always tweak more. But ML is high risk, with a possible high reward and relatively expensive compared to ‘normal code’. [Read More]

As a data scientist that wants to achieve production results, one of the best options is to make your work available in kubernetes. Because kubernetes runs on all clouds and because many organizations use kubernetes. Make your prediction API available in kubernetes and your organization can ‘just’ plug it into their systems. Many data scientists don’t know anything about docker, not to mention kubernetes and its main tool helm. I think you should learn and practice just enough helm to be dangerous1. [Read More]

Your production system just broke down. What should you do now? Can you imagine your shiny application / flask app, or your API service breaking down? As a beginning programmer, or operations (or devops) person it can be overwhelming to deal with logs, messages, metrics and other possible relevant information that is coming at you at such a point. And when something fails you want it to get back to working state as fast as possible. [Read More]