deployment

LLMs are pretty cool, but they are massive! If you want to run those for yourself you need a hefty GPU and quite a lot of engineering. But the world of machine learning is so much bigger then LLMs. In organizations all over the world, there are models forecasting time-series, predicting prices, creating embeddings, classifying categories and what not. If you have several prediction/classification steps that combine into one end- result, you could consider training one bigger model that does all of the things. [Read More]

I’ve seen a pattern in different places but it is most useful for streaming data. Data that comes in over time, with quite some volume. The core of the solution is a message broker, this could be light weight like redis1, or a heavier log-like solution like kafka2. In stead of sending data from one microservice to another through API calls, we publish data to a central place, and services subscribe to data, and publish their results back (that is why it is called PubSub; publish - subscribe). [Read More]

how do you feel when you press the ‘deploy to production’ button? Confident, slightly afraid? I bet many data scientists find it a bit scary. It’s worth it to dig a bit deeper into this fear. In my ideal world we are not scared at all. We have a devops mindset. We have no anxiety, no fears at all. You should be confident that the deployment pipeline takes care of everything. [Read More]