{"id":21104,"date":"2018-10-22T10:09:06","date_gmt":"2018-10-22T08:09:06","guid":{"rendered":"https:\/\/www.inovex.de\/blog\/?p=13233"},"modified":"2022-11-24T10:49:25","modified_gmt":"2022-11-24T09:49:25","slug":"how-to-manage-machine-learning-models","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/how-to-manage-machine-learning-models\/","title":{"rendered":"How to Manage Machine Learning Models"},"content":{"rendered":"

Developing a good machine learning model is not straight forward, but rather an iterative process which involves many steps. Mostly Data Scientists start by building a so called baseline, which can be used as a reference point to compare other models. This baseline can be created\u00a0by just calculating the average or using some simple models. After that a data scientist will probably try different models to see how they perform before doing some kind of hyper-parameter tuning to improve the most promising ones. Even if those models achieve good results, there will still be plenty of options to improve such as using more data (pre-)processing steps, creating additional features, using some form of dimensionality reduction or even applying stacking strategies.<\/p>\n

\n

Have a look at our 2022 update comparing frameworks for machine learning experiment tracking<\/a> and see how MLflow, ClearML, neptune.ai and DAGsHub hold up!<\/p>\n<\/div>\n

Clearly this is an explorative process, which requires expertise and flexibility in tooling. Therefore data scientists mostly use notebooks to quickly try new ideas, rapidly train models and compare them, most of the times using simple print statements. This works well first, but will become confusing as the number of models and parameters increases. A common approach is\u00a0write the prediction results to a table or dataframe, but even so it is difficult to track all important information such as used hyper-parameters, datasources, time of execution etc.<\/p>\n

While big companies such as Google, Facebook and Uber develop custom machine learning platforms to support data scientists in this challenge, also smaller projects arose within the last months, for example\u00a0DVC<\/a>,\u00a0\u00a0Sacred<\/a>\u00a0or Databricks‘ mlFlow<\/a>. While we currently evaluate these, we also tested another alternative named\u00a0ModelDB<\/a>\u00a0 earlier this year. The following blog article was created during this evaluation in May this year. While ModelDB might not be the best choice at this point of time, its evaluation explains the concept of machine learning model management and will be used as baseline in another article to follow in this series.\u00a0<\/p>\n

\n

<\/p>\n<\/div>