The city, neighborhoods and streets: Organizing your MLproject

Have you received a project that someone else created and did it make you go 🤯? (Was that someone else: you from a few months back?1 ) Sometimes a project organically grows into a mess of scripts and you don’t know how to make it better.

The main problem is often the project organization. I want you to think about, and organize, your project in three levels that I call city-level, neighborhood-level and street-level. This structuring helps you, your collaborators and yourself in 2 months.

city level: overall organization of the project

On this level almost every machine learning project has the same components:

• retrieve data (features) from a source system (you are not getting it from csv are you?)
• Split data into training and test sets (we split before we transform so we do not leak training information into the testset)
• Create transformations (on training data, apply those to test data too)
• Train model (on the trainingset)
• Evaluate model performance (on the testset)
• make decisions on further deployments

There are some strategic choices to make here, but it is mostly the same for every project.

Here is a rough sketch of the main function in python. This main function describes the high level (city level) design, you know where to look for details.

from dataloader import load_data
from split import split_data
from features import create_features
from model import model
from decision import make_decision

def ml_project():
"""Run the entire project"""
# get the data
# split it
training, test = split_data(data)
# create transformations on data
X_train, y_train = create_features(training)
X_test, y_test = create_features(test)
# train model
trained_model=model.train(X_train, y_train)
# evaluate model
evaluation =model.eval(X_test, y_test)
# make decision
make_decision(evaluation)

if __name__ == "main":
ml_project()


In R a package is often more convenient, use one file to define the overall logic. For example R/main.R:

#' Run the entire project
#'
#' This is the main function that gets the data,
#' trains the model, evaluates performance and
#' finaly makes a decision.
#' @export
#' @examples
#' main()
main <- function(){
data_split <- split_data(data)
training <- training(data_split)
testing <- testing(data_split)
workflow <- get_workflow(training)
model_fit <- fit(workflow, data=training)
result <- evaluate_model(model_fit, data=testing)
decision(result)
}



neighborhood level: organization within a component

For example: this is how you would organize the component retrieving data.

• connect to datasource
• get data

Each of these points can be small or big and consists of smaller decisions.

In python the retrieving data part looks like this. For example dataloader.py:

# imports here

"""main function to retrieve data"""
# we put all the neighborhood level steps here
db_con = get_db_connection()
data = retrieve_data(db_con)
return data

# here you define the functions you use in load_data()



In R we create a new file for example R/load_data.R:

#' Main function to retrieve data
db_con <- get_db_connection()
data <- retrieve_data(db_con)
data
}


street level: low level decisions

In the street level we create the low level details. How do we retrieve the database details, how do create a daily average, etc.

Here for example Connect to datasource:

• get credentials from environmental variable
• make a connection object
• pass the connection object to the data loader

I would put these low level decisions in the underlying functions. Like so:

# see above
# here you define the functions you use in load_data()

"""if necessary"""
pass

def get_db_connection():
"""f.i. sqlalchemy connection"""
db_dict = {
"database":os.environ["database"],
"url":os.environ["url"],
"port":os.environ["port"]
}
return engine

def retrieve_data(db_con):
# load maybe a sql query, or instructions
# use the db_con to get the data
return data


In R you do a similar thing.

get_db_connection <- function(){
con <- dbConnect(
bigrquery::bigquery(),
project = "publicdata",
dataset = "samples",
billing = billing
)
con
}

retrieve_data <- function(db_con){
dbGetQuery(con, sql)
}



Organization helps you think

By organizing your code in different levels you can focus on the right things at the right time. We can spend our mental energy wisely and not be distracted by other things.

• Do we talk about global order of operations (city level)? We don’t want to think about or look at the low level (street-level) details like database credentials, or the specific query that is used to retrieve the data.

• Is there a problem in the way data is retrieved (a neighborhood level issue)? You only need to look into the data retrieval module.

• You want to swap out the model used, and plug in a different model? You only need to modify the training module.

Images

The city, neighborhoods and streets: Organizing your MLproject
by  |