How to learn new languages as a senior dev

Today, we are about to learn a new programming language. We will spend an hour reviewing keywords, loops, and variance... And if we have time left, we...

Wait! This is wrong. Why would a software engineer with 5y+ of experience suffer again learning all the basics?? Scratch all that.

Instead of starting with the basics, we will be starting with clean architecture, DDD, and then we dive deeper into how to implement domain, ports, and how to deploy all this.

Let's apply this to Python.

Disclaimer: I am not a Python expert. This blog post is something I created to give me a methodology to learn it. Expect imperfections. If you find one that deserves a fix, there is a link to my Github at the end, I´d be grateful for any tips!

Pre-requisites Jump to heading #

• A good understanding of clean architecture. One really good read I suggest is Herberto Graça variant of clean architecture

• A good understanding of event storming. You will see plenty of tutorial around, like this one. make sure what you use is visual.

• No python knowledge is required. It's ok if you do not understand it all. You can look at the documentation on the go

Let's practice this in this example

With so much ML models around, users want to discover, tests, and share them. Let's make an app for this

Event-storming ideas Jump to heading #

Let's start with this:

As a user, I want to know what ML models are nice for my application

flowchart LR
m_added[model is added in the system]
m_searched[model is searched by an user]

start-->|add|m_added
m_added-->|search|m_searched

User interface Jump to heading #

For your application to handle use cases, it needs to react to user (or consumer system) input.

Always remember, your language is compiled for/interpreted in an ecosystem. Some user interfaces will be therefore better suited than others

Be careful with user interface libraries. Python support of async was added "recently". Most libraries are not letting you await I/Os operations, meaning you could block your CPU.

CLI Jump to heading #

The easiest way to have user inputs is via a command line interface. Easy to develop, can be integrated into shell scripts

Arguments Jump to heading #

Using argparse lib, you can read from command line. There are zsh/bash plugins to make it seamlessly integrated with your fav shell You can use subcommands per use-case

import argparse
import sys


# We will define that later
def add_model(name: str, tags: list[str]):
    print("model", name, "added with tags ", tags)


def search_model(tags: list[str]):
    print("searching for tags", tags)


parser = argparse.ArgumentParser()

subparsers = parser.add_subparsers(dest="command", help="subcommand help")

add_model_command = subparsers.add_parser("addmodel", help="Add a model")
search_model_command = subparsers.add_parser("search", help="Search for a model")


add_model_command.add_argument("model", type=str, help="name of the model")
add_model_command.add_argument("tags", type=str, help="tags", nargs="+")
search_model_command.add_argument("tags", type=str, help="tags", nargs="+")


opts = parser.parse_args(sys.argv[1:])

if opts.command == "addmodel":
    add_model(opts.model, opts.tags)
elif opts.command == "search":
    search_model(opts.tags)

I/O Jump to heading #

Check input and print from functions

API Jump to heading #

Often, you create a module that is imported by another module. In this case, you have a programmatic interface.

This is especially the case if your script runs inside a Jupyter notebook or google collab project In such case, make sure to expose entry functions in your python project

First step is to create your function dedicated for API. They may import in return, other stuff from your domain or infra (ex: aggregate roots, ports..)

Create a api/add_model.py file

def add_model(name: str, tags: list[str]):
    print("model", name, "added with tags ", tags)

Once you have this, create an api/__init__.py file to create a module. This construction will make your consumer unaware of which files reside your implementations

# __all__ allows someone to to import wildcard directives
__all__ = ['add_model', 'search_model']

from .add_model import add_model
from .search_model import search_model

the dot here is used to say "we are importing from the module of this directory"

After this, you will be able to import your method

from api import add_model

Follow packaging projects if you want to then publish your package

Web Jump to heading #

To make your app web-ready, you need to wire it up to controllers. As in the js world, you have plenty of libraries:

For this blog post, we are going to use fastapi

Following the documentation, create a main.py file and add the endpoints

from fastapi import FastAPI
from pydantic import BaseModel

# Import your domain
from domain import MlModels


# Assemble your aggregate roots
ml_models = MlModels()


# Define some HTML DTOs using pydantic.BaseModel
class MlModelRequestBody(BaseModel):
    name: str
    tags: list[str]

# Create a fast api app
app = FastAPI()

# Execute methods on your agregate roots
@app.post("/models", status_code=201)
def addModel(model: MlModelRequestBody):
    ml_models.add_model(model.name, model.tags)
    return

Graphical user interface Jump to heading #

Be careful here, this is a odd territory. In theory, it is possible to build "native" apps. You will see that there are some solutions such as Kivy or Beeware. As of 2024-Q4, I am unsure learning them would be strategic:

• You will not find expertise if you want to recruit for your project
• There are some specialized UIs already per-use cases such as gradio for machine learning.

Infrastructure Jump to heading #

For your application to work, it needs an infrastructure. We're going to simply quote a few libraries here. Like with user interface libraries, keep in mind that some of them do not handle async/await, therefore, they will waste your CPU resources.

Let's implement a NoSQL because it's in vogue (no, let's not discuss the CAP theorem here!)

As usual, start defining your infrastructure DTOs and contracts. In your project, you may want to use an ORM to handle all this. Let's do it by hand for now.

from dataclasses import dataclass, abstractmethod
import json

# This creates a DTO
@dataclass
class MlModelItem:
    name: str
    tags: list[str]

    def to_json(self):
        return json.dumps(self.__dict__, default=lambda x: x.__dict__, indent=4)

    @staticmethod
    def from_json(jsonStr: str):
        jsonObject = json.loads(jsonStr)
        return MlModelItem(name=jsonObject["name"], tags=jsonObject["tags"])

# Abstract class
# yes, ABC means abstract class.
from abc import ABC
from typing import Awaitable

class MlModelRepository(ABC):
    @abstractmethod
    def add(self, item: MlModelItem) -> Awaitable[None]:
        pass

    @abstractmethod
    def list(self) -> Awaitable[list[MlModelItem]]:
        pass

Port your infrastructure for MongoDB

import motor.motor_asyncio


class MongoDbMlModelRepository(MlModelRepository):
    def __init__(
        self,
    ) -> None:
        super().__init__()
        self.client = motor.motor_asyncio.AsyncIOMotorClient("localhost", 27017)
        self.db = self.client.get_database("modelsapp")
        self.models = self.db.get_collection("models")

    def close(self):
        self.client.close()

    async def add(self, item: MlModelItem) -> Awaitable[None]:
        await self.models.insert_one(json.loads(item.to_json()))

    async def list(self) -> Awaitable[list[MlModelItem]]:
        result = []
        cursor = self.models.find()
        for document in await cursor.to_list(length=100):
            result.append(MlModelItem(name=document["name"], tags=document["tags"]))
        return result

# Important: register the concrete class as
# implementing the parent class
MlModelRepository.register(MongoDbMlModelRepository)

and use your repository in your app in your app

    ml_model_repository =  MongoDbMlModelRepository()
    ml_aggregate_root = MlModels(ml_model_repository)

Domain layer Jump to heading #

Yes. This is supposed to be a DDD-centric tutorial, and we're finishing with the domain.

At this point in the document, I expect you, the reader, to have at least vague notions of how to do inversion of control using abstractions, you should also be able to write your own DTOS.

You should already have also an idea about how to write your aggregates (root or not).

So, what's missing to do a good domain layer?

You tell me!

DevOps and Dev experience Jump to heading #

Once you have an idea of how to create a solution that fits your needs, you will need to put it in a development and deployment cycle.

OOP or FP? Jump to heading #

Some languages are more suited to model objects than model functions. However, keep in mind that python still allow you to do some sort of functional programming

Functional Programming is well adapted when you handle data flows, or express an intent to describe a function rather than a concept.

How to do unit tests Jump to heading #

Pytest is a good tool to do unit tests.

1. Install it pip install pytest
2. create a file named test_something.py or something_test.py. Coming from the web file.spec.ts, I like the 2d option more.
3. Create a method prefixed by test. ex: test_should_close_db_after_use

Package management Jump to heading #

Special mention to Rye which does both platform version management and package management. I haven't tested it. It looks promising

To ship packages, the official doc is here

Interesting reads I've found while finding documentation Jump to heading #

• LATO to do Dep injection and loosely coupling
• Python DDD a good repo to start from
• python-ddd-example
• And of course, the official docs