Deploying machine learning models is more than just training — it’s about tracking, versioning, serving, and monitoring. In this post, I’ll walk you through how I built a production-ready ML pipeline using:

• MLflow for experiment tracking and model registry
• FastAPI for serving models via REST API
• MinIO for artifact storage (S3-compatible)
• Docker Compose for orchestration

👉 Full source code:
🔗 github.com/liviaerxin/mlops-fastapi-mlflow-minio

The most robust scraping architecture uses an LLM Agent for the "Discovery Phase" to map endpoints, but relies on deterministic scripts for the actual data extraction.

In modern distributed systems, the boundary between "doing the work" and "managing the work" is where most engineering teams trip up. To understand this, let's return to the restaurant analogy.

• Celery is a Line Cook. It is incredibly fast and efficient at doing exactly one atomic job ("Fry this burger", "Summarize this transcript").
• Temporal or Hatchet is the Restaurant Manager (Orchestrator). It holds the master blueprint of the customer's entire journey, tells the line cooks what to do, goes to sleep when waiting for the customer, and wakes up exactly when needed.

If you find yourself hacking Celery to act as a manager—using complex wait_for_agent logic or passing sticky notes between tasks—you are fighting the framework.

Celery is a phenomenal Task Queue (executing atomic background jobs), but it is a poor Workflow Orchestrator (managing state, pauses, external signals, and retries across disparate services).

Performance in agentic systems is co-defined by the model and the framework around it. Without structure, even a senior-level model behaves like an inconsistent junior engineer.

A practical overview of how Skills, AGENTS.md, multiple agents, and sub-agents fit together, and how to trigger each one.

A practical note for developers making the shift from writing code to harnessing AI agents effectively.

The Shift Nobody Warned You About​

In 2022, being a good engineer meant writing good code.

In 2026, being a good engineer means governing agents that write good code — and keeping them effective across sessions, phases, and time.

That's a fundamentally different skill. And most developers are still using 2022 habits with 2026 tools, wondering why their AI keeps making mistakes, forgetting context, and re-litigating decisions they already made three sessions ago.

The problem isn't the agent. The problem is the layer above it.

Captured from a public Claude artifact: https://claude.ai/public/artifacts/88e0a8b7-1dee-4401-b3a3-54475d959538

Personal note — the path from AI Engineer to AI Architect

The Three Areas of Mastery​

Area 1: Skill Design        → How to build reliable PromptOps skills
Area 2: Workflow Architecture → How to connect skills across a project
Area 3: Agent Behavior      → How to understand and prevent agent failure

Master all three. Most people stop at Area 1.

In Python code development, not all errors are created equal. While VS Code is great at catching typos, it often stays silent on the bugs that actually crash production code.

This note explores the hierarchy of Python errors and the specific tools—from traditional linters to modern AI agents—needed to catch them proactively.

The Hierarchy of Errors​

To fix bugs, we first need to classify them. Different tools specialize in different layers of this hierarchy:

1. Syntax Errors (The "Grammar" Police)​

• Definition: The code violates the rules of the language. It cannot run at all.
• Example: Missing a colon (:) after if, unmatched parentheses.
• Detection: Instant. Standard VS Code support catches this immediately.

2. Type Errors (The "Shape" Mismatch)​

• Definition: The code runs, but you are trying to combine incompatible data types.
• Example: result = "Total: " + 10 (Cannot add string and integer).
• Detection: Static Analysis. Tools like Pylance (set to 'Basic' or 'Strict') excel here.

3. Logical Errors (The "Silent" Killers)​

• Definition: The code runs without crashing, but it produces the wrong result or doesn't do what you intended.
• Example: Calling an async function without await (the function is skipped, but no error is raised).
• Detection: Difficult. Standard linters often miss these unless explicitly configured. Requires AI Logic Analysis.

4. Semantic Errors (The "Meaning" Failure)​

• Definition: The code is valid and runs, but it makes no sense in the real-world context.
• Example: Calculating discount = price + tax (logic is valid math, but semantically wrong for a discount).
• Detection: Very Difficult. Requires an AI that "understands" the intent of your feature (e.g., Windsurf, Copilot).

Level 1: The Foundation (Linters)​

These tools are deterministic. If rule X is broken, they flag it 100% of the time.

• Pylance (Microsoft): The default Python Language Server.
  • Config Tip: Set "python.analysis.typeCheckingMode": "basic" in settings to catch un-awaited coroutines and unused variables.
• Pylint / Flake8: Dedicated linters that enforce style and catch "code smells" (like functions that are too complex).

Level 2: The AI Validator (LLM Linters)​

When static rules aren't enough, we need probabilistic models to check our logic.

• Qodo Gen (formerly CodiumAI):
  • Best for: Proactive Logic Checking.
  • How it works: It analyzes your code and attempts to generate tests to break it. If it can write a test that fails, you have a logic bug.
• SonarLint:
  • Best for: Deep Code Quality.
  • How it works: Uses advanced static analysis (symbolic execution) to find complex bugs like infinite loops or null pointer dereferences before execution.

Level 3: The AI Agent (Context Aware)​

Sometimes you need a tool that understands the whole project.

• Windsurf (Cascade):
  • Best for: End-to-end error fixing.
  • How it works: Unlike a passive linter, Windsurf watches your Terminal. If you run a script and get a RuntimeWarning, Windsurf sees it and can proactively offer a fix. It connects the result of the code back to the source.

Recommended "Proactive" Setup​

To turn VS Code into a fortress against bugs, use this stack:

1. Immediate Feedback: Pylance (Mode: Basic) + SonarLint.
   • Catches syntax, types, and known code smells instantly.
2. Logic Verification: Qodo Gen.
   • Click "Review" on complex functions to catch edge cases.
3. The Fixer: Windsurf or Copilot.
   • Use chat to explain why the logic failed and rewrite the block.

Resources​

• Pylance Extension
• Qodo Gen (CodiumAI)
• SonarLint
• Windsurf IDE

With the LLM being more and more powerful, it is now possible to write markdown spec files that describe the development process. Then these spec files are written in markdown can be used to generate the running code.