—:— IST
Chennai, India ·

Karthik Palani

I design and run the systems that put AI agents into production — and the infrastructure they stand on.

Senior DevOps engineer working toward AI solutions architect. I own the infrastructure layer — Kubernetes, pipelines and hardware test rigs — and I architect what runs on top of it: autonomous code review, AI-executed test suites, local-first meeting transcription, and the shared model-routing, MCP and retrieval platform underneath them. Eight years across cloud architecture, SRE, IoT and embedded systems is what makes the AI part deployable rather than merely demonstrable.

Currently
ESAB India
Experience
8.7 years
Working toward
AI Solutions Architect
Status
Open to roles
scroll — turn the page
01

Currently.

What is actually on my desk right now — not a wish list.

A · Day job

Leading a full-stack & DevOps team at ESAB India, owning end-to-end infrastructure for a large-scale industrial platform.

B · AI

Architecting the shared agent platform — model routing, MCP tools, retrieval — that two production agents run on.

C · Embedded

Hardware-in-the-Loop test rigs, wired into the same pipelines as the software.

D · Platforms

Kubernetes, Helm and VM estates — AKS in the cloud, k3s and Proxmox on the floor.

F · In the open

Three of these are public on GitHub. The platform underneath them mostly isn't mine to publish.

E · Philosophy

I care about the compounding return of small, boring automation — the runbook that prevents a 3am page, the pipeline that makes a Friday deploy feel safe, the Helm chart a new joiner can read in ten minutes.

02

AI Systems.

Systems I designed and shipped. The first two are in production — running against real merge requests, real test suites and real hardware, on a schedule, with metrics. The third is where I go a layer down and build the inference stack myself.

↓ the part I'd want to talk about

01

In production

CodeSentinel

Autonomous code review that learns from its own history

source

A review agent that sits on GitLab merge requests. It parses the unified diff, decides where it needs more context, and goes and gets it — pulling whole files, searching the codebase semantically, and recalling how similar changes were reviewed before. Findings land as inline comments with one-click GitLab suggestion blocks, deduplicated across review iterations so it does not repeat itself when a developer pushes a fix.

  • Episodic memory Stores past reviews and developer responses, then retrieves the relevant ones — feedback stays consistent across MRs instead of resetting every run.
  • Agentic tool use Claude decides when to fetch a full file, run a codebase search, or query memory — rather than being handed a fixed context window and guessing.
  • Thread lifecycle Answers follow-up questions in a discussion thread and auto-resolves it once the issue is genuinely addressed.
  • Two run modes Long-running service polling a GitLab group for MRs where the bot is assigned reviewer, or a single-shot CLI call for pipeline integration.
  • Built to stay up SQLite state so restarts never double-review, thread-pool concurrency, Prometheus metrics, FastAPI dashboard, multi-arch Docker images.
  • Swappable brain Provider factory behind the reviewer — the model is an env var, not a rewrite.
Python 3.10+ Claude Sonnet Tool calling GitLab API SQLite FastAPI Prometheus Docker
02

In production

TestFlow

Manual test cases, executed by an agent, with no scripts written

source

QA writes a test case in plain English in TestRail. TestFlow reads it, works out what the steps mean, drives a real Chromium browser through them with Playwright, validates the outcome, captures screenshots, and writes pass/fail back to TestRail. Nobody writes a selector. Nobody sits and watches it run.

TestRailplain-English cases Agentinterprets intent Playwrightdrives the browser Resultsback into TestRail
  • Vector memory Remembers how a step resolved last time and replays it — cutting model calls by roughly 70–80% on repeat runs, which is what makes nightly execution affordable.
  • Degrades gracefully With no model key configured it falls back to rule-based parsing (regex plus HTML inspection) instead of failing the run.
  • Beyond the browser Targets web HMI, PLC endpoints and GitLab — the same case format drives industrial and software surfaces.
  • Evidence trail Full execution history, screenshots and timing metrics per step, streamed live over WebSocket while it runs.
Python Playwright Vector store TestRail API WebSocket GPT-4o-mini
03

Pre-alpha · personal

Cadence

The whole inference chain on one laptop, nothing leaving it

source

The argument in Data residency picks the topology, taken to its limit and actually built. Cadence records a meeting from the microphone and produces a speaker-labeled transcript and a structured Minutes of Meeting — decisions, action items, risks — with no cloud STT, no cloud LLM and no telemetry. Everything runs on the machine. Writing it meant working below the API line: quantised GGUF weights, RAM budgets, native binaries orchestrated from Rust. That is the part I wanted to learn, and it is what makes me trust the self-hosted half of an architecture diagram.

  • Fully local pipeline whisper.cpp and sherpa-onnx for speech-to-text, llama.cpp for the minutes — all local GGUF models, nothing sent anywhere.
  • Code-switch aware Handles multilingual input, including English, Tamil and code-switched speech, not just single-language dictation.
  • Anonymous diarization Labels distinct speakers in the transcript without identifying who they are.
  • Grounded minutes Decisions and action items are generated with timestamp references back into the transcript, not free-floating summary text.
  • Model integrity + sizing SHA-256 verification on every model it manages, with RAM-aware selection so it runs on the hardware it's actually installed on.
Rust Tauri 2 React 19 TypeScript whisper.cpp sherpa-onnx llama.cpp SQLite
04

The layer underneath

MCP servers, RAG indexes and local inference the agents share

internal

Two agents in production means one shared substrate, not two copies of it: MCP servers that wrap internal systems once, a retrieval index over repositories and engineering docs, self-hosted models for anything confidential, and the same observability every other service on the platform gets.

  • MCP serversOne tool surface, reused by every agent.
  • RAG pipelinesRepo and doc retrieval over internal patterns.
  • Local + SLM inferenceSelf-hosted models where data cannot leave.
  • Agent observabilityToken spend, latency and failure rate on the same Grafana boards as the clusters.
03

Architecture.

Where I'm heading: from building individual agents to designing the systems they live in. Below is the platform the two agents actually share, and the decisions that shaped it.

DevOps → platform → AI solutions architect

the whiteboard version Shared agent platform — every agent is a thin surface over the same four layers.
GitLab MRs TestRail cases Repos + docs CI · CLI Agent orchestrator routing · retries · deduplication · durable state machine Model router frontier ↔ self-hosted SLM chosen on cost, latency, privacy MCP tool servers GitLab · TestRail · repo · CI each system wrapped once, not per agent Retrieval index repositories + engineering docs plus episodic memory of past reviews State store a restart never re-reviews what it already reviewed Observability Prometheus · Grafana · token spend carried as an SLO
Does this need an agent at all?

Most automation requests are a script with good error messages. I keep agents for work where the input is genuinely unstructured — a diff, a test case written in prose, an hour of unlabelled speech. Everything else gets a deterministic pipeline, because it is cheaper to run and far easier to debug at 3am.

Route models, don't marry one

Frontier models for reasoning over unfamiliar code; small self-hosted models for classification, extraction and anything touching internal data. The provider sits behind a factory, so switching is a config change. Cost per merge request is a design constraint, not a bill you read at the end of the month.

Retrieval is a data problem

Chunking strategy, index freshness and deciding what not to index moved review quality more than any prompt rewrite did. The biggest single win was episodic memory of past reviews — consistency across merge requests beat a larger context window.

Tool boundaries are the API design

Each internal system gets wrapped once as an MCP server, not once per agent. The second agent reused most of the first one's tool surface, and the next one will reuse more. Getting that surface right is most of the architecture work.

Earn autonomy in stages

Agents ship in shadow mode, then as an assigned reviewer, then as the default. Regression runs over real historical merge requests before each promotion, deduplication so a re-run never spams a thread, and a human who can always overrule the machine.

Data residency picks the topology

Self-hosted inference wherever data cannot leave the network, managed APIs where it can. In a manufacturing setting that one constraint shapes more of the design than model selection does — and it is the question I get asked first in every review. Cadence is that constraint taken to its limit: the entire chain on one machine, so I know first-hand what the self-hosted side actually costs.

In the end, agents are just services

The same SLOs, the same dashboards, the same on-call rotation. Token spend, latency and failure rate live on the cluster boards next to node pressure and pod restarts — because when an agent misbehaves at 2am, it pages the person who already carries the platform. Eight years of running infrastructure is what makes the AI part deployable rather than merely demonstrable.

04

Experience.

Six roles over eight years — healthcare operations, freelance cloud architecture, SRE, DevOps, and now infrastructure leadership.

2024
— now
CURRENT

Senior DevOps Engineer

ESAB India Limited · Chennai

End-to-end infrastructure ownership for a large-scale industrial platform. Leading a full-stack and DevOps team, standing up Hardware-in-the-Loop test rigs, and designing the AI agents above — CodeSentinel and TestFlow. Increasingly the architecture role rather than the build: choosing where inference runs, designing the MCP tool contracts and retrieval layer the agents share, and setting the evaluation gates an agent has to pass before it is trusted with a merge request.

Solution architectureKubernetesRAG · LLMsMCPHiL automationTeam lead
2023
— 2024

Senior DevOps Engineer

Ebot AI IT Solutions · Bengaluru

Deployed a chatbot platform for finance firms on Kubernetes, across private and public cloud and mostly on Azure. Owned the Helm charts and the Terraform that stood up the clusters and the surrounding cloud estate, and ran the dev and production environments on top of them.

KubernetesHelmTerraformAzureFintech chatbotGitLab CI
2022
— 2023

DevOps Engineer

Motherson Technology Services · Client: ESAB India

Managed AKS clusters and the surrounding Azure estate end to end. Built and maintained CI/CD on GitLab and Jenkins, including the IoT firmware build pipelines.

AKSAzureGitLab CIJenkinsIoT
2019
— 2022

SRE Engineer

Xmedia Solutions · Chennai

Full-lifecycle CRM development and deployment on Odoo, dev through production, plus provisioning and maintenance of the on-premise cluster it ran on.

OdooOn-premDockerLinux
2018
— 2019

DevOps & Cloud Architect

Freelance · Remote

Architected cloud infrastructure for monolith-to-microservices migrations. Built the CI/CD on GitLab and Jenkins, and the Python + Pytest automation frameworks that covered the new service surface.

GitLabJenkinsPythonPytestMicroservices
2017
— 2018

Healthcare & Insurance Operations

NTT Data Services · Chennai

Healthcare process and claims adjudication. Where I first saw how much of an enterprise runs on undocumented manual steps — which is more or less why I ended up automating things for a living.

Healthcare ITOperationsAdjudication
05

Infrastructure.

The platform work underneath — clusters, pipelines, migrations and test rigs. Most of it lives in private repositories, so this is the shape rather than the source.

W.01Kubernetes

A Helm catalogue for the whole stack

Twenty-plus production Helm deployments, written and maintained as a reusable internal catalogue: Odoo and ERPNext for business systems, Supabase and Zitadel for app platform and identity, Airbyte and Spark for data, Longhorn for storage, Elasticsearch and the kube-prometheus stack for search and observability, plus SonarQube, n8n, Appsmith, Retool, Redis and GitLab Runner. One chart pattern, one values convention, so a new service is a config change rather than a project.

20+charts
Helm+ ArgoCD
AKS · k3stargets
W.02Embedded

Hardware-in-the-Loop rigs

Embedded test automation on real hardware, wired into CI so firmware changes get the same gate as application code. Reproducible fixtures, bare-metal runners, results that a firmware engineer can read without a debugger attached.

HiLbare metal
CIgated
W.03Architecture

Microservices monolith

I have run this migration in both directions, which is mostly how I learned that neither is the answer. Splitting a monolith for a platform that needed it: API gateway, Node service backends, an async messaging tier, with cloud infrastructure, CI/CD and Pytest coverage designed alongside the split rather than bolted on after. Then, elsewhere, consolidating a fragmented set of services back into one deployable unit — deploy time down about 60%, three layers of inter-service auth deleted, module boundaries kept in the code rather than in the network. The deciding question was never the architecture diagram; it was whether the team could independently deploy and operate the pieces it was asking for.

−60%deploy time
bothdirections run
W.04Pipelines

IoT & firmware build pipeline

Multi-stage GitLab and Jenkins CI on AKS producing firmware artefacts and software services in one reproducible flow, with signed outputs and promotion between environments.

AKSrunners
GitLab+ Jenkins
W.05Platform

Mixed estate: clusters and VMs

Kubernetes for services, Proxmox VMs for the legacy workloads that were never going to containerise, k3s for edge and constrained nodes — and a single CI surface that ships to all three. Trade-offs written down, a runbook per component.

K8s + VMunified
k3sedge
W.06Developer tooling

Internal packaging & release

Private PyPI publishing pipeline and language-agnostic build templates, so internal libraries get versioned, tested and released the same way every time instead of being copied between repositories.

PyPIprivate
templatesreusable CI
W.07Observability

One pane for clusters and agents

Prometheus, Grafana and Loki across the estate, with the AI agents emitting to the same boards — token spend and agent failure rate sit next to node pressure and pod restarts, because in practice they page the same person.

Prom+ Grafana
Lokilogs
06

Stack.

Length of the pencil line is depth, roughly. Honest rather than flattering.

AI architecture
  • Agent system design
  • MCP servers
  • Model routing
  • RAG pipelines
  • Local LLM (Ollama)
  • Tool calling
  • Eval & guardrails
  • Inference cost modelling
  • SLMs · fine-tuning
Containers & orchestration
  • Kubernetes
  • AKS
  • Helm
  • Docker
  • k3s
  • Proxmox
IaC & delivery
  • Terraform
  • Ansible
  • GitLab CI
  • Jenkins
  • ArgoCD
  • GitHub Actions
Cloud & infrastructure
  • Azure
  • AWS
  • GCP
  • On-prem estates
  • Linux
Languages
  • Python
  • Bash
  • Go
  • JavaScript
  • YAML (regrettably)
Observability
  • Prometheus
  • Grafana
  • Loki
  • Elastic (ECK)
Testing
  • Pytest
  • Playwright
  • TestRail
  • HiL rigs
Embedded & hardware
  • HiL setups
  • Firmware CI
  • IoT pipelines
  • PLC integration
07

Contact.

Open to AI solutions architect, platform and senior DevOps roles — full-time or contract. Happy to go deep on any decision in section 03.

LinkedIn in/karthik-palani GitHub karthik1710 GitLab karthikpalani Email Say hello