Sergio Adan Bermudez

AI engineer · PhD, sensing and measurement · Barcelona

I make AI systems fast on real hardware, and I measure every change.

Nine years of optical sensing taught me to distrust a number I have not measured. I now apply that to local language models, AI agents and the engines underneath them.

800 nm pulsed laser→mirror→objective→sample→1270 nm emission→InGaAs spectrometer

Voice, time to first word3.0 s → 0.7 sLocal model, mid-conversation
Model state snapshot0.61 s → 0.03 sllama.cpp, AMD integrated GPU
Photon order, measured by an agent2.004 ± 0.03Hidden answer: 2. Control run: 1.003

Step inside

three projects

Signature piece

photonics-lab-mcp

An AI agent runs a physics experiment, inside limits it cannot override

An MCP server gives a local 35B model an optics bench. One question, no procedure: is the 1270 nm emission excited by one photon or two? The answer is hidden in the server. Below is the agent's real run, call by call.

agent · Qwen3.6-35B · local
#25acquire_spectrum({"label":"P160mW"})max 6,636
#26fit_power_dependence({"run_ids":[6 runs],"center_nm":1270})n = 2.004 ± 0.03
#27laser_emission({"on":false})
#28safe_state({"reason":"measurement complete"})safe
→conclusion: two-photon, exponent 2.004 ± 0.03correct
peak area vs power · log-logslope = photon order
Peak area against laser power on log-log axes. Two-photon sample: slope 2.004 plus or minus 0.03. One-photon control: slope 1.003 plus or minus 0.002. 1020 50100 200 10³10⁴10⁵ Laser power at 800 nm (mW) this run: n = 2.004 ± 0.03 control run: n = 1.003 ± 0.002

Right twice. The control run uses a one-photon sample under the same 800 nm question, and the agent measured 1.003. It read the slope instead of reciting the textbook answer.

Wrong twice, too. Both closing explanations got the photophysics wrong: the sensitiser absorbs, and its triplet state passes the energy to oxygen. So the server owns safety and statistics, and a person owns interpretation.

The full lab-agent write-up →

Case study

llama.cpp · Vulkan

A 20x stall, found one measurement at a time

Every reply from my local assistant stalled for over a second. The cause: the CPU was reading model snapshots straight out of GPU memory it cannot cache. The animation runs at the measured speeds.

Reading 880 MB of model state on an integrated GPUdots move at measured throughput (14 : 1)
Before: the CPU reads GPU memory directly at 114 megabytes per second, 7,738 milliseconds. After: the GPU first copies into a cached staging buffer, 553 milliseconds. GPU memory shared with the CPU, not CPU-cached 63 MB per snapshot 2 per request CPU · host RAM llama.cpp server saves the snapshot BEFORE · CPU reads directly · 114 MB/s 7,738 ms AFTER · GPU copies to a cached buffer first cached staging buffer 553 ms one added condition: read directly only when the memory is CPU-cached

Same output word for word at temperature 0, before and after. The same fix had been merged upstream two weeks earlier (llama.cpp #30049), so this is an independent diagnosis, not a contribution.

All eight steps, the code and the verification →

The system

Odysseus-Jarvis

one laptop, 64 GB

A private assistant on one laptop, every path measured

Voice, an agent and a chat hub share one local model server, so one engine fix sped up every local path. Claude Code handles typed work.

requests in flightno cloud inference on the local paths
Voice loop, Hermes agent and Odysseus hub send requests to one llama.cpp server running Qwen3.6-35B on Vulkan. The agent runs code in a Docker sandbox; voice turns are traced in Phoenix; memory is an MCP server used by the agent and Claude Code. Voice loopwake word · 0.7 s first word Hermes agentweb search · code tool Odysseus hubself-hosted chat Claude Codetyped work, cloud llama.cpp server Qwen3.6-35B-A3B Vulkan · Radeon 8060S patched snapshot path 0.03 s per snapshot Memory (MCP)plain-text vault · 29 tests Docker sandboxagent code, no host folders Traces (Phoenix)one span per voice turn Evals (Inspect AI)13 tasks · 13/13
Observability
Live latency dashboard and OpenTelemetry traces for every voice turn
Evals
13 real tasks with deterministic scoring, used to pick settings from evidence
Think harder
A 284B model streamed from disk on demand, about 0.5 tokens per second
Brain choice
Code with Claude Code, or for free with OpenCode on the local model or free cloud models

The day job

AI engineering roles

What these roles do every day, and where I have done it

To show I can do the daily work, not just describe it, I ran one engagement end to end, from client brief to handover. The client is simulated; the code, evaluation and numbers are real.

Flagship · simulated engagement

LabDesk: a support assistant that answers only from the docs

A lab-automation team asked for PyVISA answers they could trust. Brief, testable requirements, BM25 retrieval with checked citations, a refusal rule, a read-only power-meter tool, FastAPI service with telemetry, a 41-question eval run on a local model, ADRs and a handover runbook.

31/33answers correct
32/33cite the right section
6/6off-topic questions refused
5.6 sp95 latency, local model
Read the engagement on GitHub →
RoleWhat they do dailyWhere I have done itFit
AI engineerBuild retrieval and agent systems, wire models to tools, write evals, ship them as servicesLabDesk; the local Jarvis stack; photonics-lab-mcp; jarvis-chessShown
Forward-deployed engineerSit with a customer, turn a messy problem into requirements, deploy on their infrastructure, hand it overLabDesk, brief to handover, on on-premises hardware; customer-facing technical work at SensofarShown
AI agent opsMonitor agents in production: traces, latency, cost, failures, incident runbooksTracing and latency dashboard for Jarvis; LabDesk telemetry, metrics endpoint and runbookShown
AI requirements analystTranslate business needs into testable requirements and acceptance metricsLabDesk requirements, each with a target and how it is verifiedPartly shown
Product engineerOwn a feature end to end, including the interface people useThis site; the in-browser chess analysis board with StockfishPartly shown
Experience designUser research, interaction design, usability testingNo formal user research yetGap

Engineering log

October 2026

Every change, measured, including the ones I rejected

DateChangeMeasured resultDecision
20 OctLabDesk docs assistant, 41-question eval on a local model31/33 correct, 6/6 refusals, p95 5.6 s; 2 real failures left untunedShipped
20 OctInstrument-control MCP server, agent runs an experimentHidden answer found in both runs; closing physics wrong both timesKept
20 OctAgent code tool moved into a Docker sandboxCode task 7.9 s instead of blocked; host writes fail insideKept
20 OctTracing, latency dashboard, evals in Inspect AI13/13 tasks in 45 s; one trace per voice turnKept
20 OctA 284B model streamed from disk as a "think harder" modeCorrect on a reasoning test, 0.4 to 0.6 tokens/s on CPUOn demand
20 OctContext compression proxy in front of the model0 tokens saved on real tool outputs, three test designsRejected
20 OctSnapshot read fix in the Vulkan backend0.61 s to 0.03 s, identical outputKept
20 OctSnapshots turned off entirelyVoice 0.5 s, but agent turns 21 sReverted
20 OctAgent prompt trimmed by usage data13,718 to 11,560 tokens, tools still workKept

About

From measuring light to measuring models

  1. 2024 to nowKey Account Manager and Product Specialist, Sensofar. Optical metrology for semiconductors, medical devices and aerospace. Built an internal LLM assistant for product knowledge.
  2. 2022 to 2024Postdoctoral Researcher, IQS, Universitat Ramon Llull. Biomedical photonics and sensing; EU project Light4Lungs.
  3. 2017 to 2022PhD, Intelligent Sensing and Measurement, University of Edinburgh. Two-photon detection in hollow-core photonic crystal fibres.
  4. 2014 to 2017MSc Nanoscience, University of Copenhagen; MSc Engineering, University of Glasgow.

Now: an eight-week C++ and GPU systems lab, an official llama.cpp build with the upstream fix, and the instrument server on real hardware.

Contact

Open to AI engineering roles

Applied AI, LLM systems, inference performance and evaluation. Based in Barcelona; Spanish and English.

LinkedIn