What our vetting tests
In production a model has to answer quickly under real load and stay within its budget, month after month. That is the work we test for.
Every candidate goes through three steps before reaching your shortlist: a technical skills assessment, a soft-skills evaluation and an analysis of their job history. For AI and machine learning candidates, the assessment covers production scenarios, from designing a RAG pipeline or an evaluation set to trading latency against cost and handling model drift. For some roles a live coding interview follows, where the candidate writes code and talks through each choice. A background check runs only when a client asks for one.
Questions to ask before hiring an AI engineer
Whoever you hire, from us or elsewhere, ask how they handled these in a system that was already live.
- How did they keep latency, the time a user waits for an answer, acceptable under real load?
- How did they notice model drift, whether live data moved away from the training data or a provider changed the model behind its API?
- How did they cap token and compute costs?
- How did they test output against a fixed evaluation set, with Ragas or a harness of their own, before and after each change, and what did they trace once the system was live?
A strong answer names the trade-off and what was measured. A vague one usually means the work stopped at the notebook.
When you take on AI engineers who work inside your team, you interview each candidate before anyone starts and can turn any of them down. With a managed AI engineering team, we put the team together and answer for it, and you can interview the key roles.
Access to your systems
Our engineers reach your systems, data and model APIs only through named accounts you issue, with the access their role needs. All of it is removed when they leave the engagement.
AI work we can show you
R&D tax report generation is our published generative AI case, a RAG system on Azure OpenAI that drafts a consulting firm's reports inside the firm's own Azure account. For predictive machine learning, start with logistics demand forecasting, a forecasting system we rebuilt on Databricks and Azure. More of our AI work sits under client NDAs.