Using AI to Onboard New Engineers 2x Faster

We hired three engineers in the second half of last year. Our onboarding had always been a problem — it typically took 6-8 weeks before a new hire was making meaningful contributions, and the process ate a lot of senior engineer time. With the team growing, I couldn’t afford to keep losing that much productivity to onboarding.

I experimented with using AI tools to accelerate the process, and the results were good enough that I’m writing about it. Our last two hires were making real contributions within about 3 weeks. That’s not entirely attributable to AI — we also improved our documentation and onboarding structure — but AI was the biggest force multiplier.

Here’s what we did.

The Old Onboarding Problem

Our onboarding used to look like this:

1. Week 1: Set up dev environment, read documentation (which was outdated), attend architecture overview meeting
2. Weeks 2-3: Shadow a senior engineer, ask lots of questions, do small bug fixes
3. Weeks 4-6: Work on a starter project with heavy mentoring
4. Weeks 6-8: Gradually take on real tickets

The bottleneck was always the same: understanding our codebase. We have a large, domain-specific system with lots of implicit knowledge — conventions that aren’t documented, architectural decisions whose rationale is buried in old PRs, and business logic that only makes sense if you understand the domain.

New hires would spend hours reading code, trying to trace data flows, and then asking a senior engineer “what does this function actually do?” The senior engineer would spend 15-30 minutes explaining context, the new hire would nod, and three days later they’d have a similar question about a different module. It was slow and it consumed senior engineer time — we estimated about 3-4 hours per day for the designated mentor.

The AI-Assisted Onboarding System

1. Codebase Q&A Bot

The highest-impact change was giving new hires the ability to ask questions about the codebase to an LLM preloaded with context. We set up a simple workflow using Claude’s API:

# Simplified version of our onboarding assistant
import os
from anthropic import Anthropic

def build_context(repo_path: str) -> str:
    """Build context from key files in the repo."""
    context_files = [
        "README.md",
        "docs/architecture.md",
        "docs/data-flow.md",
        "docs/conventions.md",
        "src/pipeline/README.md",
        "src/api/README.md",
    ]
    
    context = "# Codebase Documentation\n\n"
    for f in context_files:
        path = os.path.join(repo_path, f)
        if os.path.exists(path):
            with open(path) as fh:
                context += f"## {f}\n\n{fh.read()}\n\n"
    
    return context

SYSTEM_PROMPT = """You are a codebase assistant for new engineers. 
You have access to the project documentation below. Answer questions 
about the codebase, architecture, and conventions.

Rules:
- If you're not sure about something, say so explicitly
- Reference specific files and functions when possible
- Explain the WHY behind decisions, not just the WHAT
- If a question requires information you don't have, suggest who 
  to ask or where to look

{context}"""

client = Anthropic()

def ask(question: str, context: str) -> str:
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=2048,
        system=SYSTEM_PROMPT.format(context=context),
        messages=[{"role": "user", "content": question}]
    )
    return response.content[0].text

In practice, we wrapped this in a Slack bot so new hires could ask questions in a dedicated channel. The bot would answer based on our documentation, and if it wasn’t confident, it would suggest asking a specific team member.

What happened: The first new hire to use this asked it 47 questions in her first week. She told me that about 35 of those would have been questions she’d have asked a senior engineer. Even accounting for the bot getting some answers wrong (she estimated about 20% needed follow-up), that’s roughly 25 questions that didn’t consume a senior engineer’s time.

The mentor time dropped from 3-4 hours/day to about 1-1.5 hours/day. The remaining mentoring was higher quality too — focused on nuanced decisions and domain context rather than “what does this function do.”

The downsides: The bot occasionally gave confidently wrong answers. We had one incident where it explained a data flow incorrectly because the documentation hadn’t been updated after a refactoring. The new hire spent half a day implementing something based on the wrong understanding before her mentor caught it. After that, we added a disclaimer to every bot response: “Verify with a team member before making architectural decisions based on this answer.”

We also found that the bot was too good at answering surface-level questions, which sometimes prevented new hires from developing the deeper understanding that comes from digging through code themselves. There’s a balance between efficiency and learning that we’re still figuring out.

2. Guided Code Exploration

Instead of “read the codebase” (which is overwhelming), we created guided exploration exercises that used AI to make the codebase interactive:

# Onboarding Exercise 3: Understanding the Event Pipeline

## Goal
Understand how events flow from ingestion to storage.

## Instructions
1. Open `src/pipeline/consumer.py`
2. Ask the codebase assistant: "Trace the path of a single event 
   from the Kafka consumer to the database. What transformations 
   happen along the way?"
3. Read the files it references and verify its explanation
4. Find the answer to: "What happens if an event fails validation?"
5. Draw a diagram of the data flow (pen and paper is fine)

## Checkpoint
Show your diagram to your mentor. Discuss:
- Why do we validate at this stage rather than earlier?
- What happens to failed events? Should they be retried?
- Where are the potential bottlenecks?

The key design principle: the AI provides explanations, but the new hire has to verify and discuss. This keeps the learning active rather than passive.

We created 8 of these exercises covering the major systems. A new hire can work through them in about 4-5 days, and by the end they have a working mental model of the codebase that used to take 3-4 weeks to build through osmosis.

3. PR Review Training

One of the best ways to learn a codebase is reviewing other people’s PRs. But new hires don’t have enough context to give useful reviews, so they usually just read PRs passively.

We started using AI to scaffold the review process:

This turned PR review from a passive reading exercise into an active learning opportunity. New hires started leaving real review comments within their second week, which accelerated their integration into the team’s workflow.

4. Context-Aware Pair Programming

For the starter project phase, we paired new hires with an AI assistant rather than a senior engineer (with the senior available for check-ins):

The “don’t write the code for me” instruction is important. If the AI just generates the solution, the new hire doesn’t learn the codebase patterns. By guiding rather than generating, the AI acts more like a patient mentor who never gets frustrated by basic questions.

This worked better than I expected. Our second hire used this approach for his starter project and completed it in 4 days instead of the usual 8-10. More importantly, the code he wrote followed our conventions closely because the AI was constantly nudging him toward existing patterns.

Where it didn’t work: Complex architectural decisions still required human mentoring. The AI would sometimes guide the new hire toward a technically correct but architecturally wrong solution — like implementing rate limiting per-endpoint when our convention is to use middleware. The AI didn’t have enough context about our architectural preferences to catch this.

The Numbers

Across our last 3 hires, comparing to the previous 3:

These numbers are directional, not rigorous. We also improved our documentation and onboarding structure simultaneously, so I can’t attribute all the improvement to AI. My best estimate is that AI accounts for about 60% of the improvement, with better docs and structure making up the rest.

What I’d Warn You About

Don’t Replace Human Mentoring

The AI is a supplement, not a replacement. New hires still need to build relationships with the team, understand the organizational context, and absorb the unwritten culture. An AI can explain what the code does. It can’t explain why we made the tradeoffs we made, what the team values, or how to navigate the organization.

We made sure every new hire still had a designated human mentor and regular 1-on-1s. The AI just reduced the transactional part of mentoring (answering factual questions) so the human mentoring could focus on the relational part.

Keep the Documentation Updated

The AI is only as good as its context. When our documentation was stale, the bot gave wrong answers, and wrong answers during onboarding are worse than no answers — they create incorrect mental models that are hard to correct later.

We now have a rule: any PR that changes architecture must update the relevant doc. This was good practice before AI, but now it’s essential.

Watch for Over-Reliance

One of our hires started asking the AI everything instead of reading the code himself. His understanding was broad but shallow — he could describe what modules did but couldn’t debug issues because he’d never actually traced through the code. We had to course-correct by adding explicit “read the code yourself, then verify with the AI” instructions to the exercises.

The goal is a new hire who can eventually work without the AI assistant. If they can’t, the onboarding failed even if they shipped code quickly.

Making This Work for Your Team

If you want to try this, the minimum viable version is surprisingly simple:

1. Write up your key documentation — architecture overview, data flows, conventions. Even rough docs are better than none.
2. Set up a Slack bot (or any interface) backed by an LLM with your docs as context
3. Create 3-5 guided exploration exercises for your main systems
4. Tell your new hires: “Use this for factual questions. Come to me for judgment calls.”

The bot setup took me about half a day. The exercises took about two days to create. The ROI on that investment has been substantial — especially given that we’re likely to keep hiring.

You might also like

• How AI Transformed Our Sprint Planning (Lessons from 6 Months)
• 5 Ways I Use AI as an Engineering Manager (That Aren’t Coding)
• How I Use AI to Prepare Better Technical Interviews

📦 Free: AI Code Review Prompt Pack — 10 prompts I use on 15+ PRs/week.

Newsletter: One practical AI workflow per week, plus templates I don’t publish here. Subscribe →