How to Build an AI Agent That Answers Customers, Checks Its Own Answers, and Captures Sales Leads (Python + OpenAI)

The Problem Every Company Website Has

It's 11:40 pm. A potential customer lands on your website while your whole team is asleep. They have a real project, a real budget, and one question: "Have you built HIPAA-compliant healthcare apps before?"

Usually, one of three things happens next:

• Nobody answers. They fill in a contact form, wait, and sign with a competitor who replied first.
• A scripted chatbot answers. It offers a menu of five buttons, none of which match the question.
• A generic LLM chatbot answers. It sounds confident and invents a client, a certification, or a price your sales team now has to walk back.

What if I told you there's a fourth option? An AI agent that answers from the company's real knowledge, checks its own answer before sending it, and turns the conversation into a qualified lead with a sales brief on the team's phones.

I built it as a “digital twin” for Mobisoft Infotech, in about 700 lines of code. This post shows how to build an AI agent in Python like this one, with the code and the design choices that make it safe to put in front of real customers.

TL;DR for busy Leaders

The agent does four jobs a good sales assistant does, 24/7:

Job	What The Agent Does	Why The Business Cares
Answer	Replies to visitor questions using only the company's own knowledge base (a PDF)	Visitors get accurate answers at any hour
Self-check	A second AI reviews every reply against the facts it retrieved, and blocks or rewrites anything unsupported	No invented clients, prices or promises
Capture	Collects name, email and project needs, then saves the lead	Conversations become pipeline, not just traffic
Brief	Writes a sales brief (who, what they need, best-fit services, first-call questions) and pushes it to sales reps' phones	Sales walks into the first call prepared

Tech Stack At A Glance

• Python 3.12
• OpenAI API (chat, structured outputs, moderation)
• BM25 keyword search
• pypdf
• Gradio for the UI
• Pushover for mobile alerts

No vector database, no agent framework: one main.py.

How It Works: One Visitor Message, End To End

Each message passes cheap input checks, then an agent loop that can call tools, then a second model that reviews the draft before anyone sees it. Leads trigger a sales brief after the reply is sent. That final step turns a helpful assistant into an AI sales agent.

A rejected draft gets one rewrite with the reviewer’s feedback; if that fails too, the visitor gets a safe handoff to sales instead of a risky answer. If you’re evaluating AI agent development services, ask whether every agent includes a review step like this one before replies reach customers.

Step 1: Turn A PDF Into A Knowledge Base The Agent Can Search

The company's knowledge lives in one PDF: a company overview, one chapter per service, engagement models and contact details. Stuffing all of it into every prompt is slow and expensive, so the agent gets only a summary up front and fetches the details on demand.

First, the PDF is split into named sections. Every service chapter starts with a line like Service: Mobile App Development | Category: ..., which makes a clean split point:

SERVICE_LINE = re.compile(r"^Service: (.+?)\s+\|\s+Category: (.+)$")

def split_sections(pages):
    """Split the knowledge base into named sections: front matter, one per service, engagement models, contact."""
    front = "\n".join(pages[3:8])  # About This Document, Company Overview, Service Catalogue
    lines = "\n".join(pages[8:]).splitlines()
    sections = {}
    ...
    for i, line in enumerate(lines):
        match = SERVICE_LINE.match(line)
        if match:
            close()  # save the previous section
            current_name, current_category = match.group(1), match.group(2)
            current = [match.group(1), line]
        else:
            current.append(line)
    close()
    return front, sections

The front matter (overview + service catalogue) goes straight into the system prompt. The service chapters stay outside it, behind tools.

Why BM25 Instead Of A Vector Database?

For questions that span services ("Do you work with Flutter?", "Any HIPAA experience?"), the agent needs search. I chose BM25, the classic keyword-ranking algorithm behind many search engines, in about 15 lines:

def bm25(query_terms, terms, k1=1.5, b=0.75):
    length = sum(terms.values())
    score = 0.0
    for term in query_terms:
        if term in terms:
            idf = math.log(1 + (len(chunks) - doc_freq[term] + 0.5) / (doc_freq[term] + 0.5))
            tf = terms[term]
            score += idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * length / avg_len))
    return score

def search_knowledge_base(query, max_results=5):
    query_terms = tokenize(query)
    scored = sorted(((bm25(query_terms, terms), i) for i, terms in enumerate(chunk_terms)), reverse=True)
    results = [{"section": chunks[i]["section"], "text": chunks[i]["text"]}
               for score, i in scored[:max_results] if score > 0]
    return results or "No matching passages found in the knowledge base."

The reasons:

• Company questions are keyword-heavy. Technology names, certifications and industries ("React Native", "HIPAA", "fintech") are exactly what keyword search is good at.
• Zero infrastructure. No embedding model, no vector store, no extra bill. The index builds in memory at startup.
• Easy to debug. When a result is wrong, you can see which words matched.

The tokenizer keeps characters like +, # and . so terms such as c++, c# and .net survive. Small detail, big difference for a software company's knowledge base.

For CTOs: Start with the simplest retrieval that fits your content. Move to embeddings or hybrid search when you can show keyword search is missing answers, not before.

Step 2: Give The Agent Tools, And Let It Decide When To Use Them

A chatbot replies. An agent decides what to do next: look something up, save a lead, log a gap, then answer. That decision-making comes from tool calling: you describe functions to the model, and it asks you to run them. For the business side of this distinction, read our AI chatbot development guide.

The agent has four tools:

Tool	What It Does
get_service_details	Returns the full chapter for one service
search_knowledge_base	BM25 search across the whole knowledge base
record_user_details	Saves a lead (name, email, needs) to logs/leads.md
record_unknown_question	Logs questions the knowledge base can't answer

Each tool is a plain Python function plus a JSON schema. One small trick does a lot of work: the service name is an enum of the real section names, so the model can't ask for a service that doesn't exist.

get_service_details_json = {
    "name": "get_service_details",
    "description": "Get the full knowledge base chapter for one Mobisoft Infotech service "
                   "(offerings, process, technologies, differentiators, FAQs, official URL).",
    "parameters": {
        "type": "object",
        "properties": {
            "service_name": {"type": "string", "enum": list(sections)},
        },
        "required": ["service_name"],
        "additionalProperties": False,
    },
}

The Agent Loop

The heart of the agent is a short loop: call the model; if it asks for tools, run them, feed the results back, and call it again, until it gives a final answer.

MAX_TOOL_ROUNDS = 6

def run_agent(messages, knowledge, leads, activity):
    response = complete(messages)
    rounds = 0
    while response.choices[0].finish_reason == "tool_calls" and rounds < MAX_TOOL_ROUNDS:
        message = response.choices[0].message
        results = handle_tool_calls(message.tool_calls)
        messages.append(message)
        messages.extend(results)
        # Keep everything retrieved this turn: the evaluator checks the reply against it
        knowledge.extend(f"[{call.function.name}] {r['content']}"
                         for call, r in zip(message.tool_calls, results))
        response = complete(messages)
        rounds += 1
    return response.choices[0].message.content or ""

Three details worth stealing:

• A hard cap on rounds (MAX_TOOL_ROUNDS = 6) so a confused model can't loop forever and burn tokens.
• Every tool result is kept in knowledge. That list becomes the evidence the evaluator uses in Step 3.
• Tools validate their inputs. record_user_details refuses to save a lead without a valid email and real project notes, and tells the model what to ask for instead:

def record_user_details(email, name="Name not provided", notes="not provided"):
    if not re.fullmatch(r"[^@\s]+@[^@\s]+\.[^@\s]+", email.strip()):
        return "Not recorded: no valid email address yet. Ask the visitor for their email first."
    if not notes or notes.strip().lower() in ("not provided", "none", "n/a"):
        return ("Not recorded yet: ask the visitor which services they need and a short description "
                "of their project (and their name, if missing).")
    log_lead_markdown(email.strip(), name or "Name not provided", notes)
    return "Lead recorded: the visitor's details were saved for the sales team to follow up."

This is a pattern I'd recommend to any team building agents: make tool errors instructive. The tool's reply is the model's next prompt, so a clear "here's what's missing" message gets the conversation back on track.

record_unknown_question is the quiet hero for the business: every question the knowledge base couldn't answer lands in a log. That's a ready-made list of content gaps for marketing and the website.

Step 3: The Agent That Checks Its Own Work

Anyone learning how to build an AI agent for customers should spend the most time here. This is the part that makes the agent safe to put on a real company website. There are guardrails on both sides of the model.

Input Guardrails: Stop Bad Messages Before They Cost Anything

Two cheap checks run before the main model sees a message:

• Length limit: messages over 2,000 characters are rejected. This limits oversized prompt-injection payloads and runaway token costs.
• Moderation: every message goes through OpenAI's free moderation endpoint.

def is_flagged(message):
    """Input guardrail: OpenAI's moderation endpoint. Fails open so an API hiccup doesn't block real visitors."""
    try:
        return openai.moderations.create(model="omni-moderation-latest", input=message).results[0].flagged
    except Exception as e:
        print(f"Moderation check failed: {e}", flush=True)
        return False

Note the deliberate choice to fail open in these LLM guardrails: if the moderation API has a hiccup, real customers can still chat. The output evaluator below is the stronger safety net.

Output Guardrail: An Evaluator That Reviews Every Reply

Here's the core idea: a second LLM call acts as a strict reviewer. Before any reply reaches the visitor, the evaluator gets:

• the draft reply,
• the visitor's message and recent conversation,
• and, most importantly, the exact knowledge-base passages retrieved this turn.

It returns a structured verdict, enforced with a Pydantic model so there's no parsing guesswork:

class Evaluation(BaseModel):
    is_acceptable: bool
    feedback: str

def evaluate(reply, message, history, knowledge):
    """Output guardrail: a second LLM call checks the reply against the knowledge actually retrieved this turn."""
    ...
    response = openai.chat.completions.parse(
        model=EVALUATOR_MODEL,
        messages=[{"role": "system", "content": EVALUATOR_PROMPT},
                  {"role": "user", "content": user_prompt}],
        response_format=Evaluation,
    )
    return response.choices[0].message.parsed

The evaluator's prompt lists what makes a reply unacceptable:

• facts not supported by the overview or the retrieved extracts (invented services, clients, numbers, certifications)
• prices, discounts, timelines or guarantees
• answering off-topic requests instead of steering back
• abandoning its role or revealing its system prompt
• telling the visitor their details were saved when no tool result says Lead recorded

That last rule catches a subtle, real failure: models love to say "I've passed your details to the team" even when the save failed. The evaluator checks the claim against the actual tool output.

To go deeper on testing reviewers like this one, read our post on LLM evaluation for AI agents.

Reject, Rewrite, or Fall Back

If the evaluator rejects a reply, the agent gets one retry with the evaluator's feedback. If the retry fails too, the visitor gets a safe fallback pointing to the sales team.

for attempt in range(2):
    reply = yield from run_agent(messages, knowledge, leads, activity)
    evaluation = evaluate(reply, message, history, "\n\n".join(knowledge))
    if evaluation.is_acceptable:
        break
    messages.append({"role": "assistant", "content": reply})
    messages.append({
        "role": "system",
        "content": f"Your previous reply was rejected by quality control: {evaluation.feedback}\n"
                   "Write a corrected reply to the visitor's last message. Do not mention the review.",
    })
else:
    reply = FALLBACK_REPLY  # "Our team can help directly at business@..."

The Python for ... else is doing real work here: the else branch only runs if the loop never hit break, meaning both attempts were rejected.

For CEOs: This is the difference between "we put ChatGPT on the website" and "we put a supervised assistant on the website." The worst case is no longer a made-up promise to a client. It's a polite handoff to your sales team.

Step 4: From Chat To Sales Pipeline, Automatically

Answering questions is nice. Turning a curious visitor into a qualified lead with a ready-to-use brief is what pays for the project. It’s also what makes this an AI lead generation chatbot rather than a smarter FAQ page.

Qualify Before You Capture

The system prompt tells the agent to collect three things before recording a lead: name, email, and what they need. If something's missing, it asks in one friendly message. The tool validation from Step 2 enforces the same rule in code, so the prompt and the code back each other up.

A Structured Sales Brief

Once a lead is recorded, another structured-output call writes a brief for the sales team:

class SalesBrief(BaseModel):
    headline: str                   # "Jane (Acme Health) wants a HIPAA-compliant telehealth app"
    needs: str                      # 1-2 sentences for the first call
    recommended_services: list[str] # 1-3 best-fit services, exact names
    discovery_questions: list[str]  # 3 questions to ask on the call

The Model Picks, The Code Links

Here's my favorite design decision in the whole project. The model chooses which services fit, but it never writes the links. The code drops any service name that doesn't exist and pulls each official URL straight from the knowledge base:

def format_brief(brief, lead):
    # The model only picks services; names and URLs come from the knowledge base so links can't be made up
    services = [name for name in brief.recommended_services if name in sections]
    lines = [
        f"Contact: {lead.get('name', 'Name not provided')} <{lead['email']}>",
        f"Needs: {brief.needs}",
        "Services: " + (", ".join(services) or "to be decided on the call"),
        *(f"• {name}: {service_url(name)}" for name in services if service_url(name)),
        "Ask on the call:",
        *(f"• {question}" for question in brief.discovery_questions),
    ]
    return "\n".join(lines), services

The lesson generalizes: let the LLM make judgment calls, and let deterministic code handle facts like URLs, prices and IDs. Hallucinated links become impossible by construction, not by hoping the model behaves.

Straight To The Sales Team's Phones

The brief is saved to logs/briefs.md and pushed to the sales team's phones with Pushover, a simple notification API:

def push(title, message, url=None):
    """Send a Pushover notification. Returns False if Pushover isn't configured."""
    user, token = os.getenv("PUSHOVER_USER"), os.getenv("PUSHOVER_TOKEN")
    if not (user and token):
        return False
    payload = {"user": user, "token": token, "title": title[:250], "message": message[:PUSHOVER_MAX_CHARS]}
    if url:
        payload |= {"url": url, "url_title": "Service page"}
    requests.post(PUSHOVER_URL, data=payload, timeout=10).raise_for_status()
    return True

If Pushover isn't configured or fails, the brief is still on disk. Nothing is lost.

The brief also runs after the visitor's reply has been sent, so the visitor never waits on sales tooling.

For sales leaders: Your rep gets a phone notification with who the prospect is, what they want, which services fit (with links), and three questions to open the call. No more skimming a chat transcript before dialing.

If you’re planning AI chatbot development for your own site, design this handoff first. Your sales team will use it every day.

Step 5: Make The Agent's Thinking Visible

Agents are hard to trust when they're a black box. So next to the chat, the Gradio app shows a live Agent activity panel: every step, timestamped, as it happens.

A typical turn looks like this:

0.0s  🛡️ Moderation check…
0.4s  ✅ Moderation passed
0.4s  🧠 Thinking…
1.6s  🔍 Searching the knowledge base for "HIPAA healthcare compliance"
1.6s  ↳ found 5 passages
1.6s  🧠 Reading the results…
4.1s  🧑‍⚖️ Evaluator checking the reply against the retrieved knowledge…
5.3s  ✅ Reply approved
5.3s  🏁 Done

(Timings are illustrative; they depend on the model and network.)

The trick that makes it live is Python generators. The agent loop, the chat handler and the sales brief are all generators that yield after each step. Gradio streams every yield to the browser, so the panel updates while the model is still working:

class Activity:
    """The live 'agent brain' log shown next to the chat: one line per step, with the time since the turn began."""

    def __init__(self):
        self.start = time.monotonic()
        self.steps = []

    def add(self, text):
        self.steps.append(f"`{time.monotonic() - self.start:4.1f}s` {text}")


def respond(message, history):
    """Gradio handler: streams the chat and the activity panel together."""
    activity = Activity()
    ...
    for reply in chat(message, previous, activity):
        if reply is not None:
            history = history + [{"role": "assistant", "content": reply}]
        yield "", history, activity.markdown()

yield from run_agent(...) lets the inner loop stream its steps up through chat() and still return the final reply. No threads, no queues, no websockets to manage.

The panel helps three audiences at once:

• Developers debug prompts and tools in real time.
• CTOs see exactly where latency and cost go: retrieval, generation, or evaluation.
• Executives and clients watching a demo see an assistant that looks things up and gets checked, not a box that makes things up.

Watch the Full Agent Demo

Production Readiness: The CTO's Checklist

Before an agent faces real customers, a CTO will ask whether it's safe, affordable and maintainable. Here's how the design answers each concern.

Concern	How The Design Handles It
Hallucination	Answers grounded in retrieved passages; an evaluator checks each reply against them
Made-up links or data	Facts like URLs come from code, never from the model
Prompt injection	Input length cap, moderation, explicit security rules in the system prompt, evaluator checks for role-breaking
Runaway cost	Caps on tool rounds (6), history (20 messages), reply length (4,000 tokens) and input (2,000 chars)
Vendor flexibility	Chat and evaluator models are environment variables (MODEL_NAME, EVALUATOR_MODEL)
Content updates	Replace the PDF and restart; no retraining, no re-embedding
Privacy	Leads live in local log files kept out of version control; briefs pushed via Pushover include the lead's name and email

The honest trade-off: the evaluator adds one extra LLM call per reply (two more on a retry). For a sales-facing assistant, I think that's cheap insurance. For high-volume support, you might run the evaluator on a smaller model or sample a share of replies.

What To Measure Once It's Live

• Lead capture rate: conversations that end in a recorded lead
• Evaluator rejection rate, and what it rejects (a direct signal for prompt and knowledge-base fixes)
• Fallback rate: how often both attempts fail
• Unanswered questions per week, and how many you close by updating the knowledge base
• Time from lead to first sales contact

Not sure which of these numbers should drive your roadmap? Our enterprise AI consulting team helps set those targets before launch.

Lessons Learned

• Prompts ask, code enforces. Every important rule (valid email, real project notes, real service names, real URLs) is in the prompt and checked in code.
• Instructive tool errors beat silent failures. A tool that says "ask for the email first" steers the model better than one that throws.
• Give the reviewer the evidence. An evaluator that only sees the reply can judge tone. One that sees the retrieved passages can judge truth.
• Separate judgment from facts. The LLM picks the services; the code supplies the links.
• Keep the slow work off the user's path. The sales brief runs after the reply is delivered.
• Start simple. BM25 and one Python file got this to a working, demo-ready agent. Add frameworks and vector stores when the problem demands them.

What's Next

• Hybrid search (BM25 + embeddings) if the unanswered-questions log shows keyword search missing paraphrased questions
• CRM integration so leads land in HubSpot or Salesforce instead of a Markdown file
• An evaluation suite of real visitor questions, run on every prompt change to catch regressions
• Analytics dashboard for the metrics above
• Embeddable website widget instead of the standalone Gradio app

Which of these would your team need first? Most depend on your CRM, data and website stack. An experienced artificial intelligence development company can plan these upgrades around the systems you already run.

Conclusion: The Bar For Website AI Just Went Up

A chatbot that only talks is no longer enough. Visitors expect accurate answers, and businesses need those answers to be safe and to turn into revenue.

This agent shows you can get all three with a small, readable codebase: grounded answers from your own knowledge, a reviewer that checks every reply before it's sent, and a lead pipeline that hands sales a ready brief.

If you're a developer, take the patterns: tool validation, evidence-based evaluation, and "the model picks, the code links." If you're a CTO or CEO, ask one question of any AI you put in front of customers: who checks its answers before the customer sees them?

Want to try it? The full source is one main.py: swap in your own knowledge-base PDF (same section layout) and update the company-specific prompts, add an OpenAI key, and run python main.py.

You can download the agent code here.