1. System Overview
The chatbot is a controlled workflow engine. PHP owns the web app and security gates; Python owns the AI conversation decision pipeline.
2. Candidate Message Flow
This is what happens after a candidate types a message in the secure chat window.
3. Python AI Pipeline
This is the controlled decision path inside /conversation/respond.
4. How It Works
Use this section when explaining the system to a recruiter, client, or technical reviewer.
System Overview
The candidate does not talk directly to the AI model. The candidate talks to the PHP JobPortal application through a secure chat token. PHP checks the token, conversation status, AI control mode, safety locks, booking status, and other business rules before the Python AI engine is allowed to respond.
The Python AI engine is called through AiEngineClient. That client adds the shared AI token, timeout protection, and circuit-breaker behavior so the website stays stable even if the AI service is unavailable.
The database is the source of truth. It stores conversations, candidate messages, AI messages, interview bookings, candidate records, secure tokens, and runtime analysis. The AI pipeline reads that context and writes back its result, but PHP remains responsible for the web experience and email side effects.
Candidate Message Flow
When a candidate sends a message, PHP first validates the request. If the conversation is closed, locked, in human-control mode, or not ready for AI, PHP stops locally and does not call Python.
If AI is enabled, PHP sends the message and conversation ID to /conversation/respond. Python loads the recent conversation history, understands what the candidate said, decides the next scheduling action, saves the AI response and state, then returns the result to PHP. PHP then sends any required PIN, confirmation, or self-schedule emails.
Python AI Pipeline
The Python pipeline is the controlled decision path. It loads context, checks safety, verifies PIN requirements, runs the intent router, handles scheduling state, intercepts side questions like pay or benefits, builds a professional reply, and persists the result.
The model does not just generate random text. Its main skill is understanding messy candidate language and turning it into structured meaning that the workflow can trust.
Where The Trained Model Shows Its Skill
The trained model shows off at the Intent Router step. That is where the system converts real human language into intent, next_action, confidence, and extracted signals.
Yes, these intents are learned from patterns in many real or training conversation examples. The model learns that different phrases can mean the same candidate intention, even when the exact words are different.
Candidate says: Yes, I am interested Sure, that works I can do it Sounds good Yeah, let's schedule Model understands: intent = affirmative / availability_response next_action = continue scheduling
Candidate says: What is the pay? How much does this role pay? Can you tell me the compensation? What is the hourly rate? Model understands: intent = ask_question topic = compensation next_action = answer briefly, then return to scheduling
Candidate says: Tuesday morning works for me Any weekday afternoon is fine I am free tomorrow after lunch Model/extractor understands: day_preference = Tuesday / weekday / tomorrow time_preference = morning / afternoon next_action = offer matching slots
Why The System Is Hybrid
The chatbot is intentionally designed as a hybrid system: trained model + rules + state machine + safety checks + templates.
The trained model handles flexible candidate language. Rules handle exact items like email, PIN, time, and slot selection. The state machine keeps the interview scheduling flow on track. Safety checks protect the conversation. Templates keep the final replies professional and controlled.
The best summary is: the trained model learns candidate intent patterns from conversation examples, then the workflow engine uses that prediction to choose the safest next scheduling action.
5. Trained Models & Safety Defense
The engine ships five small scikit-learn models plus a deterministic rules layer. None of them free-write text — they turn messy language into structured signals the workflow can trust.
The five trained models
| Model | Artifact | What it predicts |
|---|---|---|
| Intent | intent_model.joblib | What the candidate means (affirmative, ask_question, select_slot, confirm, decline, …) |
| Next action | next_action_model.joblib | The workflow's next step (collect email, offer slots, book slot, mark confirmed, hand off) |
| Time preference | time_preference_model.joblib | Morning / afternoon / evening preference from free text |
| Day preference | day_preference_model.joblib | Day-of-week / relative day (Tuesday, weekday, tomorrow) |
| Toxicity | toxicity_model.joblib | Jigsaw-style scores (threat, severe_toxic, toxic, insult, obscene) feeding the safety gate |
Model-first, rules as rescue
A trained model is trusted only when it is confident. Intent predictions are used when confidence clears MODEL_INTENT_FLOOR (0.50); a strong local match is taken at LOCAL_CONFIDENCE_THRESHOLD (0.80); the time/day extractors apply at a 0.65 confidence floor. Below those thresholds the deterministic regex/keyword rules take over. This is the core design rule: extend the model first, fall back to rules — never grow long rule ladders to patch model gaps.
Two-layer safety gate
Every candidate message passes a safety check before scheduling logic runs:
Layer 1 — behavior rules. rules/behavior_rules.txt holds categorized regex patterns. A match is fast, deterministic, and explainable (it returns a category and reason_code tagged source: behavior_rules).
Layer 2 — toxicity model. If no rule matches, the toxicity model scores the message. Tunable floors decide the response: threat ≥ 0.12 → threatening_language (high severity, hand off to a human); severe_toxic ≥ 0.30 → abusive_language; and max(toxic, insult, obscene) over its floor → an abusive-language boundary. A hit produces a firm boundary / escalation / handoff reply and flags the conversation for recruiter review instead of continuing to schedule.
6. Templates & How The Models Are Trained
Two things keep replies professional and the models current: a layered template system and a repeatable training pipeline.
Layered template system
Conversational replies are never raw model output — they come from templates resolved in a fixed precedence: office override → DB default pool → built-in fallback. Office-specific edits live in recruiter_message_templates; the shared defaults live in ai_default_template_pool (keyed by template_key); the template_builder package owns which conversation state maps to which template and renders placeholders from a restricted allowed-variable set. Recruiters manage the defaults in the AI Templates settings page.
Recruiter-facing emails are a separate, file-based set under ai/template/*.txt (PIN, booking confirmation, next-stage invitation, reschedule, decline, questionnaire, …). PHP loads the file, fills {placeholder} values, and sends — distinct from the conversational templates above.
How the models are trained
The intent, next-action, and preference models are trained from a single labelled corpus, ai/training_data/dialog.txt, via run_all.py. Role detection is name-independent, so the corpus is not tied to specific recruiter or candidate names. Each run also mines "discovered" reply templates from the dialog so the system's phrasing reflects real conversations. The toxicity model is trained separately (Jigsaw-style data) by its own script and dropped in as toxicity_model.joblib.
Source files reviewed: CandidateChatController.php, AiEngineClient.php, conversation_respond.py, the AI conversation modules under ai/api/routes/conversation_modules, the template_builder package, rules/behavior_rules.txt, ai_default_template_pool, the trained *.joblib models, and the run_all.py training pipeline.