AgentsIQ
Incident Response ▾
Agentic Pipeline Concept Agent Pipeline Overview Pipeline Monitor (Mockup) Functional Specification Future Work Tracker
AgentsIQ — Internal Platforms Engineering
Agentic Incident Response Pipeline
Three-agent autonomous system for alert detection, root cause analysis, and remediation — with shared RAG knowledge base and internal ticket tracking.
Draft Internal Version 0.1 — June 2026 For review and feedback
What this solves

When an alert fires, multiple engineers are pulled in to triage, investigate, and fix — often spending hours on incidents that follow known patterns. This pipeline automates that entire chain, reserving human attention for genuinely novel problems.

MTTD
Faster
Alert Analyser reduces noise, surfaces real incidents faster. No human triage needed to separate genuine alerts from noise.
MTTA
Instant
RCA Agent starts investigation immediately on escalation. No human acknowledgement step for known patterns.
MTTR
Automated
Action Agent applies known fixes automatically via Jenkins. Tracked escalation to Jira when automation is not available.
Architecture overview
Agent 1
Alert Analyser
Reads OpsGenie / JSM · deduplicates · classifies genuine / noise / suspect · enriches with inventory context · RAG lookup · creates incident ticket · escalates genuine alerts
→
Shared DB
Incident ticket
Status: ESCALATED · Alert Analyser writes · RCA Agent polls · Decoupled — no direct agent-to-agent calls
→
Agent 2
RCA Agent
Polls for ESCALATED tickets · checks RAG · investigates logs / metrics / deps · produces structured findings · recommends actions · feeds RAG
↓
Shared RAG
Knowledge base
Alert patterns · RCA findings · recommended actions · fix outcomes · grows smarter with every incident · all three agents read and write
←
Shared DB
Incident ticket
Status: RCA_COMPLETE · RCA Agent writes findings · Action Agent polls · Full audit trail of every state change
←
Agent 3
Action Agent
Polls for RCA_COMPLETE tickets · checks Jenkins job catalogue · triggers fix or creates Jira ticket · records outcome · feeds results back to RAG
Dashboard
Standalone — reads incident_management DB
Independently deployable on any platform (UAP or other). Read-only over shared Postgres. Shows incident pipeline, MTTD/MTTA/MTTR KPIs, per-ticket audit timeline.
Agents communicate exclusively via the shared database — no direct HTTP calls between agents. Each agent can be deployed, restarted, or replaced independently. Tickets queue safely in DB when any agent is unavailable.
Agent pipeline — step by step
Agent 1 — Alert Analyser
Detect, classify and escalate
Existing agent — extend
On each sync cycle, Alert Analyser reads new alerts from OpsGenie / JSM. It deduplicates, scores noise, and enriches genuine alerts with inventory context (account, environment, customer). It then checks the RAG store for known patterns matching this alert's signature.
  • Deduplicate — same incident fired by multiple tools counts once
  • Classify each alert as genuine, noise, or suspect using rule-based scoring
  • Enrich genuine alerts with inventory context (customer, application, environment)
  • Look up RAG for known patterns matching this alert signature
  • Create an incident ticket with enriched context and RAG hit (if any)
  • Set ticket status to ESCALATED — RCA Agent picks it up via polling
Ticket created → status: ESCALATED
Agent 2 — RCA Agent
Investigate and find root cause
New agent — to build
RCA Agent polls the incident_management DB for tickets in ESCALATED status. For each ticket it picks up, it updates the status to INVESTIGATING, then begins autonomous analysis using the enriched alert context, RAG findings, and available data sources (logs, metrics, deployment history, service dependencies).
  • Poll DB for ESCALATED tickets — update to INVESTIGATING on pickup
  • Query RAG for previous occurrences of the same alert signature
  • Investigate logs, metrics, and service dependencies autonomously
  • Correlate with recent deployments, config changes, or dependency failures
  • Produce structured finding: root cause, evidence, confidence score
  • Recommend specific remediation actions (Jenkins job, config change, rollback)
  • Write findings back to RAG — future incidents with same pattern benefit immediately
  • Update ticket with findings and set status to RCA_COMPLETE
Ticket updated → status: RCA_COMPLETE + findings attached
Agent 3 — Action Agent
Remediate and record outcome
New agent — to build
Action Agent polls for tickets in RCA_COMPLETE status. It reads the RCA findings and recommended actions, then checks whether a Jenkins job exists for the recommended fix. If yes, it triggers the job and monitors execution. If no, it creates a Jira ticket with full RCA context for the on-call engineer.
  • Poll DB for RCA_COMPLETE tickets — update to REMEDIATING on pickup
  • Check Jenkins job catalogue for a job mapped to the recommended action
  • If job exists — trigger, monitor execution, verify resolution in OpsGenie
  • If no job — create Jira ticket with full RCA context for on-call engineer
  • Record outcome (success / failure / manual escalation) back to ticket
  • Feed outcome to RAG — successful fixes become known-good actions for future
  • Feed results back to Alert Analyser for enriched escalation context in future
  • Set ticket to RESOLVED (automated) or MANUAL (human intervention needed)
Ticket updated → status: RESOLVED or MANUAL
Shared RAG — Continuous improvement
Knowledge base grows with every incident
New shared service — to build
All three agents read from and write to a shared RAG store. Over time, the system learns which alert signatures map to which root causes, which Jenkins jobs resolve which issues, and which patterns repeat. Each incident makes the next one faster to resolve.
  • Alert patterns and signatures — dedup and noise detection improves over time
  • Root cause findings — matched by alert signature for instant RCA on repeat incidents
  • Recommended actions and their outcomes — Action Agent knows which fixes work
  • Jenkins job mappings — auto-populated as successful remediations are recorded
  • Jira tickets created — tracked alongside automated resolutions for audit
Ticket status lifecycle
NEW
Alert fired
→
ESCALATED
Alert Analyser
→
INVESTIGATING
RCA Agent picks up
→
RCA_COMPLETE
Findings attached
→
REMEDIATING
Action Agent
→
RESOLVED
MANUAL
Automated / Jira
MTTD = ESCALATED − detected_at  ·  MTTA = INVESTIGATING − detected_at  ·  MTTR = RESOLVED − detected_at
Data schema — incident_management DB
Separate logical database within the shared Postgres instance. Agents write directly; dashboard reads via a lightweight read-only API.
incidents
idUUID PK
alert_idVARCHAR — OpsGenie/JSM ID
alert_signatureVARCHAR — for RAG dedup
statusENUM — see lifecycle
priorityVARCHAR — P1/P2/P3/P4
titleTEXT
alert_payloadJSONB — enriched alert
rca_findingsJSONB — root cause + actions
action_outcomeJSONB — fix result + logs
detected_atTIMESTAMPTZ — MTTD anchor
escalated_atTIMESTAMPTZ — MTTA anchor
rca_completed_atTIMESTAMPTZ
resolved_atTIMESTAMPTZ — MTTR anchor
created_atTIMESTAMPTZ
updated_atTIMESTAMPTZ
incident_events (audit log)
idUUID PK
incident_idUUID FK → incidents
agentVARCHAR — which agent acted
event_typeENUM — see below
from_statusVARCHAR — previous state
to_statusVARCHAR — new state
payloadJSONB — event detail
created_atTIMESTAMPTZ
event_type values
STATE_CHANGE NOTE RAG_HIT ACTION_TRIGGERED JIRA_CREATED RAG_WRITE
Deployment model
Alert Analyser
Existing agent — extended
Add ticket writer. Add RAG client. Existing dedup, classify, enrich logic unchanged.
RCA Agent
New standalone agent
Polls incidents DB. Reads logs/metrics. Writes findings. Feeds RAG. Standalone FastAPI.
Action Agent
New standalone agent
Polls incidents DB. Jenkins integration. Jira escalation. Feeds outcome to RAG.
Shared Postgres
incident_management DB — existing instance
Separate logical DB in agentsiq_db Postgres. Schema owned by Alert Analyser (created on startup). All agents read/write.
Incident Dashboard
Standalone — independently deployable
Lightweight FastAPI (read-only) + nginx + HTML/JS. Deployable on UAP or any platform that can reach Postgres. Shows live pipeline, MTTD/MTTA/MTTR, audit trail.
Open questions for review
Alert Analyser → RCA Agent
What is the exact data contract? Which fields does RCA need from Alert Analyser — just the enriched alert payload, or also the RAG hit context and confidence score?
RCA Agent
Which data sources can RCA Agent query autonomously? Logs, metrics, traces, deployment history — which are available via API and which need integration work?
Action Agent
How is the Jenkins job catalogue maintained? Who maps alert types to jobs, and how do we prevent bad remediations on production without human approval?
Shared RAG
How is the RAG structured — per-service, per-alert-type, or a single global store? How do we prevent bad findings from being stored and reused in future incidents?
Governance
Which alert types are allowed to trigger automated remediation without human approval? P1 incidents — do they always go to MANUAL first, or can automation proceed?
Observability
How do we track agent decisions end-to-end for audit and debugging? Should every RAG read/write be logged in incident_events, or do we need a separate agent decision log?
AgentsIQ — Internal Platforms Engineering
Generated June 2026 · Draft v0.1 · For review