Host Intelligence — 870,000+ Hosts, 5-Pillar Threat Scoring, Catalog-Driven Roles

“Is 185.54.81.40 hostile?” is the wrong question. The right questions are: which role does this host play, what’s its historical behaviour, how does it compare to its peer-class baseline, what do external feeds say, and what’s the **provable false-positive rate** on a blackhole decision against it? RoutePulse auto-discovers **870,000+ hosts** from flow telemetry and answers all five questions per row — a 5-pillar score (Cyber 48pt + Behavioral 40pt + ML 30pt + External 40pt + FeedIntel 25pt = 183pt raw, Bayesian-sigmoid-normalised to 0–100) feeding the **8-Gate Blackhole Pipeline** and **Conviction Engine v2** (SPRT + Thompson Sampling + Conformal Prediction with FDR-controlled false-positive bound).

Roles come from the **Role Catalog** (v4.28+) — 39 service-role bindings, 16 PROFILE_* constants per security zone (PROFILE_OUTSIDE_WAN, PROFILE_INSIDE_LAN, PROFILE_HA_HEARTBEAT, PROFILE_DMZ_WEB, etc.), and 16 SNMP heuristic regex patterns that parse operator-curated `ifAlias` strings (“Cogent Zurich”, “outside-Telecom”, “vMotion”, “HA-Sync”) with one-click apply in the Pending Review tab. Trust hierarchy: manual catalog > SNMP ifAlias > PTR pattern > Shodan / nmap > traffic-composition heuristic. The legacy `ml_role` pipe-string is gone; `role_tags` (text[] + GIN index) is canonical. Behind the scenes, **Ship 3a/3b** (v4.32.86–91) split the per-host detail JSONB into a dedicated `host_threat_details` side-table — the `hosts` table reclaimed 12 GB → 745 MB (-94%) via pg_repack while readers cut over to a clean LEFT JOIN keeping the hot path on the small parent table.

5-pillar scoring — 183pt raw → Bayesian sigmoid 0–100

**Cyber Events (48pt)** — Wazuh + Suricata + FortiGate hits weighted by attack severity. **Behavioral (40pt)** — 14 components: volume, connections, ports, flow patterns, directionality, beaconing, temporal patterns, port scanning, persistence, infrastructure targeting, burst intensity, reconnaissance profile, subnet correlation, extended port scan. Computed via 9 parallel ClickHouse queries with adaptive P90/P95/P99 thresholds from live traffic. **ML Ensemble (30pt)** — 18-model output. **External Intel (40pt)** — AbuseIPDB confidence + Shodan ports/CVEs + InternetDB + Nmap scan results. **FeedIntel (25pt)** — 39 enabled threat feeds (Spamhaus DROP, Emerging Threats, FireHOL, custom GOLINE feeds).

Parallel enrichment — smart gates save 99% of API quota

**AbuseIPDB** (429-aware with CAPPED indicator + API-reset countdown) + **Shodan** (InternetDB pre-filter) + **Nmap** (parallel scanning 1–10 concurrent, default 3, periodic backfill every 30min) + **InternetDB** (free preliminary lookup). Smart score gates avoid wasting quota on clean hosts: `safeAbuseMinScore=15`, `safeShodanMinScore=15`, `safeNmapMinScore=25`. Result: 99% reduction in upstream API calls vs. blind scanning. Real-time UI bars show rate/min, queue size, blinking activity indicators.

Lifecycle state machine + hysteresis deadzone

Each host progresses **safe → observation → active → malicious** with a hysteresis deadzone (promote at score ≥35, demote at ≤25) that eliminates status flapping. Status transitions feed the **Conviction Engine v2** which applies SPRT (Sequential Probability Ratio Test) for evidence accumulation, Thompson Sampling for per-action outcome learning, and Conformal Prediction for the rigorous false-positive bound on any blackhole decision.

Ship 3a/3b storage optimisation (v4.32.86–91)

JSONB detail moved to `host_threat_details` side-table; 12 GB → 745 MB on the `hosts` parent (-94%) via pg_repack post DROP COLUMN. UPSERT-with-JSONB-merge writer pattern, FK DEFERRABLE INITIALLY DEFERRED to dodge concurrent-delete races, AFTER trigger maintains `hosts.cyber_scored` denormalised flag. Hot-path readers now LEFT JOIN; `shared_blks_read` drops ~700–850 MB/call on cohort selectors.

Attack Story Timeline — the kill sequence in one glance

The host detail page opens with a multi-lane SVG timeline that tells the whole story at once: behaviour-diary entries, security events, ML detections, threat-feed matches, AI verdicts and mitigation spans, all plotted over the host’s real traffic sparkline. One glance answers “when did it start, what fired, what did we do, did it stop” — the same component powers the incident war-room view with the incident window highlighted.

Key Capabilities

• 870K+ auto-discovered hosts from flow data with continuous AbuseIPDB, Shodan, InternetDB, Nmap enrichment
• 5-pillar 0–100 unified threat score (Cyber 48 + Behavioral 40 + ML 30 + External 40 + FeedIntel 25 = 183pt raw, Bayesian sigmoid normalised)
• 14 behavioral components computed via 9 parallel ClickHouse queries with adaptive P90/P95/P99 thresholds
• Role Catalog (v4.28+) — 39 service roles + 16 PROFILE_* per-zone constants + 16 SNMP ifAlias regex patterns, GIN-indexed `role_tags` text[]
• SNMP ifAlias one-click role suggestion in Pending Review tab (top-5 confidence-ranked candidates)
• 8-Gate Blackhole Pipeline + Conviction Engine v2 (SPRT α=0.01 β=0.05 + Thompson Sampling + Conformal Prediction with FDR bound)
• Smart enrichment gates: safeAbuseMinScore 15, safeShodanMinScore 15, safeNmapMinScore 25 — 99% API quota savings
• AbuseIPDB integration with 429 detection, CAPPED indicator + reset countdown, auto-reporting (900/day cap)
• Lifecycle state machine safe→observation→active→malicious with hysteresis deadzone (promote 35, demote 25)
• host_threat_details side-table (v4.32.86–91 Ship 3a/3b) — 12 GB → 745 MB on `hosts` parent (-94% via pg_repack)
• Reputation refresh cadence per pillar; Ed25519-signed audit entry for every score-based action
• Infrastructure auto-discovery from 7 pipelines: FortiGate SSH, SNMP, BMP, BGP-4, Flow, Role Loading, External enrichment

Engineered and operated by the GOLINE SOC & Network Engineering team.

Explore all RoutePulse features →