# Native Observability

**Native Observability brings structured, request-level observability directly inside Drupal.**

No external agents.  
No heavy APM stack.  
No vendor lock-in.

This project introduces a native observability layer built specifically for **Drupal 11**, enabling request tracing,
execution monitoring, metrics aggregation, reporting, and telemetry export directly from the application runtime.

Instead of relying on external APM systems, **Native Observability integrates observability capabilities directly into
Drupal's architecture**.

---

# 🎯 Vision

Modern Drupal applications require deep runtime introspection in order to:

- understand request lifecycle behaviour
- measure performance bottlenecks
- inspect internal service execution
- trace external API calls
- correlate AJAX and sub-requests
- analyze performance trends
- generate structured technical reports
- export telemetry to external monitoring platforms

Native Observability provides all of this **natively inside Drupal**, without requiring external agents or invasive
instrumentation.

---

# 🚀 Global Features

Native Observability provides a complete observability stack composed of:

## Request Tracing
- ULID based Request IDs
- Kernel request lifecycle tracing
- Parent / child request correlation
- AJAX request correlation
- Correlation ID injected in response headers

## Execution Instrumentation
- Execution Span Recorder API
- Service execution tracking
- HTTP client tracing
- External API instrumentation
- Span categorization

## Trace Visualization
- Trace list UI
- Trace detail inspection
- Execution timeline visualization

## Performance Metrics
- Route aggregation
- Throughput analysis
- Percentile calculation (P95 / P99)

## Observability Reporting
- Report builder UI
- Printable HTML reports
- KPI summaries
- Technical appendices

## Telemetry Export
- OpenTelemetry exporter
- Prometheus endpoint

---

# 🧱 Architecture Overview

Core contracts:

- TraceRecorderInterface  
- TraceStorageInterface  
- RequestIdGeneratorInterface  
- ExecutionSpanRecorder  

All services follow Drupal 11 best practices.

---

# 🧩 Module Architecture

## Core Module

### native_observability
Core tracing infrastructure.

---

## Extension Modules

### native_observability_execution
Execution tracing.

### native_observability_spans
Span management.

### native_observability_metrics
Performance metrics.

### native_observability_otel
Telemetry export.

### native_observability_status_block
Admin diagnostics.

### native_observability_dashboard
Provides the main observability dashboard UI.

Features:
- Aggregated metrics visualization
- Route performance tables
- AJAX and request analysis
- Cache and database insights
- Technical insight panels

This module acts as the **central visualization layer** of the system.

### native_observability_report
Structured reporting and printable HTML export.

---

# 📦 Installation

## Requirements

- Drupal 11
- PHP 8.2+
- Database (MySQL / MariaDB / PostgreSQL)

## Install via Composer

composer require drupal/native_observability

## Enable modules

drush en native_observability

Recommended:

drush en native_observability_execution
drush en native_observability_spans
drush en native_observability_metrics
drush en native_observability_dashboard
drush en native_observability_report

Optional:

drush en native_observability_otel
drush en native_observability_status_block

## Cache rebuild

drush cr

---

# 🖨️ Report Export

Reports are generated as printable HTML.

Workflow:
1. Generate preview
2. Open printable HTML
3. Use browser print → Save as PDF

---

# 📊 Metrics & Observability (Prometheus + Grafana)

The `native_observability_otel` module exposes a Prometheus-compatible metrics endpoint at `/native-observability/prometheus`.

## What is scraped

The endpoint reads **live data** from source tables at scrape time (no cron dependency) with a configurable rolling window (default: 5 minutes). Metrics follow the `drupal_*` naming convention standard in the Drupal community.

Key metrics exposed:

- `drupal_http_requests_total` / `drupal_http_requests_window` — HTTP traffic by route, method, status code
- `drupal_http_request_duration_ms{quantile}` — P50/P95/P99 latency per route
- `drupal_http_error_rate` — percentage of 5xx responses per route
- `drupal_cache_operations_total` — cache events by operation (HIT/MISS/…) and bin
- `drupal_span_duration_avg_ms` / `drupal_span_count` — execution span metrics by category

## Local stack (DDEV)

```bash
ddev add-on get MurzNN/ddev-grafana
ddev restart
```

This starts Grafana + Mimir. Three dashboards are pre-provisioned automatically:

- **Drupal — HTTP Overview** — request rate, error rate, P95 latency, top slow routes, AJAX split
- **Drupal — Cache** — operations rate per bin, HIT/MISS breakdown
- **Drupal — Spans Breakdown** — avg duration per category, donut distribution

See full documentation in [`modules/native_observability_otel/README.md`](modules/native_observability_otel/README.md).

## Bearer token (production)

Configure the bearer token in **Admin → Configuration → Development → Native Observability OpenTelemetry** (`/admin/config/development/native-observability/otel`). When set, the endpoint requires `Authorization: Bearer <token>`. The settings page shows a ready-to-copy `prometheus.yml` scrape config snippet.

---

# 🧠 Philosophy

Observability should be part of the application architecture itself.
