Integrations¶

The integrations layer provides a unified infrastructure for connecting SynthOrg to external services. It sits underneath every external consumer (MCP servers, providers, notification sinks, tools) and provides:

• Connection Catalog: typed registry for all external service credentials
• Secret Backends: pluggable encrypted credential storage
• OAuth 2.1: authorization code + PKCE, device flow, client credentials
• Webhook Receiver: signature verification, replay protection, event bus bridge
• Health Checks: per-type connection health monitoring with background prober
• Rate Limiting: tool-side rate limiter via @with_connection_rate_limit
• MCP Catalog: bundled curated MCP server catalog with install flow
• Tunnel: multi-provider webhook tunnel (Cloudflare quick tunnel default, ngrok, Dev Tunnels)

Connection Catalog¶

Central registry for external service connections. Each connection has a unique name, a typed connection type, encrypted credentials (via SecretRef), optional rate limiting and health check configuration, and a sensitive flag. When sensitive is set, the governed external-access tool routes every call against the connection (read or write) to human approval, not only write methods.

Connection Types¶

Secret Storage¶

Credentials are encrypted at rest via a pluggable SecretBackend:

Both encrypted_* backends share the same Fernet key material (AES-128-CBC + HMAC-SHA256, 32 bytes of key, URL-safe base64). The key is read from the environment variable named by master_key_env on each backend's config (default SYNTHORG_MASTER_KEY). Per-secret rotation (via SecretBackend.rotate) writes a new Fernet token under a fresh secret_id without touching other rows; losing the key loses only the stored secrets, not the rest of the org data. Master-key rotation is not currently supported: changing SYNTHORG_MASTER_KEY makes every previously stored ciphertext undecryptable, so the master key is treated as permanent for the life of the install (re-init preserves it for the same reason).

create_app auto-promotes the default encrypted_sqlite config to encrypted_postgres when the active persistence backend is Postgres, so operators do not have to keep the secret backend and persistence backend in manual sync. This automatic selection is the normal path; the only cases that require explicit config are operators who want env_var (no at-rest storage) or a custom master_key_env variable name. When SYNTHORG_MASTER_KEY is unset, both encrypted backends log an error and downgrade to env_var so the integrations subsystem still boots in a degraded-but-functional state; set the key and restart to re-enable at-rest encryption. The selection logic lives in resolve_secret_backend_config (persistence/secret_backends/factory.py) and is covered by unit tests for each branch.

synthorg init generates a fresh Fernet master key, writes it to config.json (master_key), and wires it into the backend container as SYNTHORG_MASTER_KEY whenever the Encrypt secrets at rest advanced toggle is ON (the default). Re-init preserves the existing key so already-stored ciphertext stays decryptable. The toggle can also be flipped via --encrypt-secrets=true|false in non-interactive mode.

At-rest protection of the rest of the database (non-secret rows, full-text backups, snapshots) is the operator's responsibility: use disk/filesystem encryption (LUKS, BitLocker, FileVault, cloud-provider encrypted volumes, RDS-style encryption at rest). Column-level encryption in the app is deliberately narrow: its goal is to prevent a SQL-level reader from lifting OAuth tokens and API keys, not to substitute for OS/volume encryption.

API Endpoints¶

OAuth 2.1¶

Full OAuth 2.1 implementation with three grant types:

Authorization Code + PKCE (RFC 7636)¶

Primary web flow. User clicks "Connect" in dashboard, browser redirects to provider, callback handler exchanges code for tokens.

Device Flow (RFC 8628)¶

For CLI/headless use. Displays user code and verification URL, polls for authorization.

Client Credentials¶

Machine-to-machine flow. No user interaction.

Token Lifecycle¶

OAuthTokenManager background service refreshes tokens before expiry (configurable threshold, default 5 minutes).

API Endpoints¶

Webhook Receiver¶

Generic webhook endpoint that verifies signatures and publishes events to the SynthOrg message bus.

Signature Verifiers¶

Replay Protection¶

In-memory nonce + timestamp dedup window (default 5 minutes).

Event Bus Bridge¶

Verified events are published to the #webhooks channel on the message bus. ExternalTriggerStrategy subscribes and fires workflows on matching events.

API Endpoints¶

Health Checks¶

Per-type health check implementations with a background HealthProberService.

• Smoothing: N consecutive failures before marking unhealthy (default 3)
• Interval: Configurable (default 5 minutes)
• Pattern: Matches the ProviderHealthProber design
• UNKNOWN is a no-op: a checker that cannot probe (e.g. an LLM_PROVIDER connection with no base_url) reports UNKNOWN; the prober neither resets nor increments the failure counter, so a healthy provider never escalates to UNHEALTHY over successive cycles.
• LLM_PROVIDER (LlmProviderHealthCheck): GETs the connection base_url; any sub-500 response is HEALTHY (the endpoint is reachable), a 5xx / network error / SSRF rejection is UNHEALTHY, and a connection with no base_url (litellm-routed cloud provider) is UNKNOWN. The probe is SSRF-validated and DNS-pinned before any request.
• TUNNEL (TunnelHealthCheck): resolves the same availability + credential verdict the dashboard's tunnel card shows, via a TunnelStatusLookup bound to the tunnel manager at startup (bind_tunnel_status_lookup). HEALTHY when the backing provider is available with its credential in place; UNHEALTHY when either is missing or the status lookup itself fails; UNKNOWN when no manager is bound yet or the connection maps to no known tunnel provider.

Rate Limiting¶

@with_connection_rate_limit decorator for tool implementations. Reuses RateLimiter from providers/resilience/rate_limiter.py.

MCP Server Catalog¶

Static JSON catalog (bundled.json) with 5 curated MCP server entries: GitHub, Slack, PostgreSQL, SQLite, Brave Search. Each entry is connection-gated (it declares a required_connection_type); no entry runs without a bound connection.

API Endpoints¶

Installed catalog entries are persisted in the mcp_installations table and merged into the effective MCPConfig.servers at bridge startup via merge_installed_servers() in synthorg.integrations.mcp_catalog.install. This keeps dashboard installs out-of-band from the user-owned YAML config and ensures they survive restarts without rewriting the config file.

Install and uninstall additionally trigger a best-effort runtime hot-reload (reload_runtime_services) so a bridged (or removed) server's tools go live for the next task without a restart; the startup merge above is the fallback when the runtime is not yet wired at install time. A reload failure never fails the request (the row is already persisted) and is logged as MCP_BRIDGE_RELOAD_FAILED.

Tunnel¶

Multi-provider tunnel for local webhook development. A TunnelManager facade holds one adapter per provider and delegates start/stop/status to whichever the live integrations.tunnel_provider setting selects (resolved fresh at every start, so a Settings change applies without a restart). Starting while a different provider's tunnel is running stops that tunnel first: at most one tunnel is ever up.

Providers:

• Cloudflare quick tunnel (default): needs no account; runs cloudflared tunnel --url and scrapes the ephemeral https://*.trycloudflare.com URL. Binary resolution: PATH, then bin/ under the shared tunnel state dir, then (unless integrations.tunnel.cloudflared_download_enabled: false) an HTTPS download of the official Cloudflare GitHub release asset.
• ngrok: wraps pyngrok; requires an auth token (ERR_NGROK_4018 refuses anonymous sessions). The token is dashboard-managed: pasted on the tunnel card, stored in the encrypted connection catalog as a tunnel-ngrok connection (ConnectionType.TUNNEL), and resolved fresh at every start. The env var named in integrations.tunnel.auth_token_env (default NGROK_AUTHTOKEN) is the headless fallback only.
• Dev Tunnels: drives the devtunnel CLI, resolved like cloudflared (PATH, then bin/ under the state dir, then, unless devtunnel_download_enabled: false, an HTTPS download from Microsoft's fixed aka.ms/TunnelsCliDownload/* asset URLs; the licence forbids redistribution, not a runtime download by the operator's own deployment). The product is named "Dev Tunnels"; GitHub is only the sign-in method. The credential is a GitHub device-code login (POST /device-login returns the verification URL + one-time code; the CLI completes and stores the login itself). Microsoft offers no credential-injection API (every token-minting command requires an already-logged-in CLI), so unlike the ngrok token the login cannot live in the encrypted catalog; on POSIX the adapter instead confines the login cache owned by the CLI, overriding HOME to a private owner-only devtunnels-home/ under the state dir. Because it stores no token, the manager seeds a read-only, no-secret tunnel-devtunnels Connection (empty credentials, health_check_enabled=False) lazily at status/first-login so it still appears in the Connections list and is health-checked through the generic tunnel status lookup alongside ngrok.

The manager is wired unconditionally (not gated by integrations.enabled) so the dashboard tunnel card is always functional; the tunneled port is the API's own resolved api.server_port. Credential storage requires connected persistence (the catalog); everything else works without it.

All tunnel runtime state roots at the tunnel state dir: SYNTHORG_TUNNEL_STATE_DIR (registry key integrations/tunnel_state_dir, read-only post-init, .. components rejected at boot), defaulting to ~/.synthorg bare-metal. The CLI-generated compose sets /data/tunnel, so downloaded binaries and the devtunnel login survive container recreation even though the backend rootfs is read-only and has no HOME.

GET /integrations/tunnel/status returns a TunnelSnapshot: the public URL, selected + active provider, and per-provider readiness (available, credential_kind, credential_configured, detail) so the dashboard renders the provider picker generically without ever transmitting a token.

API Endpoints¶

Configuration¶

integrations:
  enabled: true
  connections:
    max_connections_per_type: 100
  secret_backend:
    backend_type: "encrypted_sqlite"
  oauth:
    state_expiry_seconds: 3600
    pkce_required: true
    auto_refresh_threshold_seconds: 300
  webhooks:
    rate_limit_rpm: 100
    replay_window_seconds: 300
    max_payload_bytes: 1000000
    verify_signatures: true
  health:
    check_interval_seconds: 300
    unhealthy_threshold: 3
  tunnel:
    auth_token_env: "NGROK_AUTHTOKEN"
    cloudflared_download_enabled: true
    devtunnel_download_enabled: true
  mcp_catalog:
    enabled: true

integrations.tunnel.auth_token_env names the environment variable holding the headless-fallback ngrok token (the dashboard-managed catalog credential always wins). cloudflared_download_enabled: false / devtunnel_download_enabled: false require the respective operator-installed binary on PATH. The tunnel state dir is env-only (SYNTHORG_TUNNEL_STATE_DIR), not YAML. The active provider is the integrations.tunnel_provider setting (ENUM cloudflare / ngrok / devtunnels, default cloudflare; DB > env > default), not static YAML.

Provider Migration¶

ProviderConfig now supports a connection_name field that references a connection in the catalog. When set, credentials are resolved from the catalog at runtime instead of using embedded api_key / OAuth fields.

MCP Service Facades¶

The integrations domain exposes six service facades on AppState for MCP handler shims:

All destructive operations (_delete, _deactivate, _uninstall, _remove_provider) route through require_admin_guardrails() and emit MCP_ADMIN_OP_EXECUTED on success. Artifact delete performs storage deletion before index removal so the two cannot diverge silently.