Integrations

Kinds

Research folder. The poller opens the directory with a root-confined file API, so symlinks and traversal cannot escape it. Files are type-sniffed rather than trusted by extension; archives, executables, disk images and anything binary are rejected and recorded as rejected; text-like documents are extracted in-process with size caps, fingerprinted by content, and ingested once. HTML is reduced to text. Each document becomes an observation with an excerpt and a reference to the file and fingerprint, and the document ledger shows ingested, duplicate, rejected and failed items.

Git. Repositories are mirrored with hooks disabled and the external transport protocol refused; git runs with argument vectors only, never a shell. New commits become observations (author, date, subject, files) and co-change statistics are kept as integration state. A mind with a git integration gains the repository.read capability, which reads one file at one ref through the same guarded process and refuses paths that would leave the tree.

Prometheus. Queries are named expressions with optional thresholds. The poller evaluates them, records the last value, and emits observations when a threshold is crossed or the value changes by a large relative amount, with salience proportional to the size of the change. A mind gains read_metrics for ad-hoc queries against the same endpoint.

Webhooks. A delivery must carry a timestamp header and an HMAC-SHA256 signature over timestamp.body with the integration's secret. Deliveries are refused when unsigned, older than five minutes, replayed (signature already seen), larger than 256 KiB, or over the per-address rate limit. A webhook without a configured secret refuses everything. Accepted deliveries become observations with the sender's title, text, reference and salience.

Shared machinery

Integrations have an interval (minimum thirty seconds), a health status (new, ok, error, stale, disabled), the last run and error, counts of items and observations, and a run history. The scheduler ticks every minute and runs those that are due, in isolation, with a timeout, recording each run. Every observation carries the integration's id, name and kind, the trust level and a reference back to the source item, and the provenance is visible on the event page.

Secrets

Configuration never holds secrets: keys named like token, secret, password or api_key are rejected at validation. Secrets are stored as references (env:, file: or enc: under the master key) and are never rendered back into forms or API responses.

Integrations bring external data into a mind as observations. Every item is stored with its source, trust level and reference, and is treated as data: an instruction found inside an ingested file, commit message, metric label or webhook body is never followed. Integrations never execute anything.

Provenance

Each observation carries metadata.integration, metadata.integration_id, metadata.kind, metadata.trust and a reference (file path + fingerprint, commit hash, metric expression or webhook reference). Documents are listed per integration with status ingested, duplicate, rejected or failed.

Webhooks

X-Noema-Timestamp: <unix seconds>
X-Noema-Signature: sha256=HMAC_SHA256(secret, timestamp + "." + raw_body)
{"title":"...","text":"...","reference":"...","salience":0.6}

Deliveries are refused when the timestamp is more than 300 s from now, the signature does not match, the signature has been seen before (replay), the body exceeds 256 KiB, or the per-IP rate limit is exceeded. A webhook without a configured secret refuses everything.

Secrets and configuration

Configuration never holds secrets: keys named like token, secret, password or api_key are rejected. Secrets are stored as references (env:NAME, file:/path, or enc: under NOEMA_MASTER_KEY) and are never rendered back.

Capabilities exposed to minds

• read_metrics (default ALLOW, low risk): target = Prometheus integration name, param expr.
• repository.read (default ASK, medium risk): target = git integration name, params path, ref.

Both resolve only integrations connected to the invoking mind, so policy and the operator decide which endpoints a mind can reach.

Operating

• UI: Integrations lists health, last run, error and counts; the detail page shows configuration, recent runs and ingested documents.
• CLI: noemactl integrations list|add|run|runs|delete.
• The scheduler ticks every minute and runs integrations whose interval has elapsed. Failures mark the integration error and are shown in the UI; sources that fail repeatedly become stale.