0be354a920
Single-commit clean baseline after security scrub of niche-tells, project codenames, internal jargon, and contributor-email leaks. Contents: - 100 Rust crates (_primitives/_rust/) - 37 agent manifests (_manifests/) + generated specs (_generated/) - 67 user-invocable skills (skills/) - 33 hooks (hooks/) - Composition blocks (_blocks/) - Documentation (docs/, README.md) - TS adapter packages (_ts_packages/) - Assembler (_assembler/) - Roles (_roles/) - Templates (_templates/) - Forgejo CI (.forgejo/) Author: Denis Parfionovich <info@greendragon.info> License: see LICENSE.
121 lines
4.4 KiB
Markdown
121 lines
4.4 KiB
Markdown
# Phase 3 — Scrape + ship wiring
|
|
|
|
Produce two concrete config artefacts in the target repo:
|
|
- `config/prometheus.yml` (or `config/otel-collector.yaml` if `STACK == otel-vendor`)
|
|
- `config/log-ship.env` — env-var bundle for `_primitives/log-ship.sh`
|
|
|
|
## 3a — Emit AskUserQuestion (one call)
|
|
|
|
```json
|
|
{
|
|
"questions": [
|
|
{
|
|
"question": "Scrape / collect topology?",
|
|
"header": "Topology",
|
|
"multiSelect": false,
|
|
"options": [
|
|
{"label": "Prometheus pulls /metrics", "description": "Prom-native. App exposes 9090. Standard for prom-grafana."},
|
|
{"label": "OTel Collector sidecar", "description": "Per-host collector. App → collector → backend. Uniform for logs+metrics+traces."},
|
|
{"label": "OTel Collector central gateway", "description": "One collector pool for the cluster. HA, scales, single ingress point."},
|
|
{"label": "Vendor agent (Datadog / BS)", "description": "Vendor-supplied agent does discovery + shipping. Lowest ops."}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Store as `TOPOLOGY`.
|
|
|
|
## 3b — Generate scrape config
|
|
|
|
**If `TOPOLOGY == "Prometheus pulls /metrics"`** — write `config/prometheus.yml`:
|
|
|
|
```yaml
|
|
global:
|
|
scrape_interval: 15s
|
|
evaluation_interval: 15s
|
|
scrape_configs:
|
|
- job_name: "$SERVICE"
|
|
metrics_path: /metrics
|
|
static_configs:
|
|
- targets: ["${SERVICE_HOST:-localhost}:${METRICS_PORT:-9090}"]
|
|
- job_name: "node"
|
|
static_configs:
|
|
- targets: ["${NODE_HOST:-localhost}:9100"]
|
|
```
|
|
|
|
Reference: `_blocks/obs-metrics.md` for label cardinality budget, naming
|
|
conventions. Reference Prometheus config spec [VERIFIED: prometheus.io/docs/prometheus/latest/configuration/configuration/].
|
|
|
|
**If `TOPOLOGY` is an OTel variant** — write `config/otel-collector.yaml`:
|
|
|
|
```yaml
|
|
receivers:
|
|
otlp:
|
|
protocols:
|
|
grpc: { endpoint: 0.0.0.0:4317 }
|
|
http: { endpoint: 0.0.0.0:4318 }
|
|
processors:
|
|
batch: {}
|
|
memory_limiter: { check_interval: 1s, limit_mib: 512 }
|
|
exporters:
|
|
prometheusremotewrite:
|
|
endpoint: ${PROM_REMOTE_WRITE_URL}
|
|
otlphttp/traces:
|
|
endpoint: ${TRACES_BACKEND_URL}
|
|
service:
|
|
pipelines:
|
|
metrics: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [prometheusremotewrite] }
|
|
traces: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [otlphttp/traces] }
|
|
logs: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [otlphttp/traces] }
|
|
```
|
|
|
|
Reference OTel Collector spec [VERIFIED: opentelemetry.io/docs/collector/configuration/].
|
|
|
|
**If `TOPOLOGY == "Vendor agent"`** — output the vendor install snippet
|
|
(Datadog Agent, Better Stack Vector config, etc.) and skip to 3c.
|
|
|
|
## 3c — Generate log-ship invocation
|
|
|
|
Build `config/log-ship.env` referencing `_primitives/log-ship.sh` with fields
|
|
from Phase 1's `LOG_TARGET`:
|
|
|
|
```sh
|
|
# config/log-ship.env — env bundle for _primitives/log-ship.sh
|
|
# Source before piping app stdout:
|
|
# set -a && . config/log-ship.env && set +a
|
|
# ./app 2>&1 | ~/.claude/agents/_primitives/log-ship.sh --target $LOG_SHIP_TARGET --endpoint "$LOG_SHIP_ENDPOINT" --label "job=$SERVICE"
|
|
|
|
LOG_SHIP_TARGET="${LOG_SHIP_TARGET:-stdout}" # stdout | loki | datadog | http
|
|
LOG_SHIP_ENDPOINT="${LOG_SHIP_ENDPOINT:-}" # e.g. http://loki:3100/loki/api/v1/push
|
|
# LOG_SHIP_DD_API_KEY=... # ← put in ~/.claude/secrets/.env or service .env — NEVER in git
|
|
# LOG_SHIP_BEARER=... # generic HTTP target bearer — same rule
|
|
```
|
|
|
|
Map Phase 1's `LOG_TARGET` → `LOG_SHIP_TARGET`:
|
|
- `stdout-only` → `stdout` (no endpoint)
|
|
- `file` → `stdout` (container runtime captures; skip shipping)
|
|
- `ship-loki` → `loki` + endpoint
|
|
- `ship-datadog` → `datadog` + endpoint + `LOG_SHIP_DD_API_KEY` via env
|
|
- `ship-http` → `http` + endpoint + optional `LOG_SHIP_BEARER`
|
|
|
|
## 3d — Verify scrape end-to-end
|
|
|
|
Before finishing the phase, invoke `_primitives/metrics-scrape.sh` against
|
|
the freshly instrumented app:
|
|
|
|
```sh
|
|
~/.claude/agents/_primitives/metrics-scrape.sh \
|
|
"http://${SERVICE_HOST:-localhost}:${METRICS_PORT:-9090}/metrics" --format table
|
|
```
|
|
|
|
If the output is empty or the curl fails — HALT, report to user (likely Phase 2
|
|
init-call mis-wired). Do NOT proceed to Phase 4 with a silent scraper.
|
|
|
|
## Verify-criterion
|
|
|
|
- `config/prometheus.yml` OR `config/otel-collector.yaml` written.
|
|
- `config/log-ship.env` written (with `# NEVER in git` comment next to any
|
|
secret-var placeholder — RULE 0.8).
|
|
- `metrics-scrape.sh` dry-run returns > 0 lines.
|
|
- `TOPOLOGY` stored for Phase 5's alert-rule scope.
|