Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,19 @@ jobs:
OUTPUT: /tmp/release-notes.md
run: python3 scripts/build_changelog_comment.py

- name: Build llms.txt artifacts
if: github.ref_type == 'tag'
env:
OUTPUT_DIR: llms-dist
run: python3 scripts/build_llms_txt.py

- name: Create GitHub release
if: github.ref_type == 'tag'
uses: softprops/action-gh-release@v2
with:
files: dist/*
files: |
dist/*
llms-dist/*
body_path: /tmp/release-notes.md

- name: Publish package to PyPI
Expand Down
46 changes: 46 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Changelog

Notable changes to wildedge-sdk. Every behavior change lands here; entries go
under Unreleased and move into a version section at release time.

## Unreleased

## 0.2.0 - 2026-07-14

### Added

- `register_model()` accepts a `model_format` override for models without a matching extractor; `llm_api()` registers its models as format `"api"`, matching the openai/anthropic integrations instead of `"unknown"`.
- `source_from_base_url()` recognizes more provider hosts (anthropic, mistral, groq, together, deepseek, xai, google, fireworks, cerebras, perplexity, nvidia, huggingface, baseten) instead of falling back to raw hostnames, including suffix matching for per-resource subdomains (Azure OpenAI, Baseten).
- `examples/llm_api_example.py`: raw-HTTP LLM tracking with `wildedge.llm_api()`; existing examples updated to the module-level API (`wildedge.span` / `wildedge.flush` / `wildedge.register_model` instead of threading a client variable)
- Releases ship `llms.txt` and `llms-full.txt` as GitHub release assets: the full documentation for that exact version in one file, generated by `scripts/build_llms_txt.py`. README quickstart rewritten around the module-level API.
- `wildedge doctor --send-test-event`: sends one real span event through the full pipeline and reports the ingest response, proving DSN auth and connectivity end to end. The report gains an `environment` section (`WILDEDGE_*` variables, autoload PYTHONPATH status) plus `config_status` / `connectivity_status` fields.
- `wildedge.llm_api()`: provider-agnostic tracking for LLM calls made with any HTTP client (OpenRouter, vLLM, Ollama, OpenAI/Anthropic-compatible endpoints). Times the block, normalizes usage payloads from either provider shape via `call.usage()` / `call.response()`, supports TTFT marks and async use, and records exceptions as error events. See `docs/llm_api.md`.
- `register_model()` without a matching extractor defaults the model name to the explicit `model_id` instead of the placeholder object's type name.
- Process-wide default client: `wildedge.get_client()`, `wildedge.set_default_client()`, and module-level `trace`, `span`, `track_span`, `register_model`, `flush` delegating to it (#41)
- `init()` reuses the default client installed by `wildedge run` unless `dsn` is passed, so CLI and in-process init share one client (#41)
- `SpanContextManager.set_attributes()` and `fail()` for recording outcomes on an open span (#41)
- `wildedge run --strict` (env `WILDEDGE_STRICT`): exit with a reserved code instead of running untracked when bootstrap fails (120 config error, 122 internal error)
- Lazy default-client creation honors `WILDEDGE_INTEGRATIONS` and `WILDEDGE_HUBS` from the environment (#41)
- `docs/deployment.md`: the deployment contract (no-DSN behavior, strict mode, fork/exec servers, one client per process)

### Changed

- `wildedge doctor` exit codes are now differentiated: 0 pass, 1 configuration or dependency failure, 2 connectivity failure (`--network-check` or `--send-test-event`). A failing network check previously exited 1.
- The test suite isolates all default SDK state paths under tmp; tests no longer write to the machine-global state directory.

- `--strict-integrations` now takes effect: a failed required integration exits the process with code 121. Previously the enforcing code path was never invoked, so the flag was silently ignored.
- `--print-startup-report` and `--no-propagate` now work under `wildedge run`; both were only wired to the unused runner path before.
- Constructing a client without a DSN logs once per process at INFO. Previously every construction logged a WARNING.

### Removed

- `wildedge.runtime.runner`: an alternative bootstrap entry point that `wildedge run` never invoked. `sitecustomize` is the single bootstrap path; the runner's exit codes moved to `wildedge.runtime.bootstrap` and now apply under `--strict`.

## 0.1.5 - 2026-06-23

- Opt-in inference attachments (raw input/output upload)
- Fixed accelerator detection wiring; macOS CPU frequency and thermal sampling

## 0.1.4 and earlier

Predate this changelog; see the git history.
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@
1. Fork the repository and create a feature branch off `devel`.
2. Make your changes and ensure tests pass.
3. Update documentation if needed.
4. Submit a pull request targeting `devel` with a clear description of the changes.
4. Add a `CHANGELOG.md` entry under Unreleased for any user-visible or behavioral change.
5. Submit a pull request targeting `devel` with a clear description of the changes.

## Release process

Expand Down
45 changes: 32 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,12 @@ WILDEDGE_DSN="https://<secret>@ingest.wildedge.dev/<key>" \
wildedge run --integrations timm -- python app.py
```

Validate your environment before deploying:
Validate your environment before deploying. `--send-test-event` proves the
whole pipeline end to end by sending one span event and reporting the ingest
response (exit codes: 0 pass, 1 config failure, 2 connectivity failure):

```bash
wildedge doctor --integrations all --network-check
wildedge doctor --integrations all --send-test-event
```

Useful flags:
Expand All @@ -43,7 +45,8 @@ Useful flags:
| `--integrations` | Comma-separated list of integrations to activate (or `all`) |
| `--hubs` | Hub trackers to activate: `huggingface`, `torchhub` |
| `--print-startup-report` | Print per-integration status at startup |
| `--strict-integrations` | Fail if a requested integration can't be loaded |
| `--strict-integrations` | Exit (code 121) if a requested integration can't be instrumented |
| `--strict` | Exit (120 config, 122 internal) instead of running untracked when bootstrap fails |
| `--attachments` | Enable opt-in raw input/output attachment upload |
| `--no-propagate` | Don't pass WildEdge env vars to child processes |

Expand All @@ -52,18 +55,23 @@ Useful flags:
```python
import wildedge

client = wildedge.init(
dsn="...", # or WILDEDGE_DSN env var
integrations=["transformers"],
hubs=["huggingface"],
)
wildedge.init(integrations=["transformers"]) # optional under `wildedge run`

# models loaded after this point are tracked automatically
```
# models loaded after this point are tracked automatically; add traces,
# spans and LLM API calls anywhere, no client instance to pass around:
with wildedge.trace(run_id="run-1"):
with wildedge.span(kind="agent_step", name="plan"):
...

If no DSN is configured, the client becomes a no-op and logs a warning.
with wildedge.llm_api(model="openai/gpt-4o-mini", provider="openrouter") as call:
call.response(data) # LLM calls made with plain HTTP clients
```

`init(...)` is a convenience wrapper for `WildEdge(...)` + `instrument(...)`.
One client per process: `wildedge run`, `init()`, and the module-level calls
all share it, and `init()` without `dsn` reuses whatever already exists.
Without a DSN everything is a silent no-op, so dev and CI need no
configuration. See [Deployment](https://github.com/wild-edge/wildedge-python/blob/main/docs/deployment.md)
for the full contract.
## Supported integrations

**On-device**
Expand All @@ -87,6 +95,10 @@ If no DSN is configured, the client becomes a no-op and logs a warning.
| `anthropic` | [anthropic_example.py](https://github.com/wild-edge/wildedge-python/blob/main/examples/anthropic_example.py) |
| `openai` | [openai_example.py](https://github.com/wild-edge/wildedge-python/blob/main/examples/openai_example.py) |

Calling an LLM API with a plain HTTP client instead of these libraries? Use
[`wildedge.llm_api()`](https://github.com/wild-edge/wildedge-python/blob/main/docs/llm_api.md):
[llm_api_example.py](https://github.com/wild-edge/wildedge-python/blob/main/examples/llm_api_example.py).

**Hub tracking**

Pass `hubs=` to track model download provenance. Hubs are framework-agnostic and can be combined with any integration.
Expand Down Expand Up @@ -115,6 +127,7 @@ For advanced options (batching, queue tuning, dead-letter storage, attachments),

| Name | Link |
|---|---|
| outfitstudio.app | https://outfitstudio.app/ |
| agntr | [github.com/pmaciolek/agntr](https://github.com/pmaciolek/agntr) |
| demo-app | [github.com/wild-edge/demo-app](https://github.com/wild-edge/demo-app) |
| *(your project here)* | - |
Expand All @@ -131,6 +144,12 @@ Report security and privacy issues to: support@wildedge.dev

## Links

- [Deployment guide](https://github.com/wild-edge/wildedge-python/blob/main/docs/deployment.md)
- [Manual tracking](https://github.com/wild-edge/wildedge-python/blob/main/docs/manual-tracking.md)
- [LLM API tracking](https://github.com/wild-edge/wildedge-python/blob/main/docs/llm_api.md)
- [Compatibility Matrix](https://github.com/wild-edge/wildedge-python/blob/main/docs/compatibility.md)
- [Changelog](https://github.com/wild-edge/wildedge-python/releases)
- [Changelog](https://github.com/wild-edge/wildedge-python/blob/main/CHANGELOG.md)
- [License](https://github.com/wild-edge/wildedge-python/blob/main/LICENSE)

Each GitHub release ships `llms.txt` and `llms-full.txt`: the full
documentation for that exact version in one file, built for AI assistants.
4 changes: 3 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Configuration

Full reference for all `WildEdge` client parameters.
Full reference for all `WildEdge` client parameters. For runtime behavior
(no-DSN mode, strict mode and exit codes, forking servers), see
[Deployment](deployment.md).

## Core

Expand Down
122 changes: 122 additions & 0 deletions docs/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Deployment

How the SDK behaves in real process models: servers that fork, workers that
exec, build steps, and processes with no DSN configured. Everything on this
page is a contract the SDK keeps, not an implementation detail.

## How initialization happens

There are three ways a process gets its client, in order of coverage:

| Mode | Instrumentation coverage | Use when |
|---|---|---|
| `wildedge run -- <cmd>` | Guaranteed: patches before any user code runs | You control the start command |
| `wildedge.init(...)` at startup | Everything created after the call | You control application code |
| Lazy, from the first module-level call | Best effort: only what is created afterwards | Manual tracking only |

All three produce the same thing: one process-wide default client that
`wildedge.span()`, `wildedge.register_model()` and the rest delegate to.
`init()` without `dsn` reuses whatever client already exists, so code written
for in-process init runs unchanged under `wildedge run`.

The lazy mode configures itself from the environment: `WILDEDGE_DSN`,
`WILDEDGE_INTEGRATIONS`, `WILDEDGE_HUBS`. Unset integration variables mean
nothing is patched; auto-instrumentation is always opt-in.

## The no-DSN contract

Without `WILDEDGE_DSN`, the client is a no-op: no background threads, no
network, no patched frameworks, every event dropped. This is a supported mode
for development and CI, not an error; the SDK logs one INFO line per process
and stays quiet. Leave the SDK integrated and unset the variable wherever you
do not want telemetry (local dev, test runs, build and migration steps).

Under `wildedge run`, a missing DSN warns on stderr and your program runs
untracked. Telemetry failing must not take production down; that is the
default policy.

## Strict mode

`wildedge run --strict` inverts the failure policy for deployments where
running unobserved is worse than not running. Bootstrap failures then
terminate the process before your program starts:

| Exit code | Meaning |
|---|---|
| 120 | Configuration error (missing or invalid DSN) |
| 121 | A requested integration could not be instrumented (requires `--strict-integrations`) |
| 122 | Internal bootstrap error |

`--strict-integrations` is its own opt-in and exits with 121 on failure even
without `--strict`; asking for it is the request to fail.

## Forking and multi-process servers

`wildedge run` prepends `wildedge/autoload/` to `PYTHONPATH` and replaces
itself with your command. Every Python interpreter that starts under it runs
the bundled `sitecustomize.py`, which bootstraps the runtime before any user
code. Two mechanisms make this safe across worker models:

- Fresh interpreters (exec or spawn) bootstrap themselves via sitecustomize;
a `sys.modules` marker prevents double initialization within one
interpreter.
- Forked children inherit the parent's initialized runtime, and
`os.register_at_fork` hooks stop the SDK's background threads before each
fork and start fresh ones in parent and child afterward, so forked workers
transmit normally instead of inheriting dead threads.

How that plays out on common servers:

| Server | Worker model | Behavior under `wildedge run` |
|---|---|---|
| gunicorn (sync/gthread, with or without `preload_app`) | fork from master | Master bootstraps once; each worker gets fresh SDK threads via the at-fork hooks |
| uvicorn (`--workers`, `--reload`) | spawn / exec | Every worker is a fresh interpreter and bootstraps itself |
| granian | spawned worker processes | Each worker bootstraps itself |
| daphne | single process | One bootstrap, nothing special |
| waitress | single process, thread pool | One bootstrap; the client is thread-safe |
| celery (prefork pool) | fork from master | Same as gunicorn: at-fork hooks restart threads per worker |

For managed platforms, wrap only the serving command. Build steps and
migrations run in separate processes that gain nothing from telemetry; leave
them unwrapped or unset `WILDEDGE_DSN` there.

## One client per process

- Framework patches are installed at most once per process; the first client
to instrument owns the patch.
- `init()` reuses the existing default client unless you pass `dsn`
explicitly, so the CLI-installed client and application code share one
client instead of racing.
- Trace and span correlation lives in contextvars at module level, not on a
client. Auto-instrumented events emitted inside `wildedge.trace(...)` /
`wildedge.span(...)` blocks correlate into the same trace no matter which
client emits them.

## Verifying a deployment

`wildedge doctor` answers "will events actually flow from this machine"
before you rely on it:

```bash
wildedge doctor --integrations all --send-test-event --format json
```

- `--send-test-event` sends one real span event through the full pipeline
(DSN auth, ingest endpoint, batch protocol) and reports the server's
response, which a TCP-level `--network-check` cannot do.
- The report includes an `environment` section with every `WILDEDGE_*`
variable the runtime reads, plus whether the autoload dir is on
`PYTHONPATH`.
- Exit codes: 0 all pass, 1 configuration or dependency failure, 2 the
config is fine but the ingest endpoint is unreachable or rejecting.

`--format json` makes the output machine-readable; pasting it into an issue
or an AI assistant is the intended debugging flow.

## Environment propagation

By default, `wildedge run` leaves its `WILDEDGE_*` variables in the
environment so that exec'd children (reload workers) can bootstrap. Pass
`--no-propagate` to have each bootstrapped process scrub the run-scoped
variables after initialization, keeping them away from nested subprocesses
you spawn yourself.
82 changes: 82 additions & 0 deletions docs/llm_api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Tracking LLM calls without a client library

`wildedge.llm_api()` records LLM API calls made with any HTTP client: httpx or
requests against OpenRouter, vLLM, Ollama, or any OpenAI-compatible or
Anthropic-compatible endpoint. Use it when auto-instrumentation does not
apply because your app does not use the `openai` or `anthropic` packages. If
it does use them, prefer the integrations; they capture the same events with
zero code.

The name is deliberate: this tracks calls to an LLM behind an *API boundary*.
An LLM running inside your own process (llama.cpp, transformers, MLX) is
covered by the [framework integrations](manual-tracking.md), which also
capture what no API boundary can expose: quantization from the model
artifact, memory footprint, load/unload timing, and hardware linkage.

The block is timed automatically, events correlate with surrounding
`wildedge.trace()` / `wildedge.span()` blocks, and everything is a silent
no-op without a DSN.

## Quickstart

```python
import httpx
import wildedge

async def generate(prompt: str, model: str) -> dict:
with wildedge.llm_api(model=model, provider="openrouter", prompt=prompt) as call:
async with httpx.AsyncClient(timeout=300) as http:
response = await http.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {API_KEY}"},
json={"model": model, "messages": [{"role": "user", "content": prompt}]},
)
data = response.json()
call.response(data)
return data
```

`call.response()` pulls token usage, stop reason, and API metadata from the
full response payload, dict or SDK object, OpenAI shape
(`usage.prompt_tokens`, `choices[0].finish_reason`) or Anthropic shape
(`usage.input_tokens`, top-level `stop_reason`).

Runnable version: [examples/llm_api_example.py](../examples/llm_api_example.py),
stdlib urllib against OpenRouter, no client library at all.

## Recording pieces individually

When you do not have a full response payload, set what you know:

```python
with wildedge.llm_api(model="gemma-7b", base_url="http://localhost:11434") as call:
result = post_to_ollama(...)
call.usage(result["usage"]) # payload in either provider shape
call.usage(tokens_in=52, tokens_out=209) # or explicit fields; these win
call.stop_reason = "stop"
call.first_token() # TTFT mark for streaming
call.success = False # delivered but unusable output
```

An exception escaping the block records an error event with the exception
class as the error code, and no inference event.

## Model identity

Events register under `model` as the model id. `provider` names the source
directly; alternatively `base_url` derives it (`openrouter.ai` becomes
`openrouter`, unknown hosts record the hostname). Pass `prompt` or `messages`
(chat format) to attach input metadata such as prompt length.

## Agentic pipelines

Combine with traces for multi-step visibility:

```python
with wildedge.trace(run_id=run_id, agent_id="skin-generator"):
for attempt in range(2):
with wildedge.span(kind="agent_step", name="generate", step_index=attempt):
with wildedge.llm_api(model=model, provider="openrouter", prompt=prompt) as call:
data = await post_chat_completion(prompt)
call.response(data)
```
Loading
Loading