Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,19 @@ jobs:
OUTPUT: /tmp/release-notes.md
run: python3 scripts/build_changelog_comment.py

- name: Build llms.txt artifacts
if: github.ref_type == 'tag'
env:
OUTPUT_DIR: llms-dist
run: python3 scripts/build_llms_txt.py

- name: Create GitHub release
if: github.ref_type == 'tag'
uses: softprops/action-gh-release@v2
with:
files: dist/*
files: |
dist/*
llms-dist/*
body_path: /tmp/release-notes.md

- name: Publish package to PyPI
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ under Unreleased and move into a version section at release time.

### Added

- `examples/llm_api_example.py`: raw-HTTP LLM tracking with `wildedge.llm_api()`; existing examples updated to the module-level API (`wildedge.span` / `wildedge.flush` / `wildedge.register_model` instead of threading a client variable)
- Releases ship `llms.txt` and `llms-full.txt` as GitHub release assets: the full documentation for that exact version in one file, generated by `scripts/build_llms_txt.py`. README quickstart rewritten around the module-level API.
- `wildedge doctor --send-test-event`: sends one real span event through the full pipeline and reports the ingest response, proving DSN auth and connectivity end to end. The report gains an `environment` section (`WILDEDGE_*` variables, autoload PYTHONPATH status) plus `config_status` / `connectivity_status` fields.
- `wildedge.llm_api()`: provider-agnostic tracking for LLM calls made with any HTTP client (OpenRouter, vLLM, Ollama, OpenAI/Anthropic-compatible endpoints). Times the block, normalizes usage payloads from either provider shape via `call.usage()` / `call.response()`, supports TTFT marks and async use, and records exceptions as error events. See `docs/llm_api.md`.
- `register_model()` without a matching extractor defaults the model name to the explicit `model_id` instead of the placeholder object's type name.
Expand Down
35 changes: 25 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,18 +55,23 @@ Useful flags:
```python
import wildedge

client = wildedge.init(
dsn="...", # or WILDEDGE_DSN env var
integrations=["transformers"],
hubs=["huggingface"],
)
wildedge.init(integrations=["transformers"]) # optional under `wildedge run`

# models loaded after this point are tracked automatically
```
# models loaded after this point are tracked automatically; add traces,
# spans and LLM API calls anywhere, no client instance to pass around:
with wildedge.trace(run_id="run-1"):
with wildedge.span(kind="agent_step", name="plan"):
...

If no DSN is configured, the client becomes a no-op and logs a warning.
with wildedge.llm_api(model="openai/gpt-4o-mini", provider="openrouter") as call:
call.response(data) # LLM calls made with plain HTTP clients
```

`init(...)` is a convenience wrapper for `WildEdge(...)` + `instrument(...)`.
One client per process: `wildedge run`, `init()`, and the module-level calls
all share it, and `init()` without `dsn` reuses whatever already exists.
Without a DSN everything is a silent no-op, so dev and CI need no
configuration. See [Deployment](https://github.com/wild-edge/wildedge-python/blob/main/docs/deployment.md)
for the full contract.
## Supported integrations

**On-device**
Expand All @@ -90,6 +95,10 @@ If no DSN is configured, the client becomes a no-op and logs a warning.
| `anthropic` | [anthropic_example.py](https://github.com/wild-edge/wildedge-python/blob/main/examples/anthropic_example.py) |
| `openai` | [openai_example.py](https://github.com/wild-edge/wildedge-python/blob/main/examples/openai_example.py) |

Calling an LLM API with a plain HTTP client instead of these libraries? Use
[`wildedge.llm_api()`](https://github.com/wild-edge/wildedge-python/blob/main/docs/llm_api.md):
[llm_api_example.py](https://github.com/wild-edge/wildedge-python/blob/main/examples/llm_api_example.py).

**Hub tracking**

Pass `hubs=` to track model download provenance. Hubs are framework-agnostic and can be combined with any integration.
Expand Down Expand Up @@ -135,6 +144,12 @@ Report security and privacy issues to: support@wildedge.dev

## Links

- [Deployment guide](https://github.com/wild-edge/wildedge-python/blob/main/docs/deployment.md)
- [Manual tracking](https://github.com/wild-edge/wildedge-python/blob/main/docs/manual-tracking.md)
- [LLM API tracking](https://github.com/wild-edge/wildedge-python/blob/main/docs/llm_api.md)
- [Compatibility Matrix](https://github.com/wild-edge/wildedge-python/blob/main/docs/compatibility.md)
- [Changelog](https://github.com/wild-edge/wildedge-python/releases)
- [Changelog](https://github.com/wild-edge/wildedge-python/blob/main/CHANGELOG.md)
- [License](https://github.com/wild-edge/wildedge-python/blob/main/LICENSE)

Each GitHub release ships `llms.txt` and `llms-full.txt`: the full
documentation for that exact version in one file, built for AI assistants.
3 changes: 3 additions & 0 deletions docs/llm_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ full response payload, dict or SDK object, OpenAI shape
(`usage.prompt_tokens`, `choices[0].finish_reason`) or Anthropic shape
(`usage.input_tokens`, top-level `stop_reason`).

Runnable version: [examples/llm_api_example.py](../examples/llm_api_example.py),
stdlib urllib against OpenRouter, no client library at all.

## Recording pieces individually

When you do not have a full response payload, set what you know:
Expand Down
12 changes: 6 additions & 6 deletions examples/agentic_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@

import wildedge

we = wildedge.init(
wildedge.init(
app_version="1.0.0",
integrations="openai",
)
Expand Down Expand Up @@ -96,7 +96,7 @@ def calculator(expression: str) -> str:


def call_tool(name: str, arguments: dict) -> str:
with we.span(
with wildedge.span(
kind="tool",
name=name,
input_summary=json.dumps(arguments)[:200],
Expand All @@ -108,7 +108,7 @@ def call_tool(name: str, arguments: dict) -> str:

def retrieve_context(query: str) -> str:
"""Fetch relevant context from the vector store (~120ms)."""
with we.span(
with wildedge.span(
kind="retrieval",
name="vector_search",
input_summary=query[:200],
Expand All @@ -125,7 +125,7 @@ def run_agent(task: str, step_index: int, messages: list) -> str:
messages.append({"role": "user", "content": f"{task}\n\nContext: {context}"})

while True:
with we.span(
with wildedge.span(
kind="agent_step",
name="reason",
step_index=step_index,
Expand Down Expand Up @@ -172,10 +172,10 @@ def run_agent(task: str, step_index: int, messages: list) -> str:
system_prompt = "You are a helpful assistant. Use tools when needed."
messages = [{"role": "system", "content": system_prompt}]

with we.trace(agent_id="demo-agent", run_id=str(uuid.uuid4())):
with wildedge.trace(agent_id="demo-agent", run_id=str(uuid.uuid4())):
for i, task in enumerate(TASKS, start=1):
print(f"\nTask {i}: {task}")
reply = run_agent(task, step_index=i, messages=messages)
print(f"Reply: {reply}")

we.flush()
wildedge.flush()
4 changes: 2 additions & 2 deletions examples/anthropic_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

import wildedge

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="anthropic",
)
Expand All @@ -44,5 +44,5 @@
print(event.delta.text, end="", flush=True)
print("\n")

client.flush()
wildedge.flush()
print("Done. Events flushed to WildEdge.")
6 changes: 3 additions & 3 deletions examples/attachments_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def redact(attachments: list[Attachment]) -> list[Attachment]:
return [a for a in attachments if a.content_type != "application/secret"]


client = wildedge.init(
wildedge.init(
app_version="1.0.0",
attachments_enabled=True,
max_attachments_per_inference=5,
Expand All @@ -35,7 +35,7 @@ def redact(attachments: list[Attachment]) -> list[Attachment]:
attachment_filter=redact,
)

handle = client.register_model(
handle = wildedge.register_model(
object(),
model_id="doc-classifier-v1",
source="local",
Expand All @@ -59,4 +59,4 @@ def redact(attachments: list[Attachment]) -> list[Attachment]:
print(f"tracked inference {inference_id[:8]}… with 2 attachments")

# Bytes upload in the background; flush/close lets buffered events drain.
client.close()
wildedge.flush()
4 changes: 2 additions & 2 deletions examples/feedback_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@

CONFIDENCE_THRESHOLD = 0.6

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="timm",
)

model = timm.create_model("resnet18", pretrained=True)
model.eval()
handle = client.register_model(
handle = wildedge.register_model(
model
) # auto-instrumented already; returns existing handle

Expand Down
2 changes: 1 addition & 1 deletion examples/gguf_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

import wildedge

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="gguf",
hubs=["huggingface"],
Expand Down
63 changes: 63 additions & 0 deletions examples/llm_api_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# /// script
# requires-python = ">=3.10"
# dependencies = ["wildedge-sdk"]
#
# [tool.uv.sources]
# wildedge-sdk = { path = "..", editable = true }
# ///
"""Track LLM calls made over plain HTTP with wildedge.llm_api().

No openai or anthropic client library involved: requests go through stdlib
urllib against OpenRouter's OpenAI-compatible endpoint, and llm_api() records
the same inference events the integrations emit (tokens, TTFT, stop reason),
correlated into the surrounding trace and spans.

Run with: uv run llm_api_example.py
Requires: OPENROUTER_API_KEY environment variable. Set WILDEDGE_DSN to send events.
"""

import json
import os
import urllib.request
import uuid

import wildedge

wildedge.init(app_version="1.0.0") # uses WILDEDGE_DSN if set; otherwise no-op

URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "openai/gpt-4o-mini"


def chat(prompt: str) -> dict:
request = urllib.request.Request(
URL,
data=json.dumps(
{"model": MODEL, "messages": [{"role": "user", "content": prompt}]}
).encode(),
headers={
"Authorization": f"Bearer {os.getenv('OPENROUTER_API_KEY')}",
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(request, timeout=120) as response:
return json.load(response)


PROMPTS = [
"What is on-device AI in one sentence?",
"Name three edge inference runtimes.",
]

with wildedge.trace(agent_id="llm-api-example", run_id=str(uuid.uuid4())):
for step, prompt in enumerate(PROMPTS):
with wildedge.span(kind="agent_step", name="ask", step_index=step):
with wildedge.llm_api(
model=MODEL, provider="openrouter", prompt=prompt
) as call:
data = chat(prompt)
call.response(data)
print(f"Q: {prompt}\nA: {data['choices'][0]['message']['content']}\n")

wildedge.flush()
print("Done. Events flushed to WildEdge.")
4 changes: 2 additions & 2 deletions examples/mlx_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ def main() -> None:

# init() constructs the client and instruments mlx; must be called
# before any model is loaded.
client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="mlx",
hubs=["huggingface"],
Expand All @@ -73,7 +73,7 @@ def main() -> None:
print(f"[{i}] Q: {prompt}")
print(f" A: {response}\n")

client.flush()
wildedge.flush()
print("Done. Events flushed to WildEdge.")


Expand Down
2 changes: 1 addition & 1 deletion examples/onnx_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

import wildedge

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="onnx",
hubs=["huggingface"],
Expand Down
4 changes: 2 additions & 2 deletions examples/openai_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

import wildedge

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="openai",
)
Expand Down Expand Up @@ -46,5 +46,5 @@
print(chunk.choices[0].delta.content, end="", flush=True)
print("\n")

client.flush()
wildedge.flush()
print("Done. Events flushed to WildEdge.")
4 changes: 2 additions & 2 deletions examples/tensorflow_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

import wildedge

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="tensorflow",
)
Expand Down Expand Up @@ -48,4 +48,4 @@ def build_and_save_model(save_path: Path) -> None:
print("output shape:", tuple(output.shape))


client.close()
wildedge.flush()
2 changes: 1 addition & 1 deletion examples/timm_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import wildedge

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="timm",
hubs=["huggingface", "torchhub"],
Expand Down
4 changes: 2 additions & 2 deletions examples/transformers_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def main() -> None:
)
args = parser.parse_args()

client = wildedge.init(
wildedge.init(
app_version="1.0.0", # uses WILDEDGE_DSN if set; otherwise no-op
integrations="transformers",
hubs=["huggingface"],
Expand All @@ -103,7 +103,7 @@ def main() -> None:
args.task
]()

client.flush()
wildedge.flush()
print("\nDone. Events flushed to WildEdge.")


Expand Down
Loading
Loading