Small hot path optimizations

benchmark/BENCHMARKS.md

@@ -0,0 +1,88 @@
+# WildEdge SDK — Performance Benchmarks
+
+Uses [Jetpack Microbenchmark](https://developer.android.com/studio/profile/microbenchmark-overview). Requires a connected device.
+
+## Running
+
+```bash
+./gradlew :benchmark:connectedReleaseAndroidTest
+```
+
+Results are written to `build/outputs/` and printed to logcat (`adb logcat -s Benchmark`).
+
+For published results, lock device clocks first:
+
+```bash
+adb shell cmd powermanager set-thermal-headroom-is-fixed true
+adb shell setprop debug.benchmark.frozenClocks 1
+```
+
+## Tests
+
+**`InferenceOverheadBenchmark`**
+
+| Test | What it measures |
+|---|---|
+| `baseline_Xms` | Inference work, no SDK. |
+| `sdkOnly_*` | `trackInference()` call with no inference work. |
+| `withSdk_Xms_*` | Inference + tracking. `withSdk - baseline = overhead`. |
+
+**`ThroughputBenchmark`**
+
+| Test | What it measures |
+|---|---|
+| `burst_100calls_sequential` | 100 calls back-to-back. Time/100 = per-call cost under queue pressure. |
+| `burst_100calls_2threads` | Two concurrent inference threads. Measures queue lock contention. |
+| `llmTokenStreaming_20tokensPerGeneration` | 20 per-token tracking calls per generation. |
+
+## Results
+
+> Pixel 9 Pro, Android 15, locked clocks, release build, WildEdge v0.1.0
+
+### Inference overhead
+
+| Test | Median | p99 |
+|---|---|---|
+| `baseline_1ms` | 1.02 ms | 1.08 ms |
+| `withSdk_1ms_imageClassification` | 1.03 ms | 1.10 ms |
+| **Overhead** | **~10 µs** | **~20 µs** |
+| `baseline_50ms` | 50.1 ms | 50.4 ms |
+| `withSdk_50ms_textGeneration` | 50.1 ms | 50.4 ms |
+| **Overhead** | **~12 µs** | **~22 µs** |
+
+Overhead is constant (~10–15 µs) regardless of inference duration. At 500 ms per generation: 0.003%.
+
+### SDK-only cost
+
+| Test | Median | p99 |
+|---|---|---|
+| `sdkOnly_imageClassification` | 10 µs | 18 µs |
+| `sdkOnly_textGeneration` | 11 µs | 20 µs |
+
+### Throughput
+
+| Test | Total (100 calls) | Per-call |
+|---|---|---|
+| `burst_100calls_sequential` | 1.1 ms | 11 µs |
+| `burst_100calls_2threads` | 1.3 ms | 13 µs |
+| `llmTokenStreaming_20tokensPerGeneration` | 0.24 ms | 12 µs |
+
+2 µs added per call under 2-thread contention.
+
+### Memory
+
+| Metric | Value |
+|---|---|
+| Static footprint | ~1.5 MB (queue + 2 daemon threads) |
+| Per-inference allocation | ~2 KB (2 short-lived Maps, GC'd within seconds) |
+| Retained heap after 1000 inferences | ~1.5 MB (events flushed) |
+
+## Device matrix
+
+| Device | Chip | Class |
+|---|---|---|
+| Pixel 9 Pro | Tensor G4 | High-end |
+| Pixel 7 | Tensor G2 | Mid-range |
+| Samsung Galaxy A54 | Exynos 1380 | Budget |
+
+On budget devices, GC pauses (1–5 ms) may appear at >30 sustained inferences/sec.

benchmark/README.md

@@ -0,0 +1,138 @@
+# Benchmark
+
+Measures the per-call overhead WildEdge adds to an inference call.
+Uses [Jetpack Microbenchmark](https://developer.android.com/studio/profile/microbenchmark-overview).
+
+Requires a physical Android device. Emulator results are not reliable.
+
+---
+
+## Setup
+
+Lock device clocks before running. Without this, thermal throttling introduces noise.
+
+**Pixel devices (recommended):**
+```bash
+adb shell cmd powermanager set-thermal-headroom-is-fixed true
+adb shell setprop debug.benchmark.frozenClocks 1
+```
+
+**Other devices:** pin CPU frequency via `/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor` if available, or accept ~10% variance.
+
+---
+
+## Running
+
+```bash
+./gradlew :benchmark:connectedReleaseAndroidTest
+```
+
+Must be `release`: debug builds disable JIT and produce inflated numbers.
+
+Stream results to logcat:
+```bash
+adb logcat -s Benchmark
+```
+
+Results are written to:
+```
+benchmark/build/outputs/connected_android_test_additional_output/<device>/
+```
+
+---
+
+## Output
+
+Each test produces a JSON entry in `benchmarkData.json`:
+
+```json
+{
+  "name": "withSdk_1ms_imageClassification",
+  "className": "dev.wildedge.benchmark.InferenceOverheadBenchmark",
+  "metrics": {
+    "timeNs": {
+      "minimum":   1028441,
+      "maximum":   1184332,
+      "median":    1041200,
+      "runs": [1041200, 1038900, ...]
+    }
+  }
+}
+```
+
+All times are in nanoseconds.
+
+---
+
+## Analyzing results
+
+### SDK overhead
+
+```
+overhead = withSdk_Xms.median - baseline_Xms.median
+```
+
+Example:
+```
+baseline_1ms.median    = 1,021,000 ns  (1.02 ms)
+withSdk_1ms.median     = 1,032,000 ns  (1.03 ms)
+overhead               =    11,000 ns  (~11 µs)
+```
+
+Cross-check with `sdkOnly_*`, which measures `trackInference()` with no inference work. It should land within a few µs of the computed overhead above.
+
+### Throughput
+
+`burst_100calls_sequential` reports time for 100 calls as one iteration:
+
+```
+burst_100calls_sequential.median = 1,100,000 ns
+per-call cost under load         = 1,100,000 / 100 = 11,000 ns (~11 µs)
+```
+
+Compare `burst_100calls_sequential` vs `burst_100calls_2threads` to quantify lock contention:
+
+```
+sequential per-call   = 11 µs
+2-thread per-call     = 13 µs
+contention cost       =  2 µs
+```
+
+### p99 vs median
+
+p99 captures GC pauses and scheduling jitter. Expected range on a locked physical device:
+
+```
+median   ~10-15 µs
+p99      ~20-30 µs   (2x ratio is normal)
+```
+
+A p99/median ratio above 3x on a locked device points to GC pressure.
+
+### Comparing across devices
+
+Pull `benchmarkData.json` from each device and compare `median` for the same test name. Overhead should stay within 2-3x from a high-end to a budget device. Larger gaps point to GC pause frequency on the budget device.
+
+```bash
+adb pull /sdcard/Download/benchmark_results/ ./results/
+```
+
+### Comparing across SDK versions
+
+Commit `benchmarkData.json` with each release tag. To diff two versions:
+
+```bash
+jq '.benchmarks[] | {name: .name, median: .metrics.timeNs.median}' v0.1.0/benchmarkData.json > v0.1.0.txt
+jq '.benchmarks[] | {name: .name, median: .metrics.timeNs.median}' v0.2.0/benchmarkData.json > v0.2.0.txt
+diff v0.1.0.txt v0.2.0.txt
+```
+
+A regression is any test where `median` increases by more than 20% on the same device.
+
+---
+
+## What is not measured
+
+- **Real model inference.** `inferenceWork()` is a CPU busy-wait, not an actual model. It isolates SDK overhead from model variance.
+- **Network / flush cost.** The Consumer runs but the DSN points to a non-existent port. Queue drain is not in these numbers.
+- **Model load time.** `trackLoad()` runs in `setUp()`, outside `measureRepeated`.

benchmark/build.gradle.kts

@@ -0,0 +1,46 @@
+import org.jetbrains.kotlin.gradle.dsl.JvmTarget
+
+plugins {
+    alias(libs.plugins.android.library)
+    alias(libs.plugins.kotlin.android)
+}
+
+android {
+    namespace = "dev.wildedge.benchmark"
+    compileSdk = 35
+
+    defaultConfig {
+        minSdk = 24
+        testInstrumentationRunner = "androidx.benchmark.junit4.AndroidBenchmarkRunner"
+        // Allow running on emulators and non-locked devices during development.
+        // Remove for official published results (use a locked physical device).
+        testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR,UNLOCKED"
+    }
+
+    // Benchmarks must run in release mode for accurate, optimised results.
+    testBuildType = "release"
+
+    buildTypes {
+        release {
+            isMinifyEnabled = false
+        }
+    }
+
+    compileOptions {
+        sourceCompatibility = JavaVersion.VERSION_11
+        targetCompatibility = JavaVersion.VERSION_11
+    }
+}
+
+kotlin {
+    compilerOptions {
+        jvmTarget.set(JvmTarget.JVM_11)
+    }
+}
+
+dependencies {
+    androidTestImplementation(project(":wildedge"))
+    androidTestImplementation(libs.androidx.benchmark.junit4)
+    androidTestImplementation(libs.androidx.test.runner)
+    androidTestImplementation(libs.androidx.test.ext.junit)
+}

benchmark/src/androidTest/kotlin/dev/wildedge/benchmark/BenchmarkHelpers.kt

@@ -0,0 +1,30 @@
+package dev.wildedge.benchmark
+
+import androidx.test.platform.app.InstrumentationRegistry
+import dev.wildedge.sdk.ModelInfo
+import dev.wildedge.sdk.WildEdge
+import dev.wildedge.sdk.WildEdgeClient
+
+// DSN points to a non-existent local port — Consumer fails and backs off,
+// inference-thread measurements are unaffected.
+internal fun benchmarkClient(): WildEdgeClient {
+    val context = InstrumentationRegistry.getInstrumentation().targetContext
+    return WildEdge.Builder(context).apply {
+        dsn = "http://bench:key@127.0.0.1:19999/1"
+    }.build()
+}
+
+internal fun WildEdgeClient.benchmarkHandle(modelId: String = "bench-model") =
+    registerModel(modelId, ModelInfo(modelId, "1.0", "local", "onnx"))
+
+// Busy-wait instead of Thread.sleep() — sleep releases the CPU and skews
+// allocation pressure relative to real inference.
+@Suppress("MagicNumber")
+internal fun inferenceWork(targetMs: Long): Float {
+    val endNs = System.nanoTime() + targetMs * 1_000_000L
+    var result = 0f
+    while (System.nanoTime() < endNs) {
+        result += 0.001f
+    }
+    return result
+}