Skip to content

Add Algolia plugin#67

Open
andrewmumblebee wants to merge 12 commits into
mainfrom
work/ah/algolia
Open

Add Algolia plugin#67
andrewmumblebee wants to merge 12 commits into
mainfrom
work/ah/algolia

Conversation

@andrewmumblebee

@andrewmumblebee andrewmumblebee commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🔌 Plugin overview

  • Plugin name: Algolia
  • Purpose / problem solved: Monitor an Algolia search application from SquaredUp. Imports every Algolia index as an object and surfaces index size/growth, search volume, and search quality (no-result rate, top searches, top no-result searches) on out-of-the-box dashboards — so search and platform teams can track their Algolia estate alongside everything else in SquaredUp.
  • Primary audience (e.g. platform teams, SREs, product teams): Platform/search teams and SREs running Algolia-powered search; product teams tracking search quality (no-result rate, popular and failing queries).
  • Authentication method(s) (e.g. OAuth, Username/Password, API Key): API Key — Algolia Application ID + API Key sent as X-Algolia-Application-Id / X-Algolia-API-Key headers (an Admin key, or a custom key with the listIndexes, settings and analytics ACLs).

🖼️ Plugin screenshots

Plugin configuration

image

Default dashboards

image image

🧪 Testing

Built and tested end-to-end against a live, authenticated Algolia application in a SquaredUp tenant (every data stream tested live, not just authored):

  • Authentication — the configValidation Authenticate step (List Indices) probed via squaredup test returned HTTP 200.
  • Importsquaredup index completed successfully; 13 Algolia Index objects imported (records, data/file size, last build time, pending tasks) and confirmed present in the graph.
  • Data streams — all 5 verified live with squaredup test:
    • indexes (account-wide) — one flat row per index, correct numeric types.
    • searchCount, noResultRate, topSearches, noResultSearches (per-index, scoped) — tested against two different indices each and confirmed they return differing results (e.g. cloud_docs ≈ 551 searches/7d vs ce_docs ≈ 2); empty results for zero-traffic indices confirmed as valid HTTP 200 rather than errors.
  • Multi-host handling — Search streams use the templated https://{{applicationId}}.algolia.net host; Analytics streams override config.baseUrl to https://analytics.{{region}}.algolia.com. Auth headers (set once at plugin level) work across both hosts; analytics region resolved and confirmed.
  • Validation & deploysquaredup validate passes (5 data streams, 1 import definition, UI, icon, default content, config validation, custom types) and the plugin deploys cleanly.

⚠️ Known limitations

  • Search + Search Analytics only. Does not include the Algolia Usage API (operations/records time series) or Monitoring API (latency/indexing/infrastructure metrics) — those require an Algolia Premium/Enterprise plan and dedicated keys. Both are strong candidates for a v2.
  • Analytics granularity is daily — analytics tiles use timeframes of 7 days and longer (sub-daily windows like "last hour" aren't meaningful for this data).
  • Analytics need search traffic — indices with no recent searches show empty analytics tiles. This is expected, not an error.
  • Analytics region must match the app — the wrong region returns empty analytics. The Analytics region field (default US) handles EU-region apps, and the analytics baseUrl includes a || 'us' fallback so an accepted default still resolves correctly.
  • Rate limits — the Analytics API is limited to ~100 requests/minute per application; dashboards over very large numbers of indices may be throttled.
  • listIndexes ACL required — a Search-Only key will fail setup validation because it cannot list indices.
  • No monitors shipped — Search/Analytics data has no clean binary health signal, so no monitors are included (avoiding over-alerting).
  • Transient .tmp reindex indices are imported as objects and may appear/disappear between imports — a normal Algolia reindex artefact, not a plugin bug.

📚 Checklist

  • Plugin, datastream and UI naming follow SquaredUp guidelines
  • Logo added
  • One or more dashboards added
  • README added including configuration guidance
  • No secrets or credentials included
  • I agree to the Code of Conduct

@andrewmumblebee andrewmumblebee marked this pull request as ready for review June 16, 2026 15:44
@andrewmumblebee andrewmumblebee requested a review from a team June 16, 2026 15:44
@clarkd

clarkd commented Jun 17, 2026

Copy link
Copy Markdown
Member

@claude

"baseDataSourceName": "httpRequestScopedSingle",
"config": {
"httpMethod": "get",
"baseUrl": "https://analytics.{{dataSource.analyticsRegion || 'us'}}.algolia.com",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if this should be the primary baseUrl and only override it for the indexes data stream?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, have swapped it around

@clarkd

clarkd commented Jun 17, 2026

Copy link
Copy Markdown
Member

@claude review once

Comment thread plugins/Algolia/v1/metadata.json
Comment thread plugins/Algolia/v1/metadata.json Outdated
Comment thread plugins/Algolia/v1/dataStreams/indexes.json Outdated
Comment thread plugins/Algolia/v1/dataStreams/indexes.json Outdated
Comment thread plugins/Algolia/v1/ui.json Outdated
Comment thread plugins/Algolia/v1/dataStreams/noResultRate.json Outdated
Comment thread plugins/Algolia/v1/metadata.json
Comment thread plugins/Algolia/v1/dataStreams/indexes.json Outdated
Comment thread plugins/Algolia/v1/docs/README.md Outdated
Comment thread plugins/Algolia/v1/custom_types.json Outdated
Comment thread plugins/Algolia/v1/metadata.json Outdated
"importNotSupported": false,
"restrictedToPlatforms": [],
"keywords": ["algolia", "search", "analytics", "index", "search analytics"],
"objectTypes": ["Algolia Index"],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The 'Algolia Index' → 'Index' sourceType rename in commit e4473e5 was incomplete: metadata.json:13 still declares "objectTypes": ["Algolia Index"] (now out of sync with the renamed sourceType in custom_types.json, indexDefinitions/default.json, scopes.json and all four scoped data streams), and docs/README.md:39 ('What gets indexed' table) still labels the object type as Algolia Index even though custom_types.json declares singular: "Index" / plural: "Indices" so the UI now displays 'Index'/'Indices'. Update both occurrences to 'Index'. (Line 8's 'Algolia Index perspective' is fine — that's the dashboard name in algoliaIndex.dash.json, which legitimately remains 'Algolia Index'.)

Extended reasoning...

What the bug is

Commit e4473e5 ("Rename Algolia sourceType to Index as per guidelines") renamed the source type identifier from Algolia Index to Index across the plugin in response to an earlier review comment. The rename was applied to seven files but two occurrences were missed, leaving the plugin internally inconsistent and the user-facing documentation disagreeing with what the UI will actually display.

Where the rename was applied (correctly)

  • custom_types.json:3-4name and sourceType both set to "Index" (with singular: "Index" / plural: "Indices" providing the friendly display name)
  • indexDefinitions/default.json:10type.value: "Index" (this is the type every imported object is stamped with)
  • defaultContent/scopes.jsonsourceType.values: ["Index"]
  • dataStreams/noResultRate.json, dataStreams/noResultSearches.json, dataStreams/searchCount.json, dataStreams/topSearches.json — all four matches.sourceType.value set to "Index"

Where the rename was missed

1. plugins/Algolia/v1/metadata.json:13

"objectTypes": ["Algolia Index"]

The objectTypes field advertises which source types this plugin emits (used by the SquaredUp catalogue and type filters). It must match the actual emitted sourceType — but after the rename it doesn't. Sibling plugins confirm this convention: UptimeRobot/v2/metadata.json lists every type from its custom_types.json in objectTypes, and WorldCup2026/v1/metadata.json lists "World Cup Team" exactly matching its custom_types.json sourceType.

2. plugins/Algolia/v1/docs/README.md:39

| **Algolia Index** | A single search index in your application | ... |

After the rename, custom_types.json declares singular: "Index" / plural: "Indices", so the SquaredUp UI displays these objects as Index (singular) / Indices (plural) — not 'Algolia Index'. The README's user-facing 'Object type' column therefore disagrees with what the user will actually see in product.

What is intentionally NOT affected

The string 'Algolia Index' legitimately remains in two places that are not sourceType references:

  • README.md:8 — "…and an Algolia Index perspective…" refers to the dashboard whose name is literally "Algolia Index" in algoliaIndex.dash.json:2 (and the scope variable name "Algolia Index" in scopes.json). The dashboard/scope display name is separate from the sourceType.
  • algoliaIndex.dash.json:2"name": "Algolia Index" (the dashboard's display title).

These are correct and should be left as-is.

Impact

  • metadata.json: minor — objectTypes is used for catalogue/type-filter advertising. The plugin imports and queries indices correctly because every functional matcher uses "Index". But the metadata advertises a sourceType that the plugin no longer emits, which is an internal inconsistency in a brand-new plugin and directly contradicts the stated goal of commit e4473e5.
  • README.md: cosmetic/documentation — the docs claim the type is called 'Algolia Index' while users will see 'Index'/'Indices' in the UI. Documentation/UI drift on a brand-new plugin where the cost of fixing it is one line.

How to fix

Two one-line edits:

 # plugins/Algolia/v1/metadata.json line 13
-    "objectTypes": ["Algolia Index"],
+    "objectTypes": ["Index"],
 # plugins/Algolia/v1/docs/README.md line 39
-| **Algolia Index** | A single search index in your application | ... |
+| **Index**         | A single search index in your application | ... |

Step-by-step proof

  1. grep -rn '"Algolia Index"\|Algolia Index' plugins/Algolia/v1 — five hits.
  2. Two of those (algoliaIndex.dash.json:2 dashboard name, README.md:8 perspective reference) refer to the dashboard/scope display name, which is correctly still "Algolia Index".
  3. One (scopes.json) refers to the scope's own name field — also a display name, correctly preserved.
  4. The remaining two — metadata.json:13 (objectTypes) and README.md:39 ('Object type' table) — are sourceType references that should have been renamed.
  5. Cross-reference custom_types.json:3-4: name: "Index", sourceType: "Index", singular: "Index", plural: "Indices" — confirms the rename target.
  6. Cross-reference indexDefinitions/default.json:10 and all four scoped data streams' matches.sourceType.value — all "Index".
  7. Therefore the two unrenamed occurrences are leftovers from the incomplete commit, not intentional.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating metadata.json in d072d87objectTypes is now ["Index"]. The companion fix to docs/README.md:39 is still outstanding though: the 'What gets indexed' table still reads **Algolia Index** while the UI will display 'Index'/'Indices' (per custom_types.json's singular/plural). One more one-line edit:\n\ndiff\n-| **Algolia Index** | A single search index in your application | ... |\n+| **Index** | A single search index in your application | ... |\n

@@ -0,0 +1,50 @@
## What this plugin monitors

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 README headings start at level 2 (##) — line 1 opens with ## What this plugin monitors and the other top-level sections (lines 10, 27, 35, 43) are also ##. REVIEW.md:88 says "Should typically start headings from level 1. When embedded in SquaredUp, the headings will be sized appropriately", and every other plugin README in the repo (DigiCert, GoogleSheets, UptimeRobot, Huntress, Spotify, NinjaOne, UniFi, etc.) starts with #. Looks like an over-correction to the earlier # Algolia review nudge — dropping a level rather than renaming the heading. Promote the five top-level section headings back to #. Cosmetic nit only.

Extended reasoning...

What the issue is

plugins/Algolia/v1/docs/README.md opens with ## What this plugin monitors (a level-2 heading) and every other top-level section in the file (lines 10, 27, 35, 43 — ## Prerequisites — getting your credentials, ## Configuration fields, ## What gets indexed, ## Known limitations) is also level 2. There are no level-1 (#) headings anywhere in the file.

Why it matters

REVIEW.md:88 spells the convention out explicitly: "Should typically start headings from level 1. When embedded in SquaredUp, the headings will be sized appropriately." The README is rendered in-product under a built-in "Need help?" panel, and the in-product styling assumes the document's outer-most heading level is # — so a README that starts at ## ends up rendered with subtly wrong sizing/hierarchy relative to every other plugin's docs.

How this slipped through

This looks like an over-correction to the earlier review nudge against # Algolia as the opening heading. That earlier comment (inline #3433498073) correctly flagged # Algolia as a heading that just repeats the plugin name and recommended replacing it with something like # Before you start or # Prerequisites. The author instead dropped every top-level heading by one level — so the file went from # Algolia + ## ... to all ## — which now violates a different piece of the same REVIEW.md section. The intended fix is to keep the headings at level 1 but rename the first one (e.g. to # Before you start or by promoting ## Prerequisites — getting your credentials to be the first heading).

Convention across the repo

A sweep of every other plugin README under plugins/*/v*/docs/README.md confirms the convention is dominant and consistent: DigiCert, GoogleSheets, UptimeRobot, Huntress, Spotify, NinjaOne, UniFi, DattoRMM, AutoTask, Phare, Steam, WorldCup2026, RSS all start with a level-1 # Before you start; MicrosoftDefender uses # Configuring the data source. Algolia is the only outlier opening with ##. (SumoLogic also starts ## Setup, so the convention isn't strictly enforced — but it is documented and is the standard.)

Impact

Cosmetic only — no functional effect on validation, deployment, or runtime. The plugin loads and renders fine; the only consequence is that headings render with subtly wrong sizing inside the "Need help?" panel and the README's heading hierarchy doesn't match peer plugins'. Severity is nit.

How to fix

Promote the five top-level section headings from ## to #:

-## What this plugin monitors
+# What this plugin monitors
 ...
-## Prerequisites — getting your credentials
+# Prerequisites — getting your credentials
 ...
-## Configuration fields
+# Configuration fields
 ...
-## What gets indexed
+# What gets indexed
 ...
-## Known limitations
+# Known limitations

Or, to also address the earlier "don't repeat the plugin name" guidance simultaneously, rename the first heading to # Before you start while promoting the rest.

Step-by-step proof

  1. Open plugins/Algolia/v1/docs/README.md — line 1 is ## What this plugin monitors. There are no # (single-hash) headings anywhere in the file.
  2. Open REVIEW.md line 88 — "Should typically start headings from level 1. When embedded in SquaredUp, the headings will be sized appropriately."
  3. Open any other plugin README (e.g. plugins/DigiCert/v1/docs/README.md, plugins/UptimeRobot/v2/docs/README.md, plugins/Huntress/v1/docs/README.md) — every one starts with a # heading.
  4. Conclusion: the Algolia README is the only one in the repo that opens at level 2 and the only one with no level-1 heading at all; promote the top-level headings to # to match the documented convention.

Comment thread plugins/Algolia/v1/ui.json
"displayName": "Pending Tasks",
"shape": ["number", { "decimalPlaces": 0 }]
},
{ "name": "pendingTask", "displayName": "Has Pending Task" },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: the pendingTask column on plugins/Algolia/v1/dataStreams/indexes.json:42 has no shape declaration, while every other typed column in this same file (entries, dataSize, fileSize, lastBuildTimeS, numberOfPendingTasks, updatedAt) is shaped. Algolia returns pendingTask as a boolean and the displayName "Has Pending Task" reads as a yes/no question, so adding "shape": "boolean" would make the column render as a formatted indicator rather than a raw true/false string. The neighbouring primary column on line 44 is in the same situation. Pure cosmetic — the data flows correctly today.

Extended reasoning...

What the bug is

plugins/Algolia/v1/dataStreams/indexes.json:42 declares the pendingTask column with only a name and displayName:

{ "name": "pendingTask", "displayName": "Has Pending Task" },

Algolia's /1/indexes API returns pendingTask as a boolean (true if the index has tasks pending, false otherwise). The displayName "Has Pending Task" is phrased as a yes/no question, reinforcing the boolean intent.

Internal inconsistency within the same file

Every other typed column in this same stream is shaped — entries as number, dataSize/fileSize as bytes, lastBuildTimeS as seconds, numberOfPendingTasks as number, updatedAt as date (the last of which was specifically added in response to a prior review nudge in this same PR). So the shape-discipline is otherwise applied throughout the file; pendingTask is the lone exception. The neighbouring primary column on line 44 has the same omission.

Addressing the refutation

A reviewer correctly pointed out that the shape: boolean pattern is not universal across the repo — NinjaOne uses it consistently (smartCapable, isDynamic, etc.), but AutoTask boolean columns ship with no shape, and MicrosoftDefender uses shape: number for booleans. That's a fair point: there is no single repo-wide standard, and REVIEW.md's shapes guidance lists bytes/timestamp/duration as examples without naming boolean. This is why the issue is a nit, not a normal-severity bug — there's no documented rule being violated, only an internal inconsistency within this one file and a missed opportunity to use a more semantic shape.

Why it's still worth flagging

The column is rendered in the per-index Algolia Index dashboard's "Index Details" tile (a transposed datastream-properties table with no hiddenColumns filter), so users will see the raw value there — it's not entirely hidden behind the Overview dashboard's hide list. And the cost of the fix is two characters on a brand-new plugin where the author has otherwise been meticulous about shapes.

How to fix

Optionally apply the same to primary on line 44.

Step-by-step proof

  1. Open plugins/Algolia/v1/dataStreams/indexes.json:42{ "name": "pendingTask", "displayName": "Has Pending Task" } with no shape.
  2. Open lines 17–41 of the same file — every other typed column declares an appropriate shape.
  3. Cross-reference Algolia's /1/indexes API documentation — pendingTask is a boolean indicator.
  4. Cross-reference plugins/NinjaOne/v1/dataStreams/disks.json:88{ "name": "smartCapable", "displayName": "SMART Capable", "shape": "boolean" } confirms the pattern exists in the repo.
  5. Result: the column will display as a raw true/false string in the per-index dashboard rather than as a formatted boolean indicator.

{
"i": "9b9fc2b9-c56d-47de-a4c4-d151e467a210",
"x": 0,
"y": 6,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The 'Index Details' tile (y=6, h=4) overlaps rows 6-7 of the 'Search Volume' and 'No-Result Rate' tiles (both y=4, h=4) on the per-index dashboard — every other row in this dashboard is 4 units tall, so the third row should start at y=8 not y=6. React-grid-layout silently auto-resolves the collision at render time so the deployed dashboard looks fine, but the on-disk JSON describes a broken layout (no other dashboard in the repo has overlapping tiles), and any future edit through the SquaredUp UI will re-serialise with collision-resolved positions and produce a spurious diff. One-character fix: change y from 6 to 8 on line 14.

Extended reasoning...

What the bug is

plugins/Algolia/v1/defaultContent/algoliaIndex.dash.json line 14 sets the Index Details tile to y: 6. With h: 4 that tile occupies grid rows 6-9 across all four columns. The two tiles immediately above it — Search Volume (x:0, y:4, w:2, h:4) and No-Result Rate (x:2, y:4, w:2, h:4) — both occupy rows 4-7. So rows 6 and 7 in columns 0-3 are claimed by all three tiles simultaneously. This is an unambiguous grid collision in the committed JSON.

Why it still renders OK today

SquaredUp uses react-grid-layout, which silently auto-resolves overlaps at render time by pushing colliding tiles downward. So the dashboard renders as a clean 3-row stack (matching the screenshots in the PR description) — there is no user-visible bug today. The cost is structural: the file as committed no longer describes the layout that will actually be drawn. If a user opens this dashboard in the SquaredUp UI and saves any change, the re-serialised positions (which will be the collision-resolved ones) will diverge from the file, producing a noisy and confusing diff against main. Future readers reasoning about the layout straight from the JSON will also be misled.

How it slipped through

The authoring pattern is clearly a 3-row stack of h: 4 tiles: row 1 at y: 0, row 2 at y: 4, row 3 at y: 8. The y: 6 value is an off-by-two against that pattern — easy to make when hand-editing or when react-grid-layout has already rendered the dashboard correctly, masking the issue. I ran an overlap check over every other dash.json in the repo (including overview.dash.json added in this same PR) — zero of them have overlapping tile coordinates. The Algolia per-index dashboard is the only outlier in the repo.

Step-by-step proof

  1. Open plugins/Algolia/v1/defaultContent/algoliaIndex.dash.json — line 14 reads "y": 6,.
  2. Tile geometry from the file:
    • Top Searches: x=0, y=0, w=2, h=4 → rows 0-3, cols 0-1
    • Searches With No Results: x=2, y=0, w=2, h=4 → rows 0-3, cols 2-3
    • Search Volume: x=0, y=4, w=2, h=4rows 4-7, cols 0-1
    • No-Result Rate: x=2, y=4, w=2, h=4rows 4-7, cols 2-3
    • Index Details: x=0, y=6, w=4, h=4rows 6-9, cols 0-3
  3. Rows 6 and 7 in cols 0-3 are claimed by Index Details and also by Search Volume / No-Result Rate. Pairwise overlap check confirms Index Details ⇄ Search Volume and Index Details ⇄ No-Result Rate.
  4. The other rows of h=4 tiles in this dashboard sit at y=0 and y=4, so the natural third row is y=8 (stride 4).
  5. The deployed dashboard renders fine because react-grid-layout pushes the colliding tile down — but the file content disagrees with the rendered layout.

How to fix

One-character change on line 14:

-                "y": 6,
+                "y": 8,

This matches the stride-4 row pattern, lines the Index Details tile up directly below the analytics row, and brings the file in line with every other dashboard in the repo. Severity is nit because there is no runtime impact thanks to react-grid-layout's collision resolution — but worth fixing while landing a brand-new plugin since the diff is a single character.

@TimWheeler-SQUP TimWheeler-SQUP added the new-plugin Used to PR newly added plugins label Jun 18, 2026
@github-actions

Copy link
Copy Markdown

🧩 Plugin PR Summary

📦 Modified Plugins

  • plugins/Algolia/v1

📋 Results

Step Status
Validation ✅ Passed
Deployment 🚀 Deployed

🔍 Validation Details

algolia
{
  "valid": true,
  "pluginName": "algolia",
  "pluginType": "hybrid",
  "summary": {
    "Data Streams": 5,
    "Import Definitions": 1,
    "UI Configuration": true,
    "Has Icon": true,
    "Has Default Content": true,
    "Config Validation": true,
    "Custom Types": true
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new-plugin Used to PR newly added plugins

Development

Successfully merging this pull request may close these issues.

3 participants