OpenSyria Datasets API

Public read-only API for released OpenSyria datasets.

datasets-api serves stable, versioned reference data for Syria through documented HTTP endpoints. It exposes released dataset metadata, geography records, university profile records, source attribution, and machine-readable API documentation.

Table of Contents

• What This API Provides
• Data Flow
• Public Routes
• Query Conventions
• Localization
• Rate Limiting
• Local Development
• Local Read Model
• Validation
• Deployment
• Releases
• Public Documentation
• Contribution Model
• License

What This API Provides

• Dataset discovery and release metadata
• Geography endpoints for governorates, districts, subdistricts, and localities
• University endpoints for higher education institution profiles, logos, and ranking snapshots
• Stable record IDs and source attribution fields
• Offset pagination, filtering, search, and parent-child geography relationships
• Localized API response messages through lang, x-lang, or Accept-Language
• Scalar API reference and machine-readable OpenAPI documents

Data Flow

OpenSyria datasets live in separate repositories. This API does not read live main branches at runtime.

The production flow is:

dataset repositories -> versioned release artifacts -> verified JSON imports -> API read model -> public API responses

Dataset repositories own canonical JSON data, source attribution, validation rules, generated export files, and release manifests. datasets-api consumes pinned releases, verifies checksums and schemas, imports the data into read tables, then serves the public endpoints.

The pinned dataset release sources live in dataset-releases.json. Production does not automatically follow the latest GitHub release; changing the served dataset version is a reviewed code change plus a sync/import step.

See docs/dataset-loading.md, docs/release-manifest.md, and docs/read-model-architecture.md.

Public Routes

GET /health
GET /health/live
GET /health/ready

GET /api/v1/datasets
GET /api/v1/releases

GET /api/v1/geography/governorates
GET /api/v1/geography/governorates/:governorateId
GET /api/v1/geography/districts
GET /api/v1/geography/districts/:districtId
GET /api/v1/geography/subdistricts
GET /api/v1/geography/subdistricts/:subdistrictId
GET /api/v1/geography/localities
GET /api/v1/geography/localities/:localityId

GET /api/v1/universities
GET /api/v1/universities/:universityId

GET /docs
GET /swagger-ui
GET /openapi.json
GET /openapi/core.json
GET /openapi/geography.json
GET /openapi/universities.json
GET /favicon.ico

/docs and /swagger-ui use the complete /openapi.json document. The filtered /openapi/core.json, /openapi/geography.json, and /openapi/universities.json documents are available for tools that need a smaller machine-readable spec.

Query Conventions

List endpoints validate query parameters and path parameters before returning response envelopes.

Common list parameters:

Parameter	Values
page	Positive page number, default 1
limit	ten, thirty_five, or fifty
order	asc or desc
q	Search term
sourceStatus	pending_release, seed, released, or deprecated

Geography filters:

Endpoint	Extra filters
/geography/districts	governorateId
/geography/subdistricts	governorateId, districtId
/geography/localities	governorateId, districtId, subdistrictId, kind

University filters:

Endpoint	Extra filters
/universities	institutionType, governorate, hasWebsite

Example:

GET /api/v1/geography/localities?q=damascus&limit=thirty_five&order=asc&sourceStatus=released
GET /api/v1/universities?q=damascus&institutionType=public&limit=ten

Localization

Dataset text fields are returned as locale-keyed objects, such as { "en": "...", "ar": "..." }. This keeps the public reference data complete and cache-friendly.

Use X-Lang: en or X-Lang: ar to choose localized response envelope messages. The API also accepts lang query values and standard Accept-Language negotiation.

Rate Limiting

The public free tier allows 500 data API requests per client per day across /api/*. Health checks, documentation, OpenAPI JSON, and CORS preflight requests do not count against the daily quota.

When the quota is exhausted, the API returns HTTP 429 with a quota-specific message and a Retry-After header.

Until authenticated API keys are introduced, the client is identified from the trusted proxy chain, preferring Cloudflare's CF-Connecting-IP header in production.

Local Development

Requirements:

• Node.js 24+
• pnpm 11+
• Docker, when running PostgreSQL and Redis locally

Install dependencies:

corepack enable pnpm
pnpm install
pnpm run db:generate

Start the API without required external services:

pnpm run start:dev

The app uses APP_PORT, then PORT, and otherwise listens on 3000.

API documentation is available at:

http://localhost:3000/docs

Local Read Model

Start local dependencies:

docker compose up -d postgres redis
pnpm run db:migrate

Build a release in the sibling data-geography repository, then import it:

DATASETS_RELEASES_DIR=../data-geography/dist/release DATABASE_ENABLED=true pnpm run read-model:import:geography

Run a local geography smoke test:

pnpm run smoke:geography

Validation

pnpm run validate

This runs Prisma generation, Biome checks, ESLint, TypeScript type checking, unit tests, e2e tests, build verification, and a production dependency audit.

Useful focused commands:

pnpm run check
pnpm run lint
pnpm run typecheck
pnpm run test
pnpm run test:e2e
pnpm run test:integration:db
pnpm run build
pnpm run audit:prod

Deployment

Deployment notes live in docs/deployment.md.

Production deployments should normally set:

NODE_ENV=production
IS_HTTPS=true
DATABASE_ENABLED=true
DATABASE_REQUIRED=true
REDIS_ENABLED=true
REDIS_REQUIRED=true
DATASETS_REQUIRE_RELEASES=true

Set APP_TRUST_PROXY=true only when the service is behind a trusted reverse proxy or load balancer.

Build the Docker image:

docker build -t opensyria/datasets-api .

Run the built app:

docker run --rm -p 3000:3000 --env-file .env opensyria/datasets-api

Releases

Repository releases are managed by release-please. They provide version tags, GitHub Releases, and CHANGELOG.md entries for API code and documentation changes.

This is separate from production deployment, which uses SHA-pinned GHCR images, and separate from dataset artifact releases, which are pinned in dataset-releases.json.

See docs/releases.md.

Public Documentation

• API standards
• Dataset loading model
• Release manifest contract
• Read model architecture
• Deployment
• Releases
• Dataset contribution policy
• Pull request workflow
• Contributing
• Support
• Security policy
• Code of conduct

Contribution Model

This API repository is public for transparency, auditability, and reuse, but it is maintainer-led. Unsolicited feature pull requests are not currently accepted here.

Community contribution is intended primarily for the dataset repositories, where contributors can fix data, add missing records, improve sources, and propose schema improvements through a controlled review process.

License

MIT