Skip to content

Home

Jump to bottom
Jerome White edited this page Apr 8, 2025 · 3 revisions

Welcome to the prompt wiki!

Examples

These are example of how to use this platform with datasets currently in Tech4Dev workspace. Throughout, we assume

  1. You are in eval data directory:

    git clone https://github.com/ProjectTech4DevAI/file-search-eval.git
cd file-search-eval

  2. You have followed the general setup described in the README

Kunji phase data

Kunji data is split across two Github repositories differentiated by phase:

  1. Phase 1
  2. Phase 2: Haryana

They have the same structure, so using either can be reduced to variable naming. For this example:

KUNJI_DATA=veddis-eval-data
# KUNJI_DATA=kunji-haryana-data # also valid, for Phase 2 Haryana data

From there we can clone the repository and run the scripts:

git clone https://github.com/ProjectTech4DevAI/${KUNJI_DATA}.git
gt="-g $KUNJI_DATA/assessments/correct"
./bin/run-prompts.sh $gt -p $KUNJI_DATA/prompts -d $KUNJI_DATA/docs \
    | ./bin/run-evals.sh $gt -u $KUNJI_DATA/prompts/user > evaluations.jsonl

Clone this wiki locally