[gen]erate LLM completions

A CLI tool for generating LLM completions and editing files. See the manifesto to learn why it exists and where development may or may not go in the future. I have no formal plans.

See basic usage below or look at a few examples.

Basic Usage

Run gen -h for all options. Generations below are purely illustrative.

Generate a response to a prompt.

$ gen "show me an example of python dict comprehension"
{key: value for key, value in {'a': 1, 'b': 2}.items()}

Add context from stdin

$ git log -n 3 | gen "categorize each commit as chore, feature, bugfix"
bugfix: fix race condition in data pipeline
chore: remove unused variable
chore: update runtime dependencies

Ask about a file, or multiple files sequentially.

$ gen "what's this commenting style called" program.py
Docstring

Edit a file with -e or -x. Review the diff before confirming, or use --force to write without review. When confirming you can also type feedback to append to your prompt and refine the edits.

When using larger models prefer -x to perform hash based file edits. This is typically more efficient in this case (fewer tokens, faster).

The -e flag performs edits by doing full file rewrites. This is slow but small models (those fitting on a 8GB GPU) tend to have better luck with this method compared to hash based editing.

Both flags are otherwise equivalent.

$ gen -x "use placeholders instead of f strings" program1.py
---
+++
@@ -1,7 +1,8 @@
 def user_confirmation(question):
-    answer = input(f'{question} [y/N]: ')
+    answer = input('{} [y/N]: '.format(question))

     while answer.lower() not in {'y', 'n'}:
         return user_confirmation(question)

     return answer == 'y'

Confirm changes to program1.py [y/n/*]: i dont like .format, use % instead

Provide as many files as you need for added context.

$ gen "any edge cases i havent thought to test?" test_program1.py -c program1.py program1_dep.py
[... and maybe you get a useful response]

Installation

1. Clone this repo and alias cli.py to whatever command you wish to use. gen is recommended and used in the examples in this readme. You will need Python to be installed, that is the only dependency.

2. Add your LLM provider(s). To generate the completions you'll need to point the tool to a supported provider via ~/.gen/config profiles. Read more in the Configuration section below.

Configuration

You configure and use multiple profiles with --profile. These are stored in ~/.gen/config with the following options. You must define a default profile.

[default]
provider = cerebras
effort = medium
model = gpt-oss-120b
key = your-api-key

[local]
provider = ollama
endpoint = http://<ollama-endpoint>
model = gemma4:e2b

Supported Provider Configurations

Provider	Model	Endpoint	Key	Effort
cerebras	required	n/a	required	optional
grok	required	n/a	required	optional
ollama	required	required1	n/a	n/a
openai	required	n/a	required	optional

Footnotes

1. Do not include the path. An example valid endpoint is http://localhost:8000 ↩