# Intern

A Rust tool that binary-searches for the optimal GPU layer count (-ngl) when running llama-server with a GGUF model, then launches the server with that configuration.

## How it works

Tests different -ngl values (0..255) using binary search. For each candidate: Spawns llama-server with that -ngl. Checks if it outputs the expected response. Narrows down the highest valid -ngl. Spawns the final llama-server with best_fit - 1 to stay safe.

## Requirements

Rust (edition 2024), llama-server in $PATH, GGUF model(s) under ~/guff_models

## Build and run

```
Build:
  cargo build --release

Run:
  ./target/release/intern
```

## CLI arguments

```
$ intern --help
Binary search for best NGL and start llama-server

Usage: intern [OPTIONS]

Options:
  -c, --ctx <CTX>          [default: 32]
  -t, --threads <THREADS>  [default: 32]
  -p, --port <PORT>        [default: 11434]
      --host <HOST>        [default: 0.0.0.0]
  -b, --batch <BATCH>      [default: 512]
  -u, --ubatch <UBATCH>    [default: 256]
  -v                       
  -h, --help               Print help
```

## Model selection

The tool scans ~/guff_models for .gguf files, lists them as a) model1.gguf, b) model2.gguf, etc., and prompts for a letter to select.

## Behavior notes

During binary search, each test run's stdout/stderr is discarded. Results are printed to stderr as `<ngl> [ok]` or `<ngl> [fail]`. The final server runs until interrupted (Ctrl+C). Use `--verbose` to see its logs.

## Credits

This project was built using vibecoding with Qwen 3.6 quantized (served locally), in conjunction with opencode.