# estagiário

A Rust tool that binary-searches for the optimal GPU layer count (-ngl) when running llama-server with a GGUF model, then launches the server with that configuration.

## How it works

- Tests different -ngl values (0..255) using binary search.
- For each candidate:
  - Spawns llama-server with that -ngl.
  - Checks if it outputs the expected response.
- Narrows down the highest valid -ngl.
- Spawns the final llama-server with best_fit - 1 to stay safe.

## Requirements

- Rust (edition 2024)
- llama-server in $PATH
- GGUF model(s) under ~/guff_models

## Build and run

- Build:
  - cargo build --release
- Run:
  - ./target/release/estagiario

## CLI arguments

Run with `__help` for full list:

```
-c, --ctx <CTX>
  Context size (in 1024-byte units). Default: 32.

-t, --threads <THREADS>
  Threads to pass to llama-server. Default: 32.

--port <PORT>
  Port for llama-server. Default: 11434.

--host <HOST>
  Host to bind. Default: 0.0.0.0.

-b, --batch <BATCH>
  CPU offload batch size. Default: 512.

-u, --ubatch <UBATCH>
  GPU offload batch size. Default: 256.

-v, --verbose
  Print llama-server logs instead of discarding them.
```

## Model selection

- The tool scans ~/guff_models for .gguf files.
- Lists them as:
  - a) model1.gguf
  - b) model2.gguf
  - ...
- Prompts for a letter to select.

## Behavior notes

- During binary search:
  - Each test run’s stdout/stderr is discarded.
  - Results are printed to stderr as:
    - <ngl> [ok] or <ngl> [fail]
- Final server:
  - Runs until interrupted (Ctrl+C).
  - Use --verbose to see its logs.

## Credits

- This project was built using vibecoding with Qwen 3.6 quantized (served locally), in conjunction with opencode.