|
@@ -33,29 +33,21 @@ Run:
|
|
|
|
|
|
|
|
## CLI arguments
|
|
## CLI arguments
|
|
|
|
|
|
|
|
-Run with `__help` for full list:
|
|
|
|
|
-
|
|
|
|
|
```
|
|
```
|
|
|
--c, --ctx <CTX>
|
|
|
|
|
- Context size (in 1024-byte units). Default: 32.
|
|
|
|
|
-
|
|
|
|
|
--t, --threads <THREADS>
|
|
|
|
|
- Threads to pass to llama-server. Default: 32.
|
|
|
|
|
-
|
|
|
|
|
---port <PORT>
|
|
|
|
|
- Port for llama-server. Default: 11434.
|
|
|
|
|
-
|
|
|
|
|
---host <HOST>
|
|
|
|
|
- Host to bind. Default: 0.0.0.0.
|
|
|
|
|
-
|
|
|
|
|
--b, --batch <BATCH>
|
|
|
|
|
- CPU offload batch size. Default: 512.
|
|
|
|
|
-
|
|
|
|
|
--u, --ubatch <UBATCH>
|
|
|
|
|
- GPU offload batch size. Default: 256.
|
|
|
|
|
-
|
|
|
|
|
--v, --verbose
|
|
|
|
|
- Print llama-server logs instead of discarding them.
|
|
|
|
|
|
|
+$ intern --help
|
|
|
|
|
+Binary search for best NGL and start llama-server
|
|
|
|
|
+
|
|
|
|
|
+Usage: intern [OPTIONS]
|
|
|
|
|
+
|
|
|
|
|
+Options:
|
|
|
|
|
+ -c, --ctx <CTX> [default: 32]
|
|
|
|
|
+ -t, --threads <THREADS> [default: 32]
|
|
|
|
|
+ -p, --port <PORT> [default: 11434]
|
|
|
|
|
+ --host <HOST> [default: 0.0.0.0]
|
|
|
|
|
+ -b, --batch <BATCH> [default: 512]
|
|
|
|
|
+ -u, --ubatch <UBATCH> [default: 256]
|
|
|
|
|
+ -v
|
|
|
|
|
+ -h, --help Print help
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
## Model selection
|
|
## Model selection
|