|
|
@@ -26,23 +26,30 @@ A Rust tool that binary-searches for the optimal GPU layer count (-ngl) when run
|
|
|
|
|
|
## CLI arguments
|
|
|
|
|
|
-Run with --help for full list:
|
|
|
-
|
|
|
-- -c, --ctx <CTX>
|
|
|
- - Context size (in 1024-byte units). Default: 32.
|
|
|
-
|
|
|
-- -t, --threads <THREADS>
|
|
|
- - Threads to pass to llama-server. Default: 32.
|
|
|
-- --port <PORT>
|
|
|
- - Port for llama-server. Default: 11434.
|
|
|
-- --host <HOST>
|
|
|
- - Host to bind. Default: 0.0.0.0.
|
|
|
-- -b, --batch <BATCH>
|
|
|
- - CPU offload batch size. Default: 512.
|
|
|
-- -u, --ubatch <UBATCH>
|
|
|
- - GPU offload batch size. Default: 256.
|
|
|
-- -v, --verbose
|
|
|
- - Print llama-server logs instead of discarding them.
|
|
|
+Run with `__help` for full list:
|
|
|
+
|
|
|
+```
|
|
|
+-c, --ctx <CTX>
|
|
|
+ Context size (in 1024-byte units). Default: 32.
|
|
|
+
|
|
|
+-t, --threads <THREADS>
|
|
|
+ Threads to pass to llama-server. Default: 32.
|
|
|
+
|
|
|
+--port <PORT>
|
|
|
+ Port for llama-server. Default: 11434.
|
|
|
+
|
|
|
+--host <HOST>
|
|
|
+ Host to bind. Default: 0.0.0.0.
|
|
|
+
|
|
|
+-b, --batch <BATCH>
|
|
|
+ CPU offload batch size. Default: 512.
|
|
|
+
|
|
|
+-u, --ubatch <UBATCH>
|
|
|
+ GPU offload batch size. Default: 256.
|
|
|
+
|
|
|
+-v, --verbose
|
|
|
+ Print llama-server logs instead of discarding them.
|
|
|
+```
|
|
|
|
|
|
## Model selection
|
|
|
|