ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-03-04 19:10:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Docs update (#509)

Browse Source 
* use npm as deps manager and vite as bundler

* update XTC docs

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
saood06
2025-06-09 05:32:03 -05:00
committed by GitHub
parent d1ae1504c6
commit fdc60d0aae
2 changed files with 35 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
examples/main/README.md
View File

@@ -241,13 +241,11 @@ Example usage: `--mirostat 2 --mirostat-lr 0.05 --mirostat-ent 3.0`
### XTC Sampling (Exclude Top Choices)
The function of this sampler is conrolled by `--xtc-probability` and `--xtc-threshold`. `--xtc-probability` takes values between
0 and 1 (<=0 turns this sampler off) and defines the probability for randomly invoking the sampler. `--xtc-threshold`
defines the token probability threshold. Tokens with probability greater than this threshold will be excluded from the sampling.
The sampler is turned off for `threshold > 0.5`.
- --xtc-probability p: xtc probability (default: 0.0, maximum: 1.0) => (p*100)% chance to invoke the sampler.
- --xtc-threshold t : xtc threshold (default: 1.0 minimum : 0.0) => (t*100)% is the lower-bound for what probability is needed for a token to be considered a "Top choice", and only the lowest probability top choice is kept, all others are removed.
The sampler is turned off for `threshold > 0.5`, and `--xtc-probability <=0`.
- --xtc-probability p: xtc probability (default: 0.0 => disabled)
- --xtc-threshold t : xtc threshold (default: 1.0 => disabled)
### Top-n-sigma Sampling

31
examples/server/README.md
View File

@@ -290,6 +290,37 @@ logging:
cmake --build build --config Release -t llama-server
```
## Web UI
The project includes a web-based user interface that enables interaction with the model through the `/chat/completions` endpoint.
The web UI is developed using:
- `vue` framework for frontend development
- `tailwindcss` and `daisyui` for styling
- `vite` for build tooling
A pre-built version is available as a single HTML file under `/public` directory.
To build or to run the dev server (with hot reload):
```sh
# make sure you have nodejs installed
cd examples/server/webui
npm i
# to run the dev server
npm run dev
# to build the public/index.html
npm run build
```
NOTE: if you are using the vite dev server, you can change the API base URL to llama.cpp. To do that, run this code snippet in browser's console:
```js
localStorage.setItem('base', 'http://localhost:8080')
```
## Quick Start
To get started right away, run the following command, making sure to use the correct path for the model you have: