ikawrakow/ik_llama.cpp
1
0
Fork 0
mirror of https://github.com/ikawrakow/ik_llama.cpp.git synced 2026-01-26 17:20:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

README: add "Supported platforms" + update hot topics

Browse Source
This commit is contained in:
Georgi Gerganov
2023-03-12 08:41:54 +02:00
committed by GitHub
parent a9b53a036b
commit 95fb97b137
1 changed files with 8 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
README.md
View File

@@ -5,10 +5,11 @@ Inference of [Facebook's LLaMA](https://github.com/facebookresearch/llama) model
**Hot topics**
- Running on Windows: https://github.com/ggerganov/llama.cpp/issues/22
- Fix Tokenizer / Unicode support: https://github.com/ggerganov/llama.cpp/issues/11
## Description
The main goal is to run the model using 4-bit quantization on a MacBook.
The main goal is to run the model using 4-bit quantization on a MacBook
- Plain C/C++ implementation without dependencies
- Apple silicon first-class citizen - optimized via Arm Neon and Accelerate framework
@@ -22,6 +23,12 @@ Please do not make conclusions about the models based on the results from this i
For all I know, it can be completely wrong. This project is for educational purposes and is not going to be maintained properly.
New features will probably be added mostly through community contributions, if any.
Supported platformst:
- [X] Mac OS
- [X] Linux
- [ ] Windows (soon)
---
Here is a typical run using LLaMA-7B: