Kawrakow
|
1a4cfbcc53
|
Merge mainline - Aug 12 2024 (#17)
* Merge mainline
* Fix after merge
* Remove CI check
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
|
2024-08-12 15:14:32 +02:00 |
|
slaren
|
1caca63a87
|
Revert "[SYCL] Update rpc-server.cpp to include SYCL backend (#7682)" (#7808)
This reverts commit 9422c5e34b.
|
2024-06-09 01:43:39 +02:00 |
|
nickp27
|
496913a2c3
|
[SYCL] Update rpc-server.cpp to include SYCL backend (#7682)
* Update rpc-server.cpp to include SYCL backend
Draft PR to address inclusion of SYCL backend for RPC server
* Update rpc-server.cpp
|
2024-06-02 12:13:54 +03:00 |
|
Radoslav Gerganov
|
da03acb735
|
rpc : set SO_REUSEADDR for the server socket (#7320)
ref: #7293
|
2024-05-17 17:25:44 +03:00 |
|
Radoslav Gerganov
|
fe34112740
|
rpc : get available mem for the CPU backend
This can be overridden with the -m command line option
ref: #7293
|
2024-05-16 12:04:08 +03:00 |
|
Radoslav Gerganov
|
0b19253ad5
|
rpc : add command line arg for specifying backend memory
ref: #7293
|
2024-05-16 09:58:29 +03:00 |
|
Radoslav Gerganov
|
af81b28dbf
|
ggml : add RPC backend (#6829)
* ggml : add RPC backend
The RPC backend proxies all operations to a remote server which runs a
regular backend (CPU, CUDA, Metal, etc).
* set TCP_NODELAY
* add CI workflows
* Address review comments
* fix warning
* implement llama_max_devices() for RPC
* Address review comments
* Address review comments
* wrap sockfd into a struct
* implement get_alignment and get_max_size
* add get_device_memory
* fix warning
* win32 support
* add README
* readme : trim trailing whitespace
* Address review comments
* win32 fix
* Address review comments
* fix compile warnings on macos
|
2024-05-14 14:27:19 +03:00 |
|