mirror of
https://github.com/ikawrakow/ik_llama.cpp.git
synced 2026-03-02 01:50:01 +00:00
webui update (#1003)
webui: add system message in export conversation, support upload conversation with system message Webui: show upload only when in new conversation Webui: Add model name webui: increase height of chat message window when clicking editing Webui: autoclose settings dialog dropdown and maximze screen width when zoom in webui: fix date issues and add more dates webui: change error to toast.error. server: add n_past and slot_id in props_simple webui: add cache tokens, context and prompt speed in chat webui: modernize ui webui: change welcome message webui: change speed display webui: change run python icon webui: add config to use server defaults for sampler webui: put speed on left and context on right webui: recognize AsciiDoc files as valid text files (#16850) * webui: recognize AsciiDoc files as valid text files * webui: add an updated static webui build * webui: add the updated dependency list * webui: re-add an updated static webui build Add a setting to display message generation statistics (#16901) * feat: Add setting to display message generation statistics * chore: build static webui output webui: add HTML/JS preview support to MarkdownContent with sandboxed iframe (#16757) * webui: add HTML/JS preview support to MarkdownContent with sandboxed iframe dialog Extended MarkdownContent to flag previewable code languages, add a preview button alongside copy controls, manage preview dialog state, and share styling for the new button group Introduced CodePreviewDialog.svelte, a sandboxed iframe modal for rendering HTML/JS previews with consistent dialog controls * webui: fullscreen HTML preview dialog using bits-ui * Update tools/server/webui/src/lib/components/app/misc/CodePreviewDialog.svelte Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * Update tools/server/webui/src/lib/components/app/misc/MarkdownContent.svelte Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * webui: pedantic style tweak for CodePreviewDialog close button * webui: remove overengineered preview language logic * chore: update webui static build --------- Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> webui: auto-refresh /props on inference start to resync model metadata (#16784) * webui: auto-refresh /props on inference start to resync model metadata - Add no-cache headers to /props and /slots - Throttle slot checks to 30s - Prevent concurrent fetches with promise guard - Trigger refresh from chat streaming for legacy and ModelSelector - Show dynamic serverWarning when using cached data * fix: restore proper legacy behavior in webui by using unified /props refresh Updated assistant message bubbles to show each message's stored model when available, falling back to the current server model only when the per-message value is missing When the model selector is disabled, now fetches /props and prioritizes that model name over chunk metadata, then persists it with the streamed message so legacy mode properly reflects the backend configuration * fix: detect first valid SSE chunk and refresh server props once * fix: removed the slots availability throttle constant and state * webui: purge ai-generated cruft * chore: update webui static build feat(webui): improve LaTeX rendering with currency detection (#16508) * webui : Revised LaTeX formula recognition * webui : Further examples containg amounts * webui : vitest for maskInlineLaTeX * webui: Moved preprocessLaTeX to lib/utils * webui: LaTeX in table-cells * chore: update webui build output (use theirs) * webui: backslash in LaTeX-preprocessing * chore: update webui build output * webui: look-behind backslash-check * chore: update webui build output * Apply suggestions from code review Code maintenance (variable names, code formatting, string handling) Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * webui: Moved constants to lib/constants. * webui: package woff2 inside base64 data * webui: LaTeX-line-break in display formula * chore: update webui build output * webui: Bugfix (font embedding) * webui: Bugfix (font embedding) * webui: vite embeds assets * webui: don't suppress 404 (fonts) * refactor: KaTeX integration with SCSS Moves KaTeX styling to SCSS for better customization and font embedding. This change includes: - Adding `sass` as a dev dependency. - Introducing a custom SCSS file to override KaTeX variables and disable TTF/WOFF fonts, relying solely on WOFF2 for embedding. - Adjusting the Vite configuration to resolve `katex-fonts` alias and inject SCSS variables. * fix: LaTeX processing within blockquotes * webui: update webui build output --------- Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> server : add props.model_alias (#16943) * server : add props.model_alias webui: fix keyboard shortcuts for new chat & edit chat title (#17007) Better UX for handling multiple attachments in WebUI (#17246) webui: add OAI-Compat Harmony tool-call streaming visualization and persistence in chat UI (#16618) * webui: add OAI-Compat Harmony tool-call live streaming visualization and persistence in chat UI - Purely visual and diagnostic change, no effect on model context, prompt construction, or inference behavior - Captured assistant tool call payloads during streaming and non-streaming completions, and persisted them in chat state and storage for downstream use - Exposed parsed tool call labels beneath the assistant's model info line with graceful fallback when parsing fails - Added tool call badges beneath assistant responses that expose JSON tooltips and copy their payloads when clicked, matching the existing model badge styling - Added a user-facing setting to toggle tool call visibility to the Developer settings section directly under the model selector option * webui: remove scroll listener causing unnecessary layout updates (model selector) * Update tools/server/webui/src/lib/components/app/chat/ChatMessages/ChatMessageAssistant.svelte Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * Update tools/server/webui/src/lib/components/app/chat/ChatMessages/ChatMessageAssistant.svelte Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> * chore: npm run format & update webui build output * chore: update webui build output --------- Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com> webui: Fix clickability around chat processing statistics UI (#17278) * fix: Better pointer events handling in chat processing info elements * chore: update webui build output Fix merge error webui: Add a "Continue" Action for Assistant Message (#16971) * feat: Add "Continue" action for assistant messages * feat: Continuation logic & prompt improvements * chore: update webui build output * feat: Improve logic for continuing the assistant message * chore: update webui build output * chore: Linting * chore: update webui build output * fix: Remove synthetic prompt logic, use the prefill feature by sending the conversation payload ending with assistant message * chore: update webui build output * feat: Enable "Continue" button based on config & non-reasoning model type * chore: update webui build output * chore: Update packages with `npm audit fix` * fix: Remove redundant error * chore: update webui build output * chore: Update `.gitignore` * fix: Add missing change * feat: Add auto-resizing for Edit Assistant/User Message textareas * chore: update webui build output Improved file naming & structure for UI components (#17405) * refactor: Component iles naming & structure * chore: update webui build output * refactor: Dialog titles + components namig * chore: update webui build output * refactor: Imports * chore: update webui build output webui: hide border of button webui: update webui: update webui: update add vision webui: minor settings reorganization and add disable autoscroll option (#17452) * webui: added a dedicated 'Display' settings section that groups visualization options * webui: added a Display setting to toggle automatic chat scrolling * chore: update webui build output Co-authored-by: firecoperana <firecoperana>
This commit is contained in:
firecoperana
@@ -13,7 +13,7 @@ import {
|
||||
filterThoughtFromMsgs,
|
||||
normalizeMsgsForAPI,
|
||||
getSSEStreamAsync,
|
||||
getServerProps
|
||||
getServerProps,
|
||||
} from './misc';
|
||||
import { BASE_URL, CONFIG_DEFAULT, isDev } from '../Config';
|
||||
import { matchPath, useLocation, useNavigate } from 'react-router';
|
||||
@@ -110,8 +110,7 @@ export const AppContextProvider = ({
|
||||
setServerProps(props);
|
||||
})
|
||||
.catch((err) => {
|
||||
console.error(err);
|
||||
toast.error('Failed to fetch server props');
|
||||
console.error(err);
|
||||
});
|
||||
// eslint-disable-next-line
|
||||
}, []);
|
||||
@@ -216,6 +215,7 @@ export const AppContextProvider = ({
|
||||
content: null,
|
||||
parent: leafNodeId,
|
||||
children: [],
|
||||
model_name: '',
|
||||
};
|
||||
setPending(convId, pendingMsg as PendingMessage);
|
||||
}
|
||||
@@ -240,13 +240,8 @@ export const AppContextProvider = ({
|
||||
cache_prompt: true,
|
||||
reasoning_format: config.reasoning_format===''?'auto':config.reasoning_format,
|
||||
samplers: config.samplers,
|
||||
temperature: config.temperature,
|
||||
dynatemp_range: config.dynatemp_range,
|
||||
dynatemp_exponent: config.dynatemp_exponent,
|
||||
top_k: config.top_k,
|
||||
top_p: config.top_p,
|
||||
min_p: config.min_p,
|
||||
typical_p: config.typical_p,
|
||||
xtc_probability: config.xtc_probability,
|
||||
xtc_threshold: config.xtc_threshold,
|
||||
top_n_sigma: config.top_n_sigma,
|
||||
@@ -260,6 +255,13 @@ export const AppContextProvider = ({
|
||||
dry_penalty_last_n: config.dry_penalty_last_n,
|
||||
max_tokens: config.max_tokens,
|
||||
timings_per_token: !!config.showTokensPerSecond,
|
||||
...(config.useServerDefaults ? {} :{
|
||||
temperature: config.temperature,
|
||||
top_k: config.top_k,
|
||||
top_p: config.top_p,
|
||||
min_p: config.min_p,
|
||||
typical_p: config.typical_p,
|
||||
}),
|
||||
...(config.custom.length ? JSON.parse(config.custom) : {}),
|
||||
};
|
||||
|
||||
@@ -322,6 +324,8 @@ export const AppContextProvider = ({
|
||||
prompt_ms: timings.prompt_ms,
|
||||
predicted_n: timings.predicted_n,
|
||||
predicted_ms: timings.predicted_ms,
|
||||
n_ctx: timings.n_ctx,
|
||||
n_past: timings.n_past,
|
||||
};
|
||||
}
|
||||
setPending(convId, pendingMsg as PendingMessage);
|
||||
@@ -333,10 +337,7 @@ export const AppContextProvider = ({
|
||||
// user stopped the generation via stopGeneration() function
|
||||
// we can safely ignore this error
|
||||
} else {
|
||||
console.error(err);
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
alert((err as any)?.message ?? 'Unknown error');
|
||||
//throw err; // rethrow
|
||||
toast.error(err instanceof Error ? err.message : String(err));
|
||||
}
|
||||
}
|
||||
finally {
|
||||
@@ -344,7 +345,7 @@ export const AppContextProvider = ({
|
||||
if (isContinuation) {
|
||||
await StorageUtils.updateMessage(pendingMsg as Message);
|
||||
} else if (pendingMsg.content.trim().length > 0) {
|
||||
await StorageUtils.appendMsg(pendingMsg as Message, leafNodeId);
|
||||
await StorageUtils.appendMsg(pendingMsg as Message, leafNodeId, '');
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -375,6 +376,16 @@ export const AppContextProvider = ({
|
||||
const now = Date.now()+Timer.timercount;
|
||||
Timer.timercount=Timer.timercount + 2;
|
||||
const currMsgId = now;
|
||||
|
||||
let model_name:string='';
|
||||
await getServerProps(BASE_URL)
|
||||
.then((props) => {
|
||||
console.debug('Server props:', props);
|
||||
model_name = props.model_name;
|
||||
})
|
||||
.catch((err) => {
|
||||
console.error(err);
|
||||
});
|
||||
StorageUtils.appendMsg(
|
||||
{
|
||||
id: currMsgId,
|
||||
@@ -383,11 +394,13 @@ export const AppContextProvider = ({
|
||||
convId,
|
||||
role: 'user',
|
||||
content,
|
||||
model_name: model_name,
|
||||
extra,
|
||||
parent: leafNodeId,
|
||||
children: [],
|
||||
},
|
||||
leafNodeId
|
||||
leafNodeId,
|
||||
model_name
|
||||
);
|
||||
onChunk(currMsgId);
|
||||
|
||||
@@ -415,9 +428,20 @@ export const AppContextProvider = ({
|
||||
) => {
|
||||
if (isGenerating(convId)) return;
|
||||
|
||||
if (content !== null) {
|
||||
if (content !== null) {
|
||||
const now = Date.now();
|
||||
const currMsgId = now;
|
||||
|
||||
let model_name:string='';
|
||||
await getServerProps(BASE_URL)
|
||||
.then((props) => {
|
||||
console.debug('Server props:', props);
|
||||
model_name = props.model_name;
|
||||
})
|
||||
.catch((err) => {
|
||||
console.error(err);
|
||||
});
|
||||
|
||||
StorageUtils.appendMsg(
|
||||
{
|
||||
id: currMsgId,
|
||||
@@ -426,11 +450,13 @@ export const AppContextProvider = ({
|
||||
convId,
|
||||
role: 'user',
|
||||
content,
|
||||
model_name:model_name,
|
||||
extra,
|
||||
parent: parentNodeId,
|
||||
children: [],
|
||||
},
|
||||
parentNodeId
|
||||
parentNodeId,
|
||||
model_name
|
||||
);
|
||||
parentNodeId = currMsgId;
|
||||
}
|
||||
@@ -452,9 +478,9 @@ export const AppContextProvider = ({
|
||||
messageIdToContinue
|
||||
);
|
||||
if (!existingMessage || existingMessage.role !== 'assistant') {
|
||||
console.error(
|
||||
'Cannot continue non-assistant message or message not found'
|
||||
);
|
||||
// console.error(
|
||||
// 'Cannot continue non-assistant message or message not found'
|
||||
// );
|
||||
toast.error(
|
||||
'Failed to continue message: Not an assistant message or not found.'
|
||||
);
|
||||
|
||||
@@ -4,7 +4,6 @@ import { APIMessage, Message, LlamaCppServerProps, APIMessageContentPart } from
|
||||
|
||||
// ponyfill for missing ReadableStream asyncIterator on Safari
|
||||
import { asyncIterator } from '@sec-ant/readable-stream/ponyfill/asyncIterator';
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
export const isString = (x: any) => !!x.toLowerCase;
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
@@ -177,19 +176,20 @@ export const getServerProps = async (
|
||||
apiKey?: string
|
||||
): Promise<LlamaCppServerProps> => {
|
||||
try {
|
||||
const response = await fetch(`${baseUrl}/props`, {
|
||||
const response = await fetch(`${baseUrl}/v1/props`, {
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...(apiKey ? { Authorization: `Bearer ${apiKey}` } : {}),
|
||||
},
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error('Failed to fetch server props');
|
||||
//throw new Error('Failed to fetch server props');
|
||||
}
|
||||
const data = await response.json();
|
||||
return data as LlamaCppServerProps;
|
||||
} catch (error) {
|
||||
console.error('Error fetching server props:', error);
|
||||
//console.error('Error fetching server props:', error);
|
||||
//toast.error('Error fetching server props:' +error);
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
@@ -1,11 +1,12 @@
|
||||
// coversations is stored in localStorage
|
||||
// format: { [convId]: { id: string, lastModified: number, messages: [...] } }
|
||||
|
||||
import { CONFIG_DEFAULT } from '../Config';
|
||||
//import { useState } from 'react';
|
||||
import {BASE_URL, CONFIG_DEFAULT } from '../Config';
|
||||
import { Conversation, Message, TimingReport, SettingsPreset } from './types';
|
||||
import Dexie, { Table } from 'dexie';
|
||||
import {getServerProps} from './misc'
|
||||
import { exportDB as exportDexieDB } from 'dexie-export-import';
|
||||
|
||||
import toast from 'react-hot-toast';
|
||||
const event = new EventTarget();
|
||||
|
||||
type CallbackConversationChanged = (convId: string) => void;
|
||||
@@ -31,6 +32,12 @@ db.version(1).stores({
|
||||
messages: '&id, convId, [convId+id], timestamp',
|
||||
});
|
||||
|
||||
db.version(2).stores({
|
||||
// Unlike SQL, you don’t need to specify all properties but only the one you wish to index.
|
||||
conversations: '&id, lastModified, model_name',
|
||||
messages: '&id, convId, [convId+id], timestamp',
|
||||
});
|
||||
|
||||
// convId is a string prefixed with 'conv-'
|
||||
const StorageUtils = {
|
||||
|
||||
@@ -118,11 +125,22 @@ const StorageUtils = {
|
||||
async createConversation(name: string): Promise<Conversation> {
|
||||
const now = Date.now();
|
||||
const msgId = now;
|
||||
let model_name:string = '';
|
||||
//window.alert(BASE_URL);
|
||||
await getServerProps(BASE_URL)
|
||||
.then((props) => {
|
||||
console.debug('Server props:', props);
|
||||
model_name = props.model_name;
|
||||
})
|
||||
.catch((err) => {
|
||||
console.error(err);
|
||||
});
|
||||
const conv: Conversation = {
|
||||
id: `conv-${now}`,
|
||||
lastModified: now,
|
||||
currNode: msgId,
|
||||
name,
|
||||
model_name:model_name,
|
||||
};
|
||||
await db.conversations.add(conv);
|
||||
// create a root node
|
||||
@@ -133,6 +151,7 @@ const StorageUtils = {
|
||||
timestamp: now,
|
||||
role: 'system',
|
||||
content: '',
|
||||
model_name:conv.model_name,
|
||||
parent: -1,
|
||||
children: [],
|
||||
});
|
||||
@@ -143,7 +162,8 @@ const StorageUtils = {
|
||||
*/
|
||||
async appendMsg(
|
||||
msg: Exclude<Message, 'parent' | 'children'>,
|
||||
parentNodeId: Message['id']
|
||||
parentNodeId: Message['id'],
|
||||
model_name:string,
|
||||
): Promise<void> {
|
||||
if (msg.content === null) return;
|
||||
const { convId } = msg;
|
||||
@@ -161,9 +181,11 @@ const StorageUtils = {
|
||||
`Parent message ID ${parentNodeId} does not exist in conversation ${convId}`
|
||||
);
|
||||
}
|
||||
model_name = model_name!==''?model_name:conv.model_name;
|
||||
await db.conversations.update(convId, {
|
||||
lastModified: Date.now(),
|
||||
currNode: msg.id,
|
||||
model_name: model_name,
|
||||
});
|
||||
// update parent
|
||||
await db.messages.update(parentNodeId, {
|
||||
@@ -191,10 +213,10 @@ const StorageUtils = {
|
||||
|
||||
// event listeners
|
||||
onConversationChanged(callback: CallbackConversationChanged) {
|
||||
const fn = (e: Event) => callback((e as CustomEvent).detail.convId);
|
||||
onConversationChangedHandlers.push([callback, fn]);
|
||||
event.addEventListener('conversationChange', fn);
|
||||
},
|
||||
const fn = (e: Event) => callback((e as CustomEvent).detail.convId);
|
||||
onConversationChangedHandlers.push([callback, fn]);
|
||||
event.addEventListener('conversationChange', fn);
|
||||
},
|
||||
offConversationChanged(callback: CallbackConversationChanged) {
|
||||
const fn = onConversationChangedHandlers.find(([cb, _]) => cb === callback);
|
||||
if (fn) {
|
||||
@@ -295,7 +317,6 @@ async importConversation(importedData: {
|
||||
|
||||
// Refresh the page to apply changes
|
||||
window.location.reload();
|
||||
|
||||
return conversation;
|
||||
},
|
||||
/**
|
||||
@@ -329,7 +350,7 @@ async importConversation(importedData: {
|
||||
const conversation = await StorageUtils.importConversation(jsonData);
|
||||
resolve(conversation);
|
||||
} catch (error) {
|
||||
console.error('Import failed:', error);
|
||||
toast.error('Import failed:' +error);
|
||||
alert(`Import failed: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
resolve(null);
|
||||
} finally {
|
||||
@@ -367,7 +388,8 @@ async importConversation(importedData: {
|
||||
try {
|
||||
return JSON.parse(presetsJson);
|
||||
} catch (e) {
|
||||
console.error('Failed to parse presets', e);
|
||||
toast.error('Failed to parse presets: '+ e);
|
||||
|
||||
return [];
|
||||
}
|
||||
},
|
||||
@@ -444,6 +466,7 @@ async function migrationLStoIDB() {
|
||||
lastModified,
|
||||
currNode: lastMsg.id,
|
||||
name,
|
||||
model_name:'migrate_name'
|
||||
});
|
||||
const rootId = messages[0].id - 2;
|
||||
await db.messages.add({
|
||||
@@ -454,6 +477,7 @@ async function migrationLStoIDB() {
|
||||
role: 'system',
|
||||
content: '',
|
||||
parent: -1,
|
||||
model_name:'migrate_name',
|
||||
children: [firstMsg.id],
|
||||
});
|
||||
for (let i = 0; i < messages.length; i++) {
|
||||
@@ -465,6 +489,7 @@ async function migrationLStoIDB() {
|
||||
timestamp: msg.id,
|
||||
parent: i === 0 ? rootId : messages[i - 1].id,
|
||||
children: i === messages.length - 1 ? [] : [messages[i + 1].id],
|
||||
model_name:'',
|
||||
});
|
||||
}
|
||||
migratedCount++;
|
||||
|
||||
@@ -3,6 +3,8 @@ export interface TimingReport {
|
||||
prompt_ms: number;
|
||||
predicted_n: number;
|
||||
predicted_ms: number;
|
||||
n_ctx: number;
|
||||
n_past: number;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -42,6 +44,7 @@ export interface Message {
|
||||
role: 'user' | 'assistant' | 'system';
|
||||
content: string;
|
||||
timings?: TimingReport;
|
||||
model_name:string;
|
||||
extra?: MessageExtra[];
|
||||
// node based system for branching
|
||||
parent: Message['id'];
|
||||
@@ -103,6 +106,7 @@ export interface Conversation {
|
||||
lastModified: number; // timestamp from Date.now()
|
||||
currNode: Message['id']; // the current message node being viewed
|
||||
name: string;
|
||||
model_name: string;
|
||||
}
|
||||
|
||||
export interface ViewingChat {
|
||||
@@ -136,6 +140,7 @@ export interface SettingsPreset {
|
||||
// a non-complete list of props, only contains the ones we need
|
||||
export interface LlamaCppServerProps {
|
||||
model_path: string;
|
||||
model_name: string;
|
||||
n_ctx: number;
|
||||
modalities?: {
|
||||
vision: boolean;
|
||||
|
||||
Reference in New Issue
Block a user