Files
ProxmoxVE-Local/scripts/core/api.func
CanbiZ (MickLesk) d2e8ac91d1 v1.0.0: Major UI/UX Overhaul, Performance & New Features (#566)
* Add GeneratorTab and UI refactor/styles

Introduce a full-featured GeneratorTab component for building and exporting script configuration (script selector, CPU/RAM/Disk sliders, advanced networking/features, generate/copy/export/import, and execute). Apply visual and structural UI refactors: new glass-card styles across ScriptCard, ScriptCardList, ScriptDetailModal, Terminal and Footer; update button variants and interaction classes; add Heart icon and extra footer links; update layout to use Manrope + JetBrains Mono fonts, sticky navbar, ambient backgrounds and metadata tweaks. Wire GeneratorTab into the main page and add related icon imports. Misc: small code-style/formatting adjustments and improved deriveScriptPath / handler signatures for clarity.

* Add script notes, server presets & APT proxy

Add persistent script notes and server presets, plus APT proxy settings and various UI/behavior improvements.

- DB: new migration adds script_notes and server_presets tables and indexes; Prisma schema updated with ScriptNote and ServerPreset models.
- Notes: new client component ScriptNotesPanel with CRUD using a new TRPC router (scriptNotes) and integration into ScriptDetailModal to show/manage private/shared notes.
- APT proxy: new Next.js API route /api/settings/apt-proxy that reads/writes .env vars; GeneralSettingsModal and ConfigurationModal load/save the proxy and pre-fill advanced install vars.
- UX/UI: add copyable install command, version badges on script cards/detail, container ID input in advanced configuration, and styling tweaks (glass-card-static).
- Server/script detection: installedScripts router now attempts to resolve the actual Proxmox node name via SSH and relaxes ID parsing to numeric-only.
- Notifications: appriseService extended to support Gotify endpoints and allow gotify:// scheme, and URL validation adjusted accordingly.

These changes enable user-maintained notes/presets, persistent APT-cacher defaults, improved install UX, better server detection, and additional notification backend support.

* Add server presets and silent batch updates

Introduce server presets and silent batch update support across API, server, and UI. Added a new serverPresets TRPC router and wired it into the API root. ConfigurationModal now lets users save/load/delete presets tied to a server. InstalledScriptsTab adds a "Silent update" option, a sequential "Update All Containers" batch flow that runs updates with PHS_SILENT=1, and passes envVars into update executions. ScriptExecutionHandler (server.js) now accepts envVars for updates and injects export commands into both local and SSH update flows so environment flags (e.g. PHS_SILENT) are honored. Also added "updated" sort handling in ScriptsGrid, DownloadedScriptsTab and FilterBar, and small typing fixes in appriseService.

* Use per-router Prisma client with Better SQLite3

Replace usage of the shared prisma import with per-router PrismaClient instances configured with @prisma/adapter-better-sqlite3. Add getNotesDb/getPresetsDb helpers (with DATABASE_URL fallback) and update scriptNotes and serverPresets to use the new client. Also add error handling in read endpoints to return empty arrays on DB failures and ensure create/update/delete use the new client.

* Icon-only action buttons; use presets DB

Convert HelpButton, ResyncButton, ServerSettingsButton and SettingsButton from labeled outline controls to compact ghost icon-only buttons with aria-labels and smaller icons/spinners for a more compact header UI. Simplify ResyncButton markup (smaller spinner/SVG, removed extra contextual text/lastSync layout) and tweak sync message styling. Add Server icon import in ServerSettingsButton. In serverPresets router, replace direct prisma calls with getPresetsDb() and use the returned db instance for all queries and mutations to support the presets DB connection.

* Add quick filters, Dev/ARM badges, and CSS tweaks

Introduce quick filter support and visual indicators for developer/ARM scripts. Added QuickFilter type and quickFilter to FilterState (getDefaultFilters/mergeFiltersWithDefaults) and a quick-filter UI in FilterBar (All, New, Updated, In Dev, ARM). Implemented filtering logic in ScriptsGrid and added category dev counts to surface per-category "dev" counts in CategorySidebar. Added DevBadge and ArmBadge components, used in ScriptCard and ScriptDetailModal; ScriptCard also gets a subtle violet highlight when script.is_dev is true. Fixed script path generation in GeneratorTab to include a `scripts/` prefix. Minor dark-theme adjustments to .glass-card/.glass-card-static background and border-color for improved contrast.

* Add appearance settings and persist UI prefs

Introduce an Appearance tab in the Settings modal to control theme, text size and layout width (default vs wide). Preferences are saved to localStorage and applied immediately via helper functions (applyTextSize/applyLayoutWidth) and a small inline script injected into <head> to apply saved settings on first paint. Add CSS utility classes for text-size variants and wire up theme buttons (using lucide icons). Also include minor cleanup: optional chaining fix in CategorySidebar and removal of an unused import in ResyncButton.

* Normalize quotes and format components

Apply consistent formatting across several components: convert single quotes to double quotes, reformat the QuickFilter union type and quick-filters array in FilterBar, adjust JSX prop and className string formatting in HelpButton, ScriptCard, ServerSettingsButton, and SettingsButton. These are cosmetic/formatting changes only and do not alter runtime behavior.

* Refactor Badge & CategorySidebar types/styles

Normalize formatting and strengthen typings for Badge; convert single quotes to double, expand variant/type unions, and tidy JSX/props and helper badge components. Rework CategorySidebar: refactor CategoryIcon SVGs for readability, standardize className usage, improve collapsed-state layout and buttons, and ensure categories are filtered/sorted by count with correct icon mapping and counts display. Overall cleanup improves consistency, readability, and type safety across these components.

* Add modal stacking and portal support

Introduce a modal stacking system and portal to ensure modals escape parent stacking contexts. ModalStackProvider now computes a zIndex for each registered modal and returns an unregister handle; useRegisterModal returns the zIndex. A ModalPortal component (createPortal to document.body) was added and all modal components were updated to: capture the returned zIndex, wrap their markup in <ModalPortal>, and apply the dynamic zIndex instead of a hardcoded z-50 class. This improves correct layering of multiple modals and avoids backdrop-filter / transform stacking issues.

* Wrap modals in portal and use zIndex

Use ModalPortal and zIndex values from useRegisterModal across modal components to avoid hardcoded z-50 stacking. Updated ExecutionModeModal, LXCSettingsModal, PublicKeyModal, ReleaseNotesModal, SetupModal, StorageSelectionModal, and TextViewer to import ModalPortal, capture the zIndex returned by useRegisterModal, wrap modal markup in <ModalPortal>, and apply style={{ zIndex }} to backdrops; LXC result overlay uses zIndex + 10 for proper stacking. Also changed LoadingOverlay in VersionDisplay to render with createPortal(document.body) and added the import. These changes centralize stacking behavior and prevent z-index conflicts when multiple modals/overlays are present.

* Use categoryIconColorMap for sidebar icons

Update CategorySidebar.tsx to derive non-selected icon color classes from categoryIconColorMap for both the "template" and per-category icons. Selected icons still use text-primary; fallbacks preserve previous muted and group-hover classes to retain existing styling when a mapping is absent.

* Standardize modal layout and form styles

Refactor modal markup across many components to use a consistent wrapper structure and unified Tailwind utility ordering (moved zIndex to outer container, standardized backdrop and centering). Clean up form layouts: align icons, labels and inputs, normalize spacing, error styling, and button variants (including icon/ghost adjustments). Improve AuthModal and CloneCountInputModal UX (loading/error states, closer placement, validation), and normalize BackupWarningModal and other modal presentations. Update ConfigurationModal advanced UI: reorganize network fields and add/handle extra options (IPv6 static, gateway, MTU, MAC, VLAN, DNS), refine presets save/cancel flow, and apply general formatting/whitespace cleanups.

* Escape env var values; update favicons & logo

Fix shell escaping when building env export commands by escaping backslashes and quotes (prevents broken exported values) in ScriptExecutionHandler (two locations). Update app metadata to point to favicon files under /favicon, add site manifest, and replace the header Package icon with a next/image using the android-chrome PNG (import Image and adjust markup) for consistent asset handling.

* Add favicon set and logo; remove old favicon

Replace legacy root favicon.png with a full favicon set under public/favicon (android-chrome 192/512, apple-touch-icon, favicon-16x16, favicon-32x32, favicon.ico, favicon.png) and add site.webmanifest. Also add public/logo.png. This organizes favicon assets for multiple platforms and adds a webmanifest for PWA/icon declarations.

* Portalize script dropdown and add outside click

Render the script selector dropdown into document.body via createPortal to escape stacking/overflow contexts. Add triggerRef/dropdownRef, compute fixed position from the trigger on open, and attach an outside-click handler to close the dropdown. Also import useRef/useEffect and createPortal; preserve existing search, selection, and reset behaviors while adjusting z-index/positioning.

* Update favicon and logo assets

Replace favicon and logo image files with updated versions to refresh branding. Updated files: public/favicon/android-chrome-192x192.png, public/favicon/android-chrome-512x512.png, public/favicon/apple-touch-icon.png, public/favicon/favicon-16x16.png, public/favicon/favicon-32x32.png, public/favicon/favicon.ico, public/logo.png.

* Add appearance modal & improve script detail UI

Add an AppearanceModal and AppearanceButton to let users change theme, text size and layout width (persisted in localStorage). Enhance FilterBar with category filtering (selectedCategory, dropdown UI, counts, and outside-click handling). Heavily refactor ScriptDetailModal: layout and visual refresh, new icons, better loading/update/delete/install flows, memoized install command, improved copy behavior, keyboard navigation between scripts (using orderedSlugs + onSelectSlug), clearer status/messages, and various performance/UX tweaks. Wire DownloadedScriptsTab to pass orderedSlugs and onSelectSlug into the ScriptDetailModal. Overall small UI/UX and state-management improvements across components.

* Optimize rendering, lazy-load tabs, add sync modal

Performance and UX improvements: memoize computed values in InstalledScriptsTab (scriptsWithStatus, filteredScripts, uniqueServers) and use useCallback/useMemo in ScriptsGrid to avoid unnecessary re-renders. Memoize ScriptCard and ScriptCardList components and tighten their ScriptCard type alias. Introduce a new SyncModal (ResyncButton) component to run/resync scripts with a progress UI and retry/close behavior, and replace the previous ResyncButton import in page.tsx. Lazy-load heavy tab components (Downloaded, Installed, Backups, Generator) with next/dynamic and add a TabSkeleton loader; consolidate tab definitions into a memoized tabs array. Misc: small prop/prop-name fixes and minor JSX formatting adjustments.

* Add Arcane script, status UI, validation & cache

Multiple changes improving UX, reliability, and CI permissions:

- Add scripts/tools/addon/arcane.sh: installer/update/uninstall helper for Arcane (Docker Compose), creates update helper and generates secrets.
- Add ServerStatusIndicator component and integrate into header (src/app/_components/ServerStatusIndicator.tsx, src/app/page.tsx); adjust hero/layout to accommodate inline version + status.
- Enhance GeneratorTab (src/app/_components/GeneratorTab.tsx): fetch full script details by slug, derive and apply default resource values from install_methods, and show an App Defaults info panel.
- Harden repo provider detection (src/server/lib/repositoryUrlValidation.js/.ts): use URL parsing for hostname matching with safe fallback to 'custom'.
- Tune client caching (src/trpc/query-client.ts): increase staleTime to 30min, set gcTime to 1h, and disable automatic refetches on mount/window focus/reconnect.
- Update GitHub Actions permissions: node.js.yml grants contents: read; release-drafter.yml grants contents: write and pull-requests: read.

These changes aim to provide better default resource handling for scripts, visible server reachability, more robust URL parsing, improved client-side cache behavior, and explicit workflow permissions.

* Add suppressHydrationWarning to <html>

Add the React prop suppressHydrationWarning to the <html> element in RootLayout (src/app/layout.tsx) to suppress hydration mismatch warnings. This helps avoid noisy console warnings when server-rendered markup and client rendering differ (e.g., due to dynamic font class injection or other runtime-only differences).

* Refactor GeneratorTab layout and add SSH imports

Rework GeneratorTab rendering and small UI/format fixes: tidy the scriptDetail useMemo formatting and restructure the Script Defaults block so defaults (CPU, RAM, HDD) are always shown and OS/version/variant badges render correctly. Minor formatting change to ServerStatusIndicator's fetch call and a small spacing fix on the home page title. Also add imports for getSSHService and the Server type to servers router in preparation for SSH-related functionality.

* Add explicit typing for servers in indicator

Annotate the `servers` variable with an explicit array type (id, name, ip, online) and cast `data?.servers` to that type, defaulting to an empty array. This clarifies TypeScript inference and prevents `servers` from being undefined for downstream logic without changing runtime behavior.

* perf/cleanup: audit fixes - hoist iconMap, pin Next.js, strip console.log, lighten glass-card, fix dev badge

* Refactor icons, remove debug logs, pin Next

Replace the large inline SVG icon map with a compact iconPaths mapping and fallback path, simplifying CategoryIcon rendering. Tighten the dev-count badge markup/styles for better layout. Remove leftover console.debug/log statements from InstalledScriptsTab and LXCSettingsModal. Pin Next.js version to 16.2.1 in package.json (deps and devDeps). Slightly adjust dark glass-card background opacity/hover values in globals.css.

* refactor: migrate server TS files to structured logger, accessibility improvements

* perf: fix 14s page load - remove SSH from initial query, defer tab-specific data

- getAllInstalledScripts: removed SSH batchDetectContainerTypes, use DB-only
  heuristic (lxc_config presence) for VM/LXC detection
- Defer installedScripts + backups queries until their tab is activated
- Initial batch now only fires 3 fast DB queries instead of 7 (incl. SSH)

* perf: split SSH queries from tRPC batch, skip save-on-mount

- Use splitLink to route servers.checkServersStatus through a separate
  non-batched httpLink. SSH queries no longer block fast DB queries from
  returning (was causing 14s+ batch response holding all data hostage).
- Add initialization refs in ScriptsGrid and DownloadedScriptsTab so
  save-filter/view-mode POST effects skip their first fire after load,
  eliminating 2-4 unnecessary network requests per tab mount.

* refactor: restore SSH detection for installed scripts, cache badge counts

- Restore batchDetectContainerTypes SSH calls in getAllInstalledScripts
  (shows real VM/LXC state, not just DB heuristic)
- Route getAllInstalledScripts through splitLink (non-batched) so SSH
  never blocks fast DB queries
- Cache installed/backups badge counts in localStorage so tabs show
  last-known counts instantly, updated when fresh data arrives
- Query still deferred until tab is visited (no eager SSH on page load)

* Refactor init checks, footer, and parallel fetch

Make several small refactors and updates across the app:

- Expand single-line early-return checks into multi-line blocks in DownloadedScriptsTab and ScriptsGrid for clarity when skipping initial effect triggers.
- Update Footer links: add separate GitHub (ProxmoxVE-Local) and GitHub (ProxmoxVE) buttons and change the site link to community-scripts.org.
- Minor formatting tweaks in page.tsx (useState initializer and backups ternary) for readability.
- In the scripts API router, fetch script cards and metadata in parallel via Promise.all to reduce latency and return both together.

These changes improve readability and slightly optimize the scripts fetch path.

* Add in-memory PB cache and invalidate on resync

Introduce a simple server-side in-memory cache for PocketBase data with a 10-minute TTL to reduce PB requests for rarely-changing data. Adds getCached/setCache helpers and a shared _cache store, and exports invalidatePbCache() to clear cached entries. Apply caching to getScriptCards, getCategories, and getScriptTypes, and call invalidatePbCache() in the resyncScripts mutation so fresh data is fetched after a resync.

* Update react.tsx

* Show script download flow & per-node server status

GeneratorTab: add detection of locally downloaded scripts (query + mutation), compute a slug set for O(1) lookup, and gate selection/execution on download state. Update script list UI to indicate downloadable items, grayscale/opacity non-downloaded entries, and show a download confirmation modal that triggers loadScript mutation and invalidates cache. Add Loader2 and AlertTriangle icons and disable Execute when the selected script isn't downloaded.

ServerStatusIndicator: change from a single aggregate dot to per-node indicators showing each host name and status (green online with ping animation, red offline). Improve handling of loading vs no-configured-servers states and update tooltip/title text for clarity.

* Improve updater script and minor UI tweaks

Refactor update.sh to make the updater more robust and flexible: add support for specifying a target release (get_release), include tar in dependency checks, strengthen download/extract logic and error logging, improve source-dir detection, and tighten backup/restore, install/build and service start/rollback flows. Also normalize indentation and logging output and add clearer diagnostic messages on failures. Minor UI cleanups in TSX: simplify GeneratorTab conditional formatting and adjust ServerStatusIndicator class ordering.

* Bump deps and add VSCode Next.js prompt setting

Update multiple dependencies to newer patch/minor versions (Prisma, @tanstack/react-query, axios, better-sqlite3, dotenv, lucide-react, next, react, react-dom, vite, vitest tooling, eslint, jsdom, postcss, prettier, prisma, etc.). Replace legacy typescript-eslint package with @typescript-eslint/eslint-plugin and @typescript-eslint/parser. Add .vscode/settings.json to mark WillLuke.nextjs.hasPrompted as true to suppress the Next.js prompt in VS Code.

* Normalize indentation in core scripts

Reformat scripts/core/alpine-install.func and scripts/core/api.func for consistent indentation and whitespace. Changes are formatting-only (no logic changes): converted leading tabs to spaces, aligned multiline blocks, and adjusted a comment reference (error-handler → error_handler) for consistency. Improves readability and maintainability without affecting behavior.

* Show/hide install command UI and service fixes

UI: Add a Terminal icon, show/hide toggle, and display block for the install command in ScriptDetailModal (new showCommand state and button).
Server: installedScripts - default unknown container types to LXC (safe default) and simplify VM/LXC detection comments.
AppriseService: support gotifys:// (HTTPS) and gotify:// formats, select protocol correctly, and update URL regex.
BackupService: include PBS namespace (--ns) when storage.namespace is configured so proxmox-backup-client commands include the namespace.
ScriptDownloader: add fallback to attempt downloading a ct/<slug>.sh when install_methods is empty for CT/LXC scripts, improve returned success/message semantics, better 404-aware error messaging, and check for fallback CT script presence when scanning downloaded files.
Other: add /examples to .gitignore and remove restore.log.
These changes improve robustness for missing install metadata, add HTTPS Gotify support, ensure PBS namespace usage, and improve UX for viewing install commands.

* Prefer local install scripts; add envVars & token

Allow using local copies of helper/install scripts during development and forward environment variables and a GitHub token through the UI and websocket layer.

Changes:
- scripts/core/alpine-install.func & scripts/core/install.func: try to load misc/tools.func from the local scripts directory before falling back to downloading from GitHub; improved error messaging on failure.
- scripts/core/build.func: resolve SCRIPT_DIR, prefer local alpine/install function files and export their contents to FUNCTIONS_FILE_PATH; fall back to remote download if local files are missing; use local install scripts for lxc-attach when available (fallback to remote). Adjusted dev MOTD path to source from /tmp.
- src/app/_components/ConfigurationModal.tsx: added a password input for var_github_token (passed as GITHUB_TOKEN to mitigate API rate limits).
- src/server/api/websocket/handler.ts: websocket handler now accepts an envVars object and forwards it into startScriptExecution so environment variables from the client can be supplied to script runs.

Overall this improves local development workflows and enables passing custom env vars and a GitHub token to scripts.

* Replace *_json with install_methods/notes

Adapt code to the updated PocketBase schema by renaming install_methods_json and notes_json to install_methods and notes across services and routers. Update parsing in pbScripts.ts, mappings in scripts.ts, autoSyncService.js, and localScripts.ts, and adjust scriptDownloader fallback comment/logic to reference the new field. Also add new PBScript metadata fields (github_data, deleted_message, disable_message, last_update_commit) so the PBScript type reflects the extended record shape.

* Add InstallCommandBlock and integrate installer UI

Introduce a full InstallCommandBlock React component (new file) that builds, previews, and copies install commands with support for GitHub/Gitea, Alpine/Advanced modes, ARM flag, resource presets, and dev gating. Integrate it into ScriptDetailModal: compute hasAlpine and installDefaults from script.install_methods, remove the old simple install command UI/handlers, and embed the new component for non-misc scripts. Update ScriptsGrid to allow aborting batch downloads, invalidate downloaded-scripts cache after downloads, and adjust orderedSlugs passed to the modal (prefer newest when no active filters). On the server, make ScriptDownloaderService resolve dev scripts to the ProxmoxVED repository (dev-specific repo path) while preserving explicit repository_url and default repo behavior.

* Fix envStr possibly undefined in InstallCommandBlock

* Redesign Install section: inline node picker, remove old modals, drop My Notes

* Remove bash command display, pass envVars to SSH, inline Terminal below Install

* Fix envVars: always pass mode to skip whiptail dialog; fix terminal clipping

* Remove dangerous source envVar, fix View button gating, clean dead onInstallScript prop chains

* Tidy JSX formatting and remove extra whitespace

Clean up minor JSX formatting and stray blank lines across components. Removed extra blank lines in DownloadedScriptsTab and ScriptsGrid modal props, reformatted the terminal container div in InstallCommandBlock for clearer attribute layout, and simplified conditional JSX in page.tsx to single-line renders. No functional changes intended—only stylistic/formatting adjustments to improve readability.

* refactor: comprehensive code quality improvements

- Move Terminal component into GeneratorTab (self-contained)
- Replace all alert()/confirm() with modal dialogs
- Type server prop as Server instead of any
- Fix deprecated onKeyPress -> onKeyDown
- Memoize stat counters + fix O(n^2) scriptCounts with Set lookups
- Remove dead hasActions function (always returned true)
- Decompose InstalledScriptsTab: extract Stats, Filters, StatusMessage
- Add buildServerFromScript helper (deduplicates 5 identical blocks)
- Fix eslint-disable blanket: targeted disables + void floating promises

* fix: resolve LXCSettingsModal rootfs_size TS error blocking build

- Type configData as Record<string,unknown> so TS union includes rootfs_size
- Include formatter fixes for extracted sub-components

* feat: bootstrap updater - self-updating update engine

Split update system into two files:
- update.sh: Thin bootstrap (~100 lines) that fetches the latest
  update-engine.sh from main branch before executing it
- update-engine.sh: Full update logic (moved from old update.sh)
- UPDATER_VERSION: Version tracker for the engine

How it works:
1. User runs 'bash update.sh' (unchanged workflow)
2. Bootstrap checks UPDATER_VERSION on main vs local
3. If different, downloads latest update-engine.sh from main
4. Hands off to update-engine.sh with all arguments

This ensures bugfixes to the update logic reach users automatically,
without requiring a manual intervention or a new app release.

* fix: widen terminal - collapse sidebar to single column when terminal active

- Add onTerminalChange callback to InstallCommandBlock
- ScriptDetailModal switches from 2-col grid to single column when terminal runs
- Sidebar (ACCESS/DETAILS/INSTALL PROFILES) moves below terminal
- Remove max-w-4xl cap from Terminal.tsx so it fills full container width
- Reset terminalActive state when navigating between scripts

* Release prep: bump to 1.0.0-pre1 + add updater

Update project to pre-release 1.0.0-pre1 and add an interactive updater.

- Bump VERSION and package.json version to 1.0.0-pre1.
- Add pre-release-updater.sh: interactive script to list GitHub prereleases, backup current install, download/extract a selected prerelease, install deps, run migrations, build, and restart the service.
- Minor JSX formatting cleanup in ScriptDetailModal.tsx (split long className into multiple lines for readability; no functional change).

* fix: correct INSTALL_DIR to /opt/ProxmoxVE-Local

* fix(modal): resolve z-index race in ModalStackProvider; refactor(installed-scripts): UI overhaul - stats cards, filters, toolbar, table

* feat(installed-scripts): add Restart/Reboot action; fix orphaned cleanup for missing-server records

* fix(cleanup): delete SSH records with server but no container_id (Pass 1.5); feat(backups): add Create Backup via vzdump in BackupsTab

* fix(generator): await refetch after download; add server selection for SSH execution

- Fix download race condition: await getAllDownloadedScripts.refetch() before
  closing dialog and selecting script so isScriptDownloaded returns true
- Add Server type import + state + useEffect to fetch /api/servers on mount
- Add server selection pill buttons above Execute (auto-selects single server)
- Execute button label changes to 'Execute on <name>' or 'Execute Locally'
- Pass mode + server to Terminal for correct local vs SSH execution
- Add selectedServer to handleExecute useCallback dependencies

* fix(generator): restore missing setTimeout body (syntax error)

* style: auto-format BackupsTab and GeneratorTab

* feat(pre2): ProxmoxVED toggle, floating shell dialog, settings UI polish

- Settings: remove Theme block (use AppearanceButton instead), restyle tabs
  to match main nav pill style
- Settings: add ProxmoxVED / Dev Scripts toggle (default: off); persisted in
  localStorage; StorageEvent keeps ScriptsGrid in sync without context
- ScriptsGrid: filter is_dev scripts from Newest carousel and full grid when
  ProxmoxVED is disabled; add showDevScripts -> FilterBar prop
- FilterBar: hide 'In Dev' quick-filter pill when ProxmoxVED is disabled
- ShellContext: new React context (open/minimize/restore/close) for shell sessions
- FloatingShell: new floating dialog with minimize-to-pill, maximize, close;
  persists across tab switches; renders via createPortal
- InstalledScriptsTab: delegate shell open to useShell() context instead of
  local inline terminal; remove inline shell JSX
- page.tsx: wrap Home in ShellProvider, render FloatingShell at page level

* feat: implement execute_in container picker for addon scripts

- Add execute_in?: string[] | null to Script type
- Forward execute_in from PocketBase via pbToScript() mapper
- Add listContainersOnServer tRPC query (pct list + qm list via SSH)
- GeneratorTab: show container/VM picker when script has execute_in flags
  - lxc, pbs, pmg -> show LXC picker (pbs/pmg pin matching containers)
  - vm -> show VM picker
  - Passes CTID env var to the script on execution
- Picker resets on server or script change

* fix(generator): move container-reset useEffect after selectedSlug declaration

* feat: add prerelease update channel toggle + bump version to 1.0.0-pre2

- Add ALLOW_PRERELEASE env var to env.js
- Add /api/settings/prerelease GET/POST route (persists to .env)
- version.ts: getVersionStatus + getLatestRelease use /releases (all) when
  ALLOW_PRERELEASE=true, otherwise /releases/latest (stable only)
- GitHubRelease interface: add prerelease boolean field
- GeneralSettingsModal: add Update Channel toggle in General tab
- VERSION: bump to 1.0.0-pre2
- page.tsx: destructure isAuthenticated from useAuth()
- GeneratorTab.tsx: move container-reset useEffect after selectedSlug decl

* fix(page): destructure logout from useAuth

* feat: floating terminal for backup + New Backup dialog + fixes

- FloatingShell: support backup task mode (isBackup Terminal when
  session.backupStorage is set); adds HardDrive icon, onComplete callback
- ShellContext: extend ShellSession with backupStorage, title, onComplete
- BackupsTab: replace blocking createBackupMutation with FloatingShell backup;
  add 'New Backup' dialog (server -> container -> storage selection)
- InstalledScriptsTab: fix fetchStorages to not show blocking error modal
- VERSION: bump to 1.0.0-pre3, fixed UTF-8 BOM encoding

* fix: keep Terminal mounted on minimize, add drag-to-move support

- FloatingShell is now a single always-mounted window instead of
  conditional branches — visibility:hidden on minimize keeps the
  xterm.js WebSocket alive so 'starting shell session' no longer
  appears on restore
- Header acts as drag handle: mouse-drag repositions the floating
  window freely so the background remains usable
- Maximize still covers full viewport; restore returns to last
  dragged position (or CSS-centered if never dragged)

* fix: Server.host -> Server.ip in BackupsTab new backup dialog

* fix: use lxc/vm fields from listContainersOnServer (no containers/vmid)

* fix: bundle core.func+error-handler.func into FUNCTIONS_FILE_PATH

* fix: typed container list in New Backup dialog (no unsafe any)

* feat: v1.0.0-pre4 - multi-session shell, multi-LXC backup, auto-discover after backup, local generator command, addon execute_in

* feat: add Refresh button to Downloaded Scripts tab

* fix: add executeInContainer to WebSocketMessage typedef

* fix: installationId typedef + verbose logo error logging

* fix: add JSDoc type annotations to implicit any callback params

* fix: guard against undefined results[j] in logo cache loop

* Refactor Backups modal and minor UI cleanups

Refactor the Backups create dialog for clarity and better state handling: reorganized JSX, normalized formatting, extract typed container lists, and ensure closing the dialog clears selected storage. Adjust server/container/storage selection flows and button behaviors to improve readability and UX. Also apply small readability/formatting fixes in DownloadedScriptsTab (expand onClick), FloatingShell (simplify Math.min expression), and GeneratorTab (wrap ternary for containerId). These are mostly non-functional cleanup and UI clarity changes across the mentioned components.

* feat: execute_in container routing + install UI overhaul

- GeneratorTab: remove && !!selectedServer from execInContainer so addon
  scripts work in local mode (pct exec works without SSH)
- InstallCommandBlock: add executeIn prop + container picker via tRPC
  listContainersOnServer; pass executeInContainer/containerId/containerType
  to Terminal so addon scripts run inside the container, not on the host
- InstallCommandBlock: remove GitHub/Gitea source toggle; promote
  My Defaults + App Defaults to top-level tabs alongside Default/Alpine/Advanced
- ScriptDetailModal: pass execute_in to InstallCommandBlock; add Runs In
  badges to Details panel

* fix: GitHub PAT not applied at runtime + light mode contrast

- route.ts: also set process.env.GITHUB_TOKEN in memory after writing to
  .env file - tokens saved via UI now take effect immediately without restart
- github.ts: read process.env.GITHUB_TOKEN directly instead of the frozen
  t3-env snapshot (env.GITHUB_TOKEN is captured once at startup)
- github.ts: switch Authorization to 'Bearer' scheme (works for both classic
  ghp_ and fine-grained github_pat_ tokens per GitHub docs)
- GeneralSettingsModal: fix text-success-foreground / text-error-foreground
  -> text-success / text-error so messages are readable in light mode
  (success-foreground is white, which is invisible on bg-success/10)
- GeneralSettingsModal: update PAT description to mention both token types

* fix: unblock pre-release updater TypeScript build + cut pre5

- InstallCommandBlock: fix trpc input type for listContainersOnServer
  (serverId now numeric fallback instead of string)
- InstallCommandBlock: align container list mapping to API response shape
  (use id/name instead of vmid)
- InstallCommandBlock: narrow running.containerType type to 'lxc' | 'vm'
- VERSION: bump to 1.0.0-pre5
- package.json: bump app version to 1.0.0-pre5 (installer/build logs)

This resolves the pre4 updater build failures reported in PR comments.

* feat: backup dialog UX + generator local-node workflow fixes

Backups:
- render create-backup dialog in portal with full-screen backdrop
- fix multi-select flow: explicit steps (server -> containers -> storage)
- keep container selection until user clicks Continue
- show storage free capacity (GB) from pvesm status
- show estimated max backup size from selected container disk templates

Generator:
- remove 'This machine (local)' execution option
- require node selection; execute over SSH only
- generated command now always uses local script path (scripts/...)
  instead of remote curl command
- add install mode tabs: Default / My Defaults / App Defaults / Advanced
- apply mode to command/envVars (mode=default|mydefaults|appdefaults|generated)
- use selected container resources as template defaults (cpu/ram/disk)

API:
- getBackupStorages now enriches storages with available/used/total GB
- add getContainersResourceTemplates for per-container cpu/ram/disk templates

* Add advanced options to generator tab

Expose many new advanced/container, network and feature settings in GeneratorTab: password, tags, timezone, container/template storage, protection, IPv6 method/ip/gateway, search domain, nameserver, TUN, keyctl, mknod, verbose, APT cacher (with IP), mount filesystems and SSH authorized key. Wire these into command overrides, env-vars generation, export/import and reset logic. UI updated with grouped sections, IPv6 selector buttons, new inputs and toggles, and FieldInput now supports input type and hint. Also minor refactor to templateDefaults lookup formatting.

* Format JSX in BackupsTab and GeneralSettingsModal

Apply code style/formatting updates across BackupsTab.tsx and GeneralSettingsModal.tsx: reflow JSX props and elements, add consistent line breaks/commas and minor spacing adjustments (including a spacing fix in the GitHub token text). No logic or behavioral changes.

* feat: filter addon/pve types from sync, display, and script loading

- Add UNSUPPORTED_TYPES constant ['addon', 'pve'] in scripts router
- Filter these types from getScriptCards, getScriptCardsWithCategories
- Block loadScript and loadMultipleScripts for unsupported types
- Remove addon/pve filter options from FilterBar UI
- Remove unused Wrench/Server icon imports from FilterBar

* chore: bump version to 1.0.0-pre6

* fix: add pbs_username to hand-written PBSStorageCredential type

* chore: update dependencies and regenerate lockfile

* feat: hide dev count badge in category sidebar when dev mode is off

* feat: add DHCP/STATIC IPv4 mode pills to generator, show IP/gateway fields only when static

* Fix CategorySidebar JSX and format GeneratorTab

Close a mismatched <span> in CategorySidebar so the inner "dev" badge is properly nested and rendered. Also reformat several single-line conditionals and adjust className ordering/formatting in GeneratorTab for improved readability; these are stylistic changes with no functional behavior changes.

* fix: keep floating shell windows mounted when minimized to preserve session state

Instead of unmounting FloatingShellWindow (which destroyed xterm + WebSocket),
all session windows are now kept in the DOM and hidden via CSS (display: none)
when minimized. This preserves the terminal session, history, and WebSocket
connection across minimize/restore cycles.

* feat: add floating shell access to Scripts, Downloaded, and Detail modal

Shell buttons now appear everywhere an installed container is recognized:

- ScriptDetailModal sidebar: 'Containers' section shows each matching
  installed container with a direct Shell button (covers Scripts +
  Downloaded tabs via the modal)
- ScriptsGrid / ScriptCard: shell button appears on cards (card and list
  view) for scripts that have an installed container
- DownloadedScriptsTab: same shell button on cards and list rows
- ScriptCardList: shell button in the header row alongside website link

Matching is done by normalizing the installed script_name against the
card slug (lowercase, strip extension, replace non-alphanumeric with dash).
Only containers with a valid container_id and non-failed status are shown.

* fix: restore downloaded addon/pve scripts visibility in Downloaded tab

The UNSUPPORTED_TYPES filter was applied at the router level in
getScriptCardsWithCategories and getScriptCards. Since DownloadedScriptsTab
builds its list by cross-referencing local files against the GitHub cards
from this endpoint, addon/pve scripts that were already downloaded locally
were invisibly dropped.

Fix: remove the filter from the router, apply it client-side in
ScriptsGrid's combinedScripts memo instead. This means:
- Scripts grid: still hides addon/pve (can't install them)
- Downloaded tab: shows ALL locally downloaded scripts regardless of type
- loadScript / loadMultipleScripts: still block new addon/pve downloads

* Format imports and object indentation

Reformat code in DownloadedScriptsTab.tsx and ScriptDetailModal.tsx for readability: split the React import across multiple lines, adjust placement/indentation of eslint-disable comments, and reflow object property lines in ScriptDetailModal. These are stylistic changes only and do not modify runtime behavior.

* feat: v1.0.0-pre7 - updater channel fix, configurable detection tag, static IP guard

Fixes:
- fix: updater no longer downgrades prerelease installs to stable
  version.ts executeUpdate now uses the v1.0.0 branch when VERSION
  contains 'pre', so prerelease users always get the prerelease update.sh
- fix: static IP guard in build.func - if NET is the literal word 'static'
  (no CIDR entered), fall back to dhcp with a warning instead of creating
  a container with ip=static
- fix: ConfigurationModal no longer sends var_net=static to the script
  when no IP address was entered in the static IP field

Features:
- feat: configurable container detection tag (Settings > General)
  Users can override 'community-script' with any tag (e.g. 'cs')
  Stored in .env as CONTAINER_DETECTION_TAG, used by autoDetectLXCContainers

Version:
- bump VERSION + package.json to 1.0.0-pre7

* fix: preserve downloaded scripts across updates

- pre-release updater now backs up/restores scripts ct/tools/vm/vw
- prevent rsync from overwriting downloaded script directories
- include scripts/vw in update-engine backup/restore/rollback/excludes

This prevents downloaded scripts from disappearing after update and
explains why only bundled defaults (e.g. debian + arcane) remained.

* feat(ui): unify floating terminals and enable addon execution

* chore: bump version to 1.0.0-pre8

* fix(updater): skip engine self-update gracefully when UPDATER_VERSION not on main (pre-release)

* fix: SSH interactive input, suppress rsync output, 4-theme picker, sort containers by ID

* fix(ui): overhaul terminal themes with strong contrast and real preview cards

* feat(pre9): execute_in policy matrix, full type visibility, lxc-only generator

* fix(pre9): make VM/LXC status and type detection server-scoped

---------

Co-authored-by: MickLesk <mickey.leskowitz@levelbuild.com>
2026-05-05 15:26:13 +02:00

1490 lines
53 KiB
Plaintext

# Copyright (c) 2021-2026 community-scripts ORG
# Author: michelroegl-brunner | MickLesk
# License: MIT | https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/LICENSE
# ==============================================================================
# API.FUNC - TELEMETRY & DIAGNOSTICS API
# ==============================================================================
#
# Provides functions for sending anonymous telemetry data via the community
# telemetry ingest service at telemetry.community-scripts.org.
#
# Features:
# - Container/VM creation statistics
# - Installation success/failure tracking
# - Error code mapping and reporting
# - Privacy-respecting anonymous telemetry
#
# Usage:
# source <(curl -fsSL .../api.func)
# post_to_api # Report LXC container creation
# post_to_api_vm # Report VM creation
# post_update_to_api # Report installation status
#
# Privacy:
# - Only anonymous statistics (no personal data)
# - User can opt-out via DIAGNOSTICS=no
# - Random UUID for session tracking only
# - Data retention: 30 days
#
# ==============================================================================
# ==============================================================================
# Telemetry Configuration
# ==============================================================================
TELEMETRY_URL="https://telemetry.community-scripts.org/telemetry"
# Timeout for telemetry requests (seconds)
# Progress pings (validation/configuring) use the short timeout
TELEMETRY_TIMEOUT=5
# Final status updates (success/failed) use the longer timeout
# PocketBase may need more time under load (FindRecord + UpdateRecord)
STATUS_TIMEOUT=10
# ==============================================================================
# SECTION 0: REPOSITORY SOURCE DETECTION
# ==============================================================================
# ------------------------------------------------------------------------------
# detect_repo_source()
#
# - Dynamically detects which GitHub/Gitea repo the scripts were loaded from
# - Inspects /proc/$$/cmdline and $0 to find the source URL
# - Maps detected repo to one of three canonical values:
# * "ProxmoxVE" — official community-scripts/ProxmoxVE (production)
# * "ProxmoxVED" — official community-scripts/ProxmoxVED (development)
# * "external" — any fork or unknown source
# - Fallback: "ProxmoxVED" (CI sed transforms ProxmoxVED → ProxmoxVE on promotion)
# - Sets and exports REPO_SOURCE global variable
# - Skips detection if REPO_SOURCE is already set (e.g., by environment)
# ------------------------------------------------------------------------------
detect_repo_source() {
# Allow explicit override via environment
[[ -n "${REPO_SOURCE:-}" ]] && return 0
local content="" owner_repo=""
# Method 1: Read from /proc/$$/cmdline
# When invoked via: bash -c "$(curl -fsSL https://.../ct/app.sh)"
# the full CT/VM script content is in /proc/$$/cmdline (same PID through source chain)
if [[ -r /proc/$$/cmdline ]]; then
content=$(tr '\0' ' ' </proc/$$/cmdline 2>/dev/null) || true
fi
# Method 2: Read from the original script file (bash ct/app.sh / bash vm/app.sh)
if [[ -z "$content" ]] || ! echo "$content" | grep -qE 'githubusercontent\.com|community-scripts\.org' 2>/dev/null; then
if [[ -f "$0" ]] && [[ "$0" != *bash* ]]; then
content=$(head -10 "$0" 2>/dev/null) || true
fi
fi
# Extract owner/repo from URL patterns found in the script content
if [[ -n "$content" ]]; then
# GitHub raw URL: raw.githubusercontent.com/OWNER/REPO/...
owner_repo=$(echo "$content" | grep -oE 'raw\.githubusercontent\.com/[^/]+/[^/]+' | head -1 | sed 's|raw\.githubusercontent\.com/||') || true
# Gitea URL: git.community-scripts.org/OWNER/REPO/...
if [[ -z "$owner_repo" ]]; then
owner_repo=$(echo "$content" | grep -oE 'git\.community-scripts\.org/[^/]+/[^/]+' | head -1 | sed 's|git\.community-scripts\.org/||') || true
fi
fi
# Map detected owner/repo to canonical repo_source value
case "$owner_repo" in
community-scripts/ProxmoxVE) REPO_SOURCE="ProxmoxVE" ;;
community-scripts/ProxmoxVED) REPO_SOURCE="ProxmoxVED" ;;
"")
# No URL detected — use hardcoded fallback
# This value must match the repo: ProxmoxVE for production, ProxmoxVED for dev
REPO_SOURCE="ProxmoxVE"
;;
*)
# Fork or unknown repo
REPO_SOURCE="external"
;;
esac
export REPO_SOURCE
}
# Run detection immediately when api.func is sourced
detect_repo_source
# ==============================================================================
# SECTION 1: ERROR CODE DESCRIPTIONS
# ==============================================================================
# ------------------------------------------------------------------------------
# explain_exit_code()
#
# - Maps numeric exit codes to human-readable error descriptions
# - Canonical source of truth for ALL exit code mappings
# - Used by both api.func (telemetry) and error_handler.func (error display)
# - Supports:
# * Generic/Shell errors (1-3, 10, 124-132, 134, 137, 139, 141, 143-146)
# * curl/wget errors (4-8, 16, 18, 22-28, 30, 32-36, 39, 44-48, 51-52, 55-57, 59, 61, 63, 75, 78-79, 92, 95)
# * Package manager errors (APT, DPKG: 100-102, 255)
# * Script Validation & Setup (103-123)
# * BSD sysexits (64-78)
# * Systemd/Service errors (150-154)
# * Python/pip/uv errors (160-162)
# * PostgreSQL errors (170-173)
# * MySQL/MariaDB errors (180-183)
# * MongoDB errors (190-193)
# * Proxmox custom codes (200-231)
# * Tools & Addon Scripts (232-238)
# * Node.js/npm errors (239, 243, 245-249)
# * Application Install/Update errors (250-254)
# - Returns description string for given exit code
# ------------------------------------------------------------------------------
explain_exit_code() {
local code="$1"
case "$code" in
# --- Generic / Shell ---
1) echo "General error / Operation not permitted" ;;
2) echo "Misuse of shell builtins (e.g. syntax error)" ;;
3) echo "General syntax or argument error" ;;
10) echo "Docker / privileged mode required (unsupported environment)" ;;
# --- curl / wget errors (commonly seen in downloads) ---
4) echo "curl: Feature not supported or protocol error" ;;
5) echo "curl: Could not resolve proxy" ;;
6) echo "curl: DNS resolution failed (could not resolve host)" ;;
7) echo "curl: Failed to connect (network unreachable / host down)" ;;
8) echo "curl: Server reply error (FTP/SFTP or apk untrusted key)" ;;
16) echo "curl: HTTP/2 framing layer error" ;;
18) echo "curl: Partial file (transfer not completed)" ;;
22) echo "curl: HTTP error returned (404, 429, 500+)" ;;
23) echo "curl: Write error (disk full or permissions)" ;;
24) echo "curl: Write to local file failed" ;;
25) echo "curl: Upload failed" ;;
26) echo "curl: Read error on local file (I/O)" ;;
27) echo "curl: Out of memory (memory allocation failed)" ;;
28) echo "curl: Operation timeout (network slow or server not responding)" ;;
30) echo "curl: FTP port command failed" ;;
32) echo "curl: FTP SIZE command failed" ;;
33) echo "curl: HTTP range error" ;;
34) echo "curl: HTTP post error" ;;
35) echo "curl: SSL/TLS handshake failed (certificate error)" ;;
36) echo "curl: FTP bad download resume" ;;
39) echo "curl: LDAP search failed" ;;
44) echo "curl: Internal error (bad function call order)" ;;
45) echo "curl: Interface error (failed to bind to specified interface)" ;;
46) echo "curl: Bad password entered" ;;
47) echo "curl: Too many redirects" ;;
48) echo "curl: Unknown command line option specified" ;;
51) echo "curl: SSL peer certificate or SSH host key verification failed" ;;
52) echo "curl: Empty reply from server (got nothing)" ;;
55) echo "curl: Failed sending network data" ;;
56) echo "curl: Receive error (connection reset by peer)" ;;
57) echo "curl: Unrecoverable poll/select error (system I/O failure)" ;;
59) echo "curl: Couldn't use specified SSL cipher" ;;
61) echo "curl: Bad/unrecognized transfer encoding" ;;
63) echo "curl: Maximum file size exceeded" ;;
75) echo "Temporary failure (retry later)" ;;
78) echo "curl: Remote file not found (404 on FTP/file)" ;;
79) echo "curl: SSH session error (key exchange/auth failed)" ;;
92) echo "curl: HTTP/2 stream error (protocol violation)" ;;
95) echo "curl: HTTP/3 layer error" ;;
# --- Package manager / APT / DPKG ---
100) echo "APT: Package manager error (broken packages / dependency problems)" ;;
101) echo "APT: Configuration error (bad sources.list, malformed config)" ;;
102) echo "APT: Lock held by another process (dpkg/apt still running)" ;;
# --- Script Validation & Setup (103-123) ---
103) echo "Validation: Shell is not Bash" ;;
104) echo "Validation: Not running as root (or invoked via sudo)" ;;
105) echo "Validation: Proxmox VE version not supported" ;;
106) echo "Validation: Architecture not supported (ARM / PiMox)" ;;
107) echo "Validation: Kernel key parameters unreadable" ;;
108) echo "Validation: Kernel key limits exceeded" ;;
109) echo "Proxmox: No available container ID after max attempts" ;;
110) echo "Proxmox: Failed to apply default.vars" ;;
111) echo "Proxmox: App defaults file not available" ;;
112) echo "Proxmox: Invalid install menu option" ;;
113) echo "LXC: Under-provisioned — user aborted update" ;;
114) echo "LXC: Storage too low — user aborted update" ;;
115) echo "Download: install.func download failed or incomplete" ;;
116) echo "Proxmox: Default bridge vmbr0 not found" ;;
117) echo "LXC: Container did not reach running state" ;;
118) echo "LXC: No IP assigned to container after timeout" ;;
119) echo "Proxmox: No valid storage for rootdir content" ;;
120) echo "Proxmox: No valid storage for vztmpl content" ;;
121) echo "LXC: Container network not ready (no IP after retries)" ;;
122) echo "LXC: No internet connectivity — user declined to continue" ;;
123) echo "LXC: Local IP detection failed" ;;
# --- BSD sysexits.h (64-78) ---
64) echo "Usage error (wrong arguments)" ;;
65) echo "Data format error (bad input data)" ;;
66) echo "Input file not found (cannot open input)" ;;
67) echo "User not found (addressee unknown)" ;;
68) echo "Host not found (hostname unknown)" ;;
69) echo "Service unavailable" ;;
70) echo "Internal software error" ;;
71) echo "System error (OS-level failure)" ;;
72) echo "Critical OS file missing" ;;
73) echo "Cannot create output file" ;;
74) echo "I/O error" ;;
76) echo "Remote protocol error" ;;
77) echo "Permission denied" ;;
# --- Common shell/system errors ---
124) echo "Command timed out (timeout command)" ;;
125) echo "Command failed to start (Docker daemon or execution error)" ;;
126) echo "Command invoked cannot execute (permission problem?)" ;;
127) echo "Command not found" ;;
128) echo "Invalid argument to exit" ;;
129) echo "Killed by SIGHUP (terminal closed / hangup)" ;;
130) echo "Aborted by user (SIGINT)" ;;
131) echo "Killed by SIGQUIT (core dumped)" ;;
132) echo "Killed by SIGILL (illegal CPU instruction)" ;;
134) echo "Process aborted (SIGABRT - possibly Node.js heap overflow)" ;;
137) echo "Killed (SIGKILL / Out of memory?)" ;;
139) echo "Segmentation fault (core dumped)" ;;
141) echo "Broken pipe (SIGPIPE - output closed prematurely)" ;;
143) echo "Terminated (SIGTERM)" ;;
144) echo "Killed by signal 16 (SIGUSR1 / SIGSTKFLT)" ;;
146) echo "Killed by signal 18 (SIGTSTP)" ;;
# --- Systemd / Service errors (150-154) ---
150) echo "Systemd: Service failed to start" ;;
151) echo "Systemd: Service unit not found" ;;
152) echo "Permission denied (EACCES)" ;;
153) echo "Build/compile failed (make/gcc/cmake)" ;;
154) echo "Node.js: Native addon build failed (node-gyp)" ;;
# --- Python / pip / uv (160-162) ---
160) echo "Python: Virtualenv / uv environment missing or broken" ;;
161) echo "Python: Dependency resolution failed" ;;
162) echo "Python: Installation aborted (permissions or EXTERNALLY-MANAGED)" ;;
# --- PostgreSQL (170-173) ---
170) echo "PostgreSQL: Connection failed (server not running / wrong socket)" ;;
171) echo "PostgreSQL: Authentication failed (bad user/password)" ;;
172) echo "PostgreSQL: Database does not exist" ;;
173) echo "PostgreSQL: Fatal error in query / syntax" ;;
# --- MySQL / MariaDB (180-183) ---
180) echo "MySQL/MariaDB: Connection failed (server not running / wrong socket)" ;;
181) echo "MySQL/MariaDB: Authentication failed (bad user/password)" ;;
182) echo "MySQL/MariaDB: Database does not exist" ;;
183) echo "MySQL/MariaDB: Fatal error in query / syntax" ;;
# --- MongoDB (190-193) ---
190) echo "MongoDB: Connection failed (server not running)" ;;
191) echo "MongoDB: Authentication failed (bad user/password)" ;;
192) echo "MongoDB: Database not found" ;;
193) echo "MongoDB: Fatal query error" ;;
# --- Proxmox Custom Codes (200-231) ---
200) echo "Proxmox: Failed to create lock file" ;;
203) echo "Proxmox: Missing CTID variable" ;;
204) echo "Proxmox: Missing PCT_OSTYPE variable" ;;
205) echo "Proxmox: Invalid CTID (<100)" ;;
206) echo "Proxmox: CTID already in use" ;;
207) echo "Proxmox: Password contains unescaped special characters" ;;
208) echo "Proxmox: Invalid configuration (DNS/MAC/Network format)" ;;
209) echo "Proxmox: Container creation failed" ;;
210) echo "Proxmox: Cluster not quorate" ;;
211) echo "Proxmox: Timeout waiting for template lock" ;;
212) echo "Proxmox: Storage type 'iscsidirect' does not support containers (VMs only)" ;;
213) echo "Proxmox: Storage type does not support 'rootdir' content" ;;
214) echo "Proxmox: Not enough storage space" ;;
215) echo "Proxmox: Container created but not listed (ghost state)" ;;
216) echo "Proxmox: RootFS entry missing in config" ;;
217) echo "Proxmox: Storage not accessible" ;;
218) echo "Proxmox: Template file corrupted or incomplete" ;;
219) echo "Proxmox: CephFS does not support containers - use RBD" ;;
220) echo "Proxmox: Unable to resolve template path" ;;
221) echo "Proxmox: Template file not readable" ;;
222) echo "Proxmox: Template download failed" ;;
223) echo "Proxmox: Template not available after download" ;;
224) echo "Proxmox: PBS storage is for backups only" ;;
225) echo "Proxmox: No template available for OS/Version" ;;
226) echo "Proxmox: VM disk import or post-creation setup failed" ;;
231) echo "Proxmox: LXC stack upgrade failed" ;;
# --- Tools & Addon Scripts (232-238) ---
232) echo "Tools: Wrong execution environment (run on PVE host, not inside LXC)" ;;
233) echo "Tools: Application not installed (update prerequisite missing)" ;;
234) echo "Tools: No LXC containers found or available" ;;
235) echo "Tools: Backup or restore operation failed" ;;
236) echo "Tools: Required hardware not detected" ;;
237) echo "Tools: Dependency package installation failed" ;;
238) echo "Tools: OS or distribution not supported for this addon" ;;
# --- Node.js / npm / pnpm / yarn (239-249) ---
239) echo "npm/Node.js: Unexpected runtime error or dependency failure" ;;
243) echo "Node.js: Out of memory (JavaScript heap out of memory)" ;;
245) echo "Node.js: Invalid command-line option" ;;
246) echo "Node.js: Internal JavaScript Parse Error" ;;
247) echo "Node.js: Fatal internal error" ;;
248) echo "Node.js: Invalid C++ addon / N-API failure" ;;
249) echo "npm/pnpm/yarn: Unknown fatal error" ;;
# --- Application Install/Update Errors (250-254) ---
250) echo "App: Download failed or version not determined" ;;
251) echo "App: File extraction failed (corrupt or incomplete archive)" ;;
252) echo "App: Required file or resource not found" ;;
253) echo "App: Data migration required — update aborted" ;;
254) echo "App: User declined prompt or input timed out" ;;
# --- DPKG ---
255) echo "DPKG: Fatal internal error" ;;
# --- Default ---
*) echo "Unknown error" ;;
esac
}
# ------------------------------------------------------------------------------
# json_escape()
#
# - Escapes a string for safe JSON embedding
# - Strips ANSI escape sequences and non-printable control characters
# - Handles backslashes, quotes, newlines, tabs, and carriage returns
# ------------------------------------------------------------------------------
json_escape() {
# Escape a string for safe JSON embedding using awk (handles any input size).
# Pipeline: strip ANSI → remove control chars → escape \ " TAB → join lines with \n
printf '%s' "$1" |
sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' |
tr -d '\000-\010\013\014\016-\037\177\r' |
awk '
BEGIN { ORS = "" }
{
gsub(/\\/, "\\\\") # backslash → \\
gsub(/"/, "\\\"") # double quote → \"
gsub(/\t/, "\\t") # tab → \t
if (NR > 1) printf "\\n"
printf "%s", $0
}'
}
# ------------------------------------------------------------------------------
# get_error_text()
#
# - Returns last 20 lines of the active log (INSTALL_LOG or BUILD_LOG)
# - Falls back to combined log or BUILD_LOG if primary is not accessible
# - Handles container paths that don't exist on the host
# ------------------------------------------------------------------------------
get_error_text() {
local logfile=""
if declare -f get_active_logfile >/dev/null 2>&1; then
logfile=$(get_active_logfile)
elif [[ -n "${INSTALL_LOG:-}" ]]; then
logfile="$INSTALL_LOG"
elif [[ -n "${BUILD_LOG:-}" ]]; then
logfile="$BUILD_LOG"
fi
# If logfile is inside container (e.g. /root/.install-*), try the host copy
if [[ -n "$logfile" && ! -s "$logfile" ]]; then
# Try combined log: /tmp/<app>-<CTID>-<SESSION_ID>.log
if [[ -n "${CTID:-}" && -n "${SESSION_ID:-}" ]]; then
local combined_log="/tmp/${NSAPP:-lxc}-${CTID}-${SESSION_ID}.log"
if [[ -s "$combined_log" ]]; then
logfile="$combined_log"
fi
fi
fi
# Also try BUILD_LOG as fallback if primary log is empty/missing
if [[ -z "$logfile" || ! -s "$logfile" ]] && [[ -n "${BUILD_LOG:-}" && -s "${BUILD_LOG}" ]]; then
logfile="$BUILD_LOG"
fi
# Try SILENT_LOGFILE as last resort (captures $STD command output)
if [[ -z "$logfile" || ! -s "$logfile" ]] && [[ -n "${SILENT_LOGFILE:-}" && -s "${SILENT_LOGFILE}" ]]; then
logfile="$SILENT_LOGFILE"
fi
if [[ -n "$logfile" && -s "$logfile" ]]; then
tail -n 20 "$logfile" 2>/dev/null | sed 's/\r$//' | sed 's/\x1b\[[0-9;]*[a-zA-Z]//g'
fi
}
# ------------------------------------------------------------------------------
# get_full_log()
#
# - Returns the FULL installation log (build + install combined)
# - Calls ensure_log_on_host() to pull container log if needed
# - Strips ANSI escape codes and carriage returns
# - Truncates to max_bytes (default: 120KB) to stay within API limits
# - Used for the error telemetry field (full trace instead of 20 lines)
# ------------------------------------------------------------------------------
get_full_log() {
local max_bytes="${1:-122880}" # 120KB default
local logfile=""
# Ensure logs are available on host (pulls from container if needed)
if declare -f ensure_log_on_host >/dev/null 2>&1; then
ensure_log_on_host
fi
# Try combined log first (most complete)
if [[ -n "${CTID:-}" && -n "${SESSION_ID:-}" ]]; then
local combined_log="/tmp/${NSAPP:-lxc}-${CTID}-${SESSION_ID}.log"
if [[ -s "$combined_log" ]]; then
logfile="$combined_log"
fi
fi
# Fall back to INSTALL_LOG
if [[ -z "$logfile" || ! -s "$logfile" ]]; then
if [[ -n "${INSTALL_LOG:-}" && -s "${INSTALL_LOG}" ]]; then
logfile="$INSTALL_LOG"
fi
fi
# Fall back to BUILD_LOG
if [[ -z "$logfile" || ! -s "$logfile" ]]; then
if [[ -n "${BUILD_LOG:-}" && -s "${BUILD_LOG}" ]]; then
logfile="$BUILD_LOG"
fi
fi
# Fall back to SILENT_LOGFILE (captures $STD command output)
if [[ -z "$logfile" || ! -s "$logfile" ]]; then
if [[ -n "${SILENT_LOGFILE:-}" && -s "${SILENT_LOGFILE}" ]]; then
logfile="$SILENT_LOGFILE"
fi
fi
if [[ -n "$logfile" && -s "$logfile" ]]; then
# Strip ANSI codes, carriage returns, and anonymize IP addresses (GDPR)
sed 's/\r$//' "$logfile" 2>/dev/null |
sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' |
sed -E 's/([0-9]{1,3}\.)[0-9]{1,3}\.[0-9]{1,3}/\1x.x/g' |
head -c "$max_bytes"
fi
}
# ------------------------------------------------------------------------------
# build_error_string()
#
# - Builds a structured error string for telemetry reporting
# - Format: "exit_code=<N> | <explanation>\n---\n<last 20 log lines>"
# - If no log lines available, returns just the explanation
# - Arguments:
# * $1: exit_code (numeric)
# * $2: log_text (optional, output from get_error_text)
# - Returns structured error string via stdout
# ------------------------------------------------------------------------------
build_error_string() {
local exit_code="${1:-1}"
local log_text="${2:-}"
local explanation
explanation=$(explain_exit_code "$exit_code")
if [[ -n "$log_text" ]]; then
# Structured format: header + separator + log lines
printf 'exit_code=%s | %s\n---\n%s' "$exit_code" "$explanation" "$log_text"
else
# No log available - just the explanation with exit code
printf 'exit_code=%s | %s' "$exit_code" "$explanation"
fi
}
# ==============================================================================
# SECTION 2: TELEMETRY FUNCTIONS
# ==============================================================================
# ------------------------------------------------------------------------------
# detect_gpu()
#
# - Detects GPU vendor, model, and passthrough type
# - Sets GPU_VENDOR, GPU_MODEL, and GPU_PASSTHROUGH globals
# - Used for GPU analytics
# ------------------------------------------------------------------------------
detect_gpu() {
GPU_VENDOR="unknown"
GPU_MODEL=""
GPU_PASSTHROUGH="unknown"
local gpu_line
gpu_line=$(lspci 2>/dev/null | grep -iE "VGA|3D|Display" | head -1 || true)
if [[ -n "$gpu_line" ]]; then
# Extract model: everything after the colon, clean up
GPU_MODEL=$(echo "$gpu_line" | sed 's/.*: //' | sed 's/ (rev .*)$//' | cut -c1-64)
# Detect vendor and passthrough type
if echo "$gpu_line" | grep -qi "Intel"; then
GPU_VENDOR="intel"
GPU_PASSTHROUGH="igpu"
elif echo "$gpu_line" | grep -qi "AMD\|ATI"; then
GPU_VENDOR="amd"
if echo "$gpu_line" | grep -qi "Radeon RX\|Radeon Pro"; then
GPU_PASSTHROUGH="dgpu"
else
GPU_PASSTHROUGH="igpu"
fi
elif echo "$gpu_line" | grep -qi "NVIDIA"; then
GPU_VENDOR="nvidia"
GPU_PASSTHROUGH="dgpu"
fi
fi
export GPU_VENDOR GPU_MODEL GPU_PASSTHROUGH
}
# ------------------------------------------------------------------------------
# detect_cpu()
#
# - Detects CPU vendor and model
# - Sets CPU_VENDOR (intel/amd/arm/unknown) and CPU_MODEL globals
# - Used for CPU analytics
# ------------------------------------------------------------------------------
detect_cpu() {
CPU_VENDOR="unknown"
CPU_MODEL=""
if [[ -f /proc/cpuinfo ]]; then
local vendor_id
vendor_id=$(grep -m1 "vendor_id" /proc/cpuinfo 2>/dev/null | cut -d: -f2 | tr -d ' ' || true)
case "$vendor_id" in
GenuineIntel) CPU_VENDOR="intel" ;;
AuthenticAMD) CPU_VENDOR="amd" ;;
*)
# ARM doesn't have vendor_id, check for CPU implementer
if grep -qi "CPU implementer" /proc/cpuinfo 2>/dev/null; then
CPU_VENDOR="arm"
fi
;;
esac
# Extract model name and clean it up
CPU_MODEL=$(grep -m1 "model name" /proc/cpuinfo 2>/dev/null | cut -d: -f2 | sed 's/^ *//' | sed 's/(R)//g' | sed 's/(TM)//g' | sed 's/ */ /g' | cut -c1-64 || true)
fi
export CPU_VENDOR CPU_MODEL
}
# ------------------------------------------------------------------------------
# detect_ram()
#
# - Detects RAM speed using dmidecode
# - Sets RAM_SPEED global (e.g., "4800" for DDR5-4800)
# - Requires root access for dmidecode
# - Returns empty if not available or if speed is "Unknown" (nested VMs)
# ------------------------------------------------------------------------------
detect_ram() {
RAM_SPEED=""
if command -v dmidecode &>/dev/null; then
# Get configured memory speed (actual running speed)
# Use || true to handle "Unknown" values in nested VMs (no numeric match)
RAM_SPEED=$(dmidecode -t memory 2>/dev/null | grep -m1 "Configured Memory Speed:" | grep -oE "[0-9]+" | head -1) || true
# Fallback to Speed: if Configured not available
if [[ -z "$RAM_SPEED" ]]; then
RAM_SPEED=$(dmidecode -t memory 2>/dev/null | grep -m1 "Speed:" | grep -oE "[0-9]+" | head -1) || true
fi
fi
export RAM_SPEED
}
# ------------------------------------------------------------------------------
# post_to_api()
#
# - Sends LXC container creation statistics to telemetry ingest service
# - Only executes if:
# * curl is available
# * DIAGNOSTICS=yes
# * RANDOM_UUID is set
# - Payload includes:
# * Container type, disk size, CPU cores, RAM
# * OS type and version
# * Application name (NSAPP)
# * Installation method
# * PVE version
# * Status: "installing"
# * Random UUID for session tracking
# - Anonymous telemetry (no personal data)
# - Never blocks or fails script execution
# ------------------------------------------------------------------------------
post_to_api() {
# Prevent duplicate submissions (post_to_api is called from multiple places)
[[ "${POST_TO_API_DONE:-}" == "true" ]] && return 0
# Silent fail - telemetry should never break scripts
command -v curl &>/dev/null || {
[[ "${DEV_MODE:-}" == "true" ]] && echo "[DEBUG] curl not found, skipping" >&2
return 0
}
[[ "${DIAGNOSTICS:-no}" == "no" ]] && {
[[ "${DEV_MODE:-}" == "true" ]] && echo "[DEBUG] DIAGNOSTICS=no, skipping" >&2
return 0
}
[[ -z "${RANDOM_UUID:-}" ]] && {
[[ "${DEV_MODE:-}" == "true" ]] && echo "[DEBUG] RANDOM_UUID empty, skipping" >&2
return 0
}
[[ "${DEV_MODE:-}" == "true" ]] && echo "[DEBUG] post_to_api() DIAGNOSTICS=$DIAGNOSTICS RANDOM_UUID=$RANDOM_UUID NSAPP=$NSAPP" >&2
# Set type for later status updates (preserve if already set, e.g. turnkey)
TELEMETRY_TYPE="${TELEMETRY_TYPE:-lxc}"
local pve_version=""
if command -v pveversion &>/dev/null; then
pve_version=$(pveversion 2>/dev/null | awk -F'[/ ]' '{print $2}') || true
fi
# Detect GPU if not already set
if [[ -z "${GPU_VENDOR:-}" ]]; then
detect_gpu
fi
local gpu_vendor="${GPU_VENDOR:-unknown}"
local gpu_model
gpu_model=$(json_escape "${GPU_MODEL:-}")
local gpu_passthrough="${GPU_PASSTHROUGH:-unknown}"
# Detect CPU if not already set
if [[ -z "${CPU_VENDOR:-}" ]]; then
detect_cpu
fi
local cpu_vendor="${CPU_VENDOR:-unknown}"
local cpu_model
cpu_model=$(json_escape "${CPU_MODEL:-}")
# Detect RAM if not already set
if [[ -z "${RAM_SPEED:-}" ]]; then
detect_ram
fi
local ram_speed="${RAM_SPEED:-}"
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
{
"random_id": "${RANDOM_UUID}",
"execution_id": "${EXECUTION_ID:-${RANDOM_UUID}}",
"type": "${TELEMETRY_TYPE}",
"nsapp": "${NSAPP:-unknown}",
"status": "installing",
"ct_type": ${CT_TYPE:-1},
"disk_size": ${DISK_SIZE:-0},
"core_count": ${CORE_COUNT:-0},
"ram_size": ${RAM_SIZE:-0},
"os_type": "${var_os:-}",
"os_version": "${var_version:-}",
"pve_version": "${pve_version}",
"method": "${METHOD:-default}",
"cpu_vendor": "${cpu_vendor}",
"cpu_model": "${cpu_model}",
"gpu_vendor": "${gpu_vendor}",
"gpu_model": "${gpu_model}",
"gpu_passthrough": "${gpu_passthrough}",
"ram_speed": "${ram_speed}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
[[ "${DEV_MODE:-}" == "true" ]] && echo "[DEBUG] Sending to: $TELEMETRY_URL" >&2
[[ "${DEV_MODE:-}" == "true" ]] && echo "[DEBUG] Payload: $JSON_PAYLOAD" >&2
# Send initial "installing" record with retry.
# This record MUST exist for all subsequent updates to succeed.
local http_code="" attempt
local _post_success=false
for attempt in 1 2 3; do
if [[ "${DEV_MODE:-}" == "true" ]]; then
http_code=$(curl -sS -w "%{http_code}" -m "${TELEMETRY_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" -o /dev/stderr 2>&1) || http_code="000"
echo "[DEBUG] post_to_api attempt $attempt HTTP=$http_code" >&2
else
http_code=$(curl -sS -w "%{http_code}" -m "${TELEMETRY_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" -o /dev/null 2>/dev/null) || http_code="000"
fi
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
_post_success=true
break
fi
[[ "$attempt" -lt 3 ]] && sleep 1
done
# Only mark done if at least one attempt succeeded.
# If all 3 failed, POST_TO_API_DONE stays false so post_update_to_api
# and on_exit() know the initial record was never created.
# The server has fallback logic to create a new record on status updates,
# so subsequent calls can still succeed even without the initial record.
POST_TO_API_DONE=${_post_success}
}
# ------------------------------------------------------------------------------
# post_to_api_vm()
#
# - Sends VM creation statistics to telemetry ingest service
# - Reads DIAGNOSTICS from /usr/local/community-scripts/diagnostics file
# - Payload differences from LXC:
# * ct_type=2 (VM instead of LXC)
# * type="vm"
# * Disk size without 'G' suffix
# - Includes hardware detection: CPU, GPU, RAM speed
# - Only executes if DIAGNOSTICS=yes and RANDOM_UUID is set
# - Never blocks or fails script execution
# ------------------------------------------------------------------------------
post_to_api_vm() {
# Read diagnostics setting from file
if [[ -f /usr/local/community-scripts/diagnostics ]]; then
DIAGNOSTICS=$(grep -i "^DIAGNOSTICS=" /usr/local/community-scripts/diagnostics 2>/dev/null | awk -F'=' '{print $2}') || true
fi
# Silent fail - telemetry should never break scripts
command -v curl &>/dev/null || return 0
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
[[ -z "${RANDOM_UUID:-}" ]] && return 0
# Set type for later status updates
TELEMETRY_TYPE="vm"
local pve_version=""
if command -v pveversion &>/dev/null; then
pve_version=$(pveversion 2>/dev/null | awk -F'[/ ]' '{print $2}') || true
fi
# Detect GPU if not already set
if [[ -z "${GPU_VENDOR:-}" ]]; then
detect_gpu
fi
local gpu_vendor="${GPU_VENDOR:-unknown}"
local gpu_model
gpu_model=$(json_escape "${GPU_MODEL:-}")
local gpu_passthrough="${GPU_PASSTHROUGH:-unknown}"
# Detect CPU if not already set
if [[ -z "${CPU_VENDOR:-}" ]]; then
detect_cpu
fi
local cpu_vendor="${CPU_VENDOR:-unknown}"
local cpu_model
cpu_model=$(json_escape "${CPU_MODEL:-}")
# Detect RAM if not already set
if [[ -z "${RAM_SPEED:-}" ]]; then
detect_ram
fi
local ram_speed="${RAM_SPEED:-}"
# Remove 'G' suffix from disk size
local DISK_SIZE_API="${DISK_SIZE%G}"
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
{
"random_id": "${RANDOM_UUID}",
"execution_id": "${EXECUTION_ID:-${RANDOM_UUID}}",
"type": "vm",
"nsapp": "${NSAPP:-unknown}",
"status": "installing",
"ct_type": 2,
"disk_size": ${DISK_SIZE_API:-0},
"core_count": ${CORE_COUNT:-0},
"ram_size": ${RAM_SIZE:-0},
"os_type": "${var_os:-}",
"os_version": "${var_version:-}",
"pve_version": "${pve_version}",
"method": "${METHOD:-default}",
"cpu_vendor": "${cpu_vendor}",
"cpu_model": "${cpu_model}",
"gpu_vendor": "${gpu_vendor}",
"gpu_model": "${gpu_model}",
"gpu_passthrough": "${gpu_passthrough}",
"ram_speed": "${ram_speed}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
# Send initial "installing" record with retry (must succeed for updates to work)
local http_code="" attempt
local _post_success=false
for attempt in 1 2 3; do
http_code=$(curl -sS -w "%{http_code}" -m "${TELEMETRY_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" -o /dev/null 2>/dev/null) || http_code="000"
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
_post_success=true
break
fi
[[ "$attempt" -lt 3 ]] && sleep 1
done
POST_TO_API_DONE=${_post_success}
}
# ------------------------------------------------------------------------------
# post_progress_to_api()
#
# - Lightweight progress ping from host or container
# - Updates the existing telemetry record status
# - Arguments:
# * $1: status (optional, default: "configuring")
# Valid values: "validation", "configuring"
# - Signals that the installation is actively progressing (not stuck)
# - Fire-and-forget: never blocks or fails the script
# - Only executes if DIAGNOSTICS=yes and RANDOM_UUID is set
# - Can be called multiple times safely
# ------------------------------------------------------------------------------
post_progress_to_api() {
command -v curl &>/dev/null || return 0
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
[[ -z "${RANDOM_UUID:-}" ]] && return 0
local progress_status="${1:-configuring}"
local app_name="${NSAPP:-${app:-unknown}}"
local telemetry_type="${TELEMETRY_TYPE:-lxc}"
curl -fsS -m 5 -X POST "${TELEMETRY_URL:-https://telemetry.community-scripts.org/telemetry}" \
-H "Content-Type: application/json" \
-d "{\"random_id\":\"${RANDOM_UUID}\",\"execution_id\":\"${EXECUTION_ID:-${RANDOM_UUID}}\",\"type\":\"${telemetry_type}\",\"nsapp\":\"${app_name}\",\"status\":\"${progress_status}\"}" &>/dev/null || true
}
# ------------------------------------------------------------------------------
# post_update_to_api()
#
# - Reports installation completion status to telemetry ingest service
# - Prevents duplicate submissions via POST_UPDATE_DONE flag
# - Arguments:
# * $1: status ("done" or "failed")
# * $2: exit_code (numeric, default: 1 for failed, 0 for done)
# - Payload includes:
# * Final status (mapped: "done"→"success", "failed"→"failed")
# * Error description via explain_exit_code()
# * Numeric exit code
# - Only executes once per session
# - Never blocks or fails script execution
# ------------------------------------------------------------------------------
post_update_to_api() {
# Silent fail - telemetry should never break scripts
command -v curl &>/dev/null || return 0
# Support "force" mode (3rd arg) to bypass duplicate check for retries after cleanup
local force="${3:-}"
POST_UPDATE_DONE=${POST_UPDATE_DONE:-false}
if [[ "$POST_UPDATE_DONE" == "true" && "$force" != "force" ]]; then
return 0
fi
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
[[ -z "${RANDOM_UUID:-}" ]] && return 0
local status="${1:-failed}"
local raw_exit_code="${2:-1}"
local exit_code=0 error="" pb_status error_category=""
# Get GPU info (if detected)
local gpu_vendor="${GPU_VENDOR:-unknown}"
local gpu_model
gpu_model=$(json_escape "${GPU_MODEL:-}")
local gpu_passthrough="${GPU_PASSTHROUGH:-unknown}"
# Get CPU info (if detected)
local cpu_vendor="${CPU_VENDOR:-unknown}"
local cpu_model
cpu_model=$(json_escape "${CPU_MODEL:-}")
# Get RAM info (if detected)
local ram_speed="${RAM_SPEED:-}"
# Map status to telemetry values: installing, success, failed, unknown
case "$status" in
done | success)
pb_status="success"
exit_code=0
error=""
error_category=""
;;
failed)
pb_status="failed"
;;
*)
pb_status="unknown"
;;
esac
# For failed/unknown status, resolve exit code and error description
local short_error="" medium_error=""
if [[ "$pb_status" == "failed" ]] || [[ "$pb_status" == "unknown" ]]; then
if [[ "$raw_exit_code" =~ ^[0-9]+$ ]]; then
exit_code="$raw_exit_code"
else
exit_code=1
fi
# Get full installation log for error field
local log_text=""
log_text=$(get_full_log 122880) || true # 120KB max
if [[ -z "$log_text" ]]; then
# Fallback to last 20 lines
log_text=$(get_error_text)
fi
local full_error
full_error=$(build_error_string "$exit_code" "$log_text")
error=$(json_escape "$full_error")
short_error=$(json_escape "$(explain_exit_code "$exit_code")")
error_category=$(categorize_error "$exit_code")
[[ -z "$error" ]] && error="Unknown error"
# Build medium error for attempt 2: explanation + last 100 log lines (≤16KB)
# This is the critical middle ground between full 120KB log and generic-only description
local medium_log=""
medium_log=$(get_full_log 16384) || true # 16KB max
if [[ -z "$medium_log" ]]; then
medium_log=$(get_error_text) || true
fi
local medium_full
medium_full=$(build_error_string "$exit_code" "$medium_log")
medium_error=$(json_escape "$medium_full")
[[ -z "$medium_error" ]] && medium_error="$short_error"
fi
# Calculate duration if timer was started
local duration=0
if [[ -n "${INSTALL_START_TIME:-}" ]]; then
duration=$(($(date +%s) - INSTALL_START_TIME))
fi
# Get PVE version
local pve_version=""
if command -v pveversion &>/dev/null; then
pve_version=$(pveversion 2>/dev/null | awk -F'[/ ]' '{print $2}') || true
fi
local http_code=""
# Strip 'G' suffix from disk size (VMs set DISK_SIZE=32G)
local DISK_SIZE_API="${DISK_SIZE:-0}"
DISK_SIZE_API="${DISK_SIZE_API%G}"
[[ ! "$DISK_SIZE_API" =~ ^[0-9]+$ ]] && DISK_SIZE_API=0
# ── Attempt 1: Full payload with complete error text (includes full log) ──
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
{
"random_id": "${RANDOM_UUID}",
"execution_id": "${EXECUTION_ID:-${RANDOM_UUID}}",
"type": "${TELEMETRY_TYPE:-lxc}",
"nsapp": "${NSAPP:-unknown}",
"status": "${pb_status}",
"ct_type": ${CT_TYPE:-1},
"disk_size": ${DISK_SIZE_API},
"core_count": ${CORE_COUNT:-0},
"ram_size": ${RAM_SIZE:-0},
"os_type": "${var_os:-}",
"os_version": "${var_version:-}",
"pve_version": "${pve_version}",
"method": "${METHOD:-default}",
"exit_code": ${exit_code},
"error": "${error}",
"error_category": "${error_category}",
"install_duration": ${duration},
"cpu_vendor": "${cpu_vendor}",
"cpu_model": "${cpu_model}",
"gpu_vendor": "${gpu_vendor}",
"gpu_model": "${gpu_model}",
"gpu_passthrough": "${gpu_passthrough}",
"ram_speed": "${ram_speed}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
http_code=$(curl -sS -w "%{http_code}" -m "${STATUS_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" -o /dev/null 2>/dev/null) || http_code="000"
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
POST_UPDATE_DONE=true
return 0
fi
# ── Attempt 2: Medium error text (truncated log ≤16KB instead of full 120KB) ──
sleep 1
local RETRY_PAYLOAD
RETRY_PAYLOAD=$(
cat <<EOF
{
"random_id": "${RANDOM_UUID}",
"execution_id": "${EXECUTION_ID:-${RANDOM_UUID}}",
"type": "${TELEMETRY_TYPE:-lxc}",
"nsapp": "${NSAPP:-unknown}",
"status": "${pb_status}",
"ct_type": ${CT_TYPE:-1},
"disk_size": ${DISK_SIZE_API},
"core_count": ${CORE_COUNT:-0},
"ram_size": ${RAM_SIZE:-0},
"os_type": "${var_os:-}",
"os_version": "${var_version:-}",
"pve_version": "${pve_version}",
"method": "${METHOD:-default}",
"exit_code": ${exit_code},
"error": "${medium_error}",
"error_category": "${error_category}",
"install_duration": ${duration},
"cpu_vendor": "${cpu_vendor}",
"cpu_model": "${cpu_model}",
"gpu_vendor": "${gpu_vendor}",
"gpu_model": "${gpu_model}",
"gpu_passthrough": "${gpu_passthrough}",
"ram_speed": "${ram_speed}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
http_code=$(curl -sS -w "%{http_code}" -m "${STATUS_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$RETRY_PAYLOAD" -o /dev/null 2>/dev/null) || http_code="000"
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
POST_UPDATE_DONE=true
return 0
fi
# ── Attempt 3: Minimal payload with medium error (bare minimum to set status) ──
sleep 2
local MINIMAL_PAYLOAD
MINIMAL_PAYLOAD=$(
cat <<EOF
{
"random_id": "${RANDOM_UUID}",
"execution_id": "${EXECUTION_ID:-${RANDOM_UUID}}",
"type": "${TELEMETRY_TYPE:-lxc}",
"nsapp": "${NSAPP:-unknown}",
"status": "${pb_status}",
"exit_code": ${exit_code},
"error": "${medium_error}",
"error_category": "${error_category}",
"install_duration": ${duration}
}
EOF
)
http_code=$(curl -sS -w "%{http_code}" -m "${STATUS_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$MINIMAL_PAYLOAD" -o /dev/null 2>/dev/null) || http_code="000"
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
POST_UPDATE_DONE=true
return 0
fi
# All 3 attempts failed — do NOT set POST_UPDATE_DONE=true.
# This allows the EXIT trap (on_exit in error_handler.func) to retry.
# No infinite loop risk: EXIT trap fires exactly once.
}
# ==============================================================================
# SECTION 3: EXTENDED TELEMETRY FUNCTIONS
# ==============================================================================
# ------------------------------------------------------------------------------
# categorize_error()
#
# - Maps exit codes to error categories for better analytics
# - Categories: network, storage, dependency, permission, timeout, config, resource, unknown
# - Used to group errors in dashboard
# ------------------------------------------------------------------------------
categorize_error() {
# Allow build.func to override category based on log analysis (exit code 1 subclassification)
if [[ -n "${ERROR_CATEGORY_OVERRIDE:-}" ]]; then
echo "$ERROR_CATEGORY_OVERRIDE"
return
fi
local code="$1"
case "$code" in
# Network errors (curl/wget)
6 | 7 | 22 | 35) echo "network" ;;
# Docker / Privileged mode required
10) echo "config" ;;
# Timeout errors
28 | 124 | 211) echo "timeout" ;;
# Storage errors (Proxmox storage)
214 | 217 | 219 | 224) echo "storage" ;;
# Dependency/Package errors (APT, DPKG, pip, commands)
100 | 101 | 102 | 127 | 160 | 161 | 162 | 255) echo "dependency" ;;
# Permission errors
126 | 152) echo "permission" ;;
# Configuration errors (Proxmox config, invalid args)
128 | 203 | 204 | 205 | 206 | 207 | 208) echo "config" ;;
# Proxmox container/template errors
200 | 209 | 210 | 212 | 213 | 215 | 216 | 218 | 220 | 221 | 222 | 223 | 225 | 231) echo "proxmox" ;;
# Service/Systemd errors
150 | 151 | 153 | 154) echo "service" ;;
# Database errors (PostgreSQL, MySQL, MongoDB)
170 | 171 | 172 | 173 | 180 | 181 | 182 | 183 | 190 | 191 | 192 | 193) echo "database" ;;
# Node.js / JavaScript runtime errors
243 | 245 | 246 | 247 | 248 | 249) echo "runtime" ;;
# Python environment errors
# (already covered: 160-162 under dependency)
# Aborted by user (SIGHUP=terminal closed, SIGINT=Ctrl+C, SIGTERM=killed)
129 | 130 | 143) echo "user_aborted" ;;
# Resource errors (OOM, SIGKILL, SIGABRT)
134 | 137) echo "resource" ;;
# Signal/Process errors (SIGPIPE, SIGSEGV)
139 | 141) echo "signal" ;;
# Shell errors (general error, syntax error)
1 | 2) echo "shell" ;;
# Default - truly unknown
*) echo "unknown" ;;
esac
}
# ------------------------------------------------------------------------------
# start_install_timer()
#
# - Captures start time for installation duration tracking
# - Call at the beginning of installation
# - Sets INSTALL_START_TIME global variable
# ------------------------------------------------------------------------------
start_install_timer() {
INSTALL_START_TIME=$(date +%s)
export INSTALL_START_TIME
}
# ------------------------------------------------------------------------------
# get_install_duration()
#
# - Returns elapsed seconds since start_install_timer() was called
# - Returns 0 if timer was not started
# ------------------------------------------------------------------------------
get_install_duration() {
if [[ -z "${INSTALL_START_TIME:-}" ]]; then
echo "0"
return
fi
local now=$(date +%s)
echo $((now - INSTALL_START_TIME))
}
# ------------------------------------------------------------------------------
# _telemetry_report_exit()
#
# - Internal handler called by EXIT trap set in init_tool_telemetry()
# - Determines success/failure from exit code and reports via appropriate API
# - Arguments:
# * $1: exit_code from the script
# ------------------------------------------------------------------------------
_telemetry_report_exit() {
local ec="${1:-0}"
local status="success"
[[ "$ec" -ne 0 ]] && status="failed"
# Lazy name resolution: use explicit name, fall back to $APP, then "unknown"
local name="${TELEMETRY_TOOL_NAME:-${APP:-unknown}}"
if [[ "${TELEMETRY_TOOL_TYPE:-pve}" == "addon" ]]; then
post_addon_to_api "$name" "$status" "$ec"
else
post_tool_to_api "$name" "$status" "$ec"
fi
}
# ------------------------------------------------------------------------------
# init_tool_telemetry()
#
# - One-line telemetry setup for tools/addon scripts
# - Reads DIAGNOSTICS from /usr/local/community-scripts/diagnostics
# (persisted on PVE host during first build, and inside containers by install.func)
# - Starts install timer for duration tracking
# - Sets EXIT trap to automatically report success/failure on script exit
# - Arguments:
# * $1: tool_name (optional, falls back to $APP at exit time)
# * $2: type ("pve" for PVE host scripts, "addon" for container addons)
# - Usage:
# source <(curl -fsSL .../misc/api.func) 2>/dev/null || true
# init_tool_telemetry "post-pve-install" "pve"
# init_tool_telemetry "" "addon" # uses $APP at exit time
# ------------------------------------------------------------------------------
init_tool_telemetry() {
local name="${1:-}"
local type="${2:-pve}"
[[ -n "$name" ]] && TELEMETRY_TOOL_NAME="$name"
TELEMETRY_TOOL_TYPE="$type"
# Read diagnostics opt-in/opt-out
if [[ -f /usr/local/community-scripts/diagnostics ]]; then
DIAGNOSTICS=$(grep -i "^DIAGNOSTICS=" /usr/local/community-scripts/diagnostics 2>/dev/null | awk -F'=' '{print $2}') || true
fi
start_install_timer
# EXIT trap: automatically report telemetry when script ends
trap '_telemetry_report_exit "$?"' EXIT
}
# ------------------------------------------------------------------------------
# post_tool_to_api()
#
# - Reports tool usage to telemetry
# - Arguments:
# * $1: tool_name (e.g., "microcode", "lxc-update", "post-pve-install")
# * $2: status ("success" or "failed")
# * $3: exit_code (optional, default: 0 for success, 1 for failed)
# - For PVE host tools, not container installations
# ------------------------------------------------------------------------------
post_tool_to_api() {
command -v curl &>/dev/null || return 0
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
local tool_name="${1:-unknown}"
local status="${2:-success}"
local exit_code="${3:-0}"
local error="" error_category=""
local uuid duration
# Generate UUID for this tool execution
uuid=$(cat /proc/sys/kernel/random/uuid 2>/dev/null || uuidgen 2>/dev/null || echo "tool-$(date +%s)")
duration=$(get_install_duration)
# Map status
[[ "$status" == "done" ]] && status="success"
if [[ "$status" == "failed" ]]; then
[[ ! "$exit_code" =~ ^[0-9]+$ ]] && exit_code=1
local error_text=""
error_text=$(get_error_text)
local full_error
full_error=$(build_error_string "$exit_code" "$error_text")
error=$(json_escape "$full_error")
error_category=$(categorize_error "$exit_code")
fi
local pve_version=""
if command -v pveversion &>/dev/null; then
pve_version=$(pveversion 2>/dev/null | awk -F'[/ ]' '{print $2}') || true
fi
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
{
"random_id": "${uuid}",
"execution_id": "${EXECUTION_ID:-${uuid}}",
"type": "pve",
"nsapp": "${tool_name}",
"status": "${status}",
"exit_code": ${exit_code},
"error": "${error}",
"error_category": "${error_category}",
"install_duration": ${duration:-0},
"pve_version": "${pve_version}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
curl -fsS -m "${TELEMETRY_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" &>/dev/null || true
}
# ------------------------------------------------------------------------------
# post_addon_to_api()
#
# - Reports addon installation to telemetry
# - Arguments:
# * $1: addon_name (e.g., "filebrowser", "netdata")
# * $2: status ("success" or "failed")
# * $3: exit_code (optional)
# - For addons installed inside containers
# ------------------------------------------------------------------------------
post_addon_to_api() {
command -v curl &>/dev/null || return 0
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
local addon_name="${1:-unknown}"
local status="${2:-success}"
local exit_code="${3:-0}"
local error="" error_category=""
local uuid duration
# Generate UUID for this addon installation
uuid=$(cat /proc/sys/kernel/random/uuid 2>/dev/null || uuidgen 2>/dev/null || echo "addon-$(date +%s)")
duration=$(get_install_duration)
# Map status
[[ "$status" == "done" ]] && status="success"
if [[ "$status" == "failed" ]]; then
[[ ! "$exit_code" =~ ^[0-9]+$ ]] && exit_code=1
local error_text=""
error_text=$(get_error_text)
local full_error
full_error=$(build_error_string "$exit_code" "$error_text")
error=$(json_escape "$full_error")
error_category=$(categorize_error "$exit_code")
fi
# Detect OS info
local os_type="" os_version=""
if [[ -f /etc/os-release ]]; then
os_type=$(grep "^ID=" /etc/os-release | cut -d= -f2 | tr -d '"' || true)
os_version=$(grep "^VERSION_ID=" /etc/os-release | cut -d= -f2 | tr -d '"' || true)
fi
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
{
"random_id": "${uuid}",
"execution_id": "${EXECUTION_ID:-${uuid}}",
"type": "addon",
"nsapp": "${addon_name}",
"status": "${status}",
"exit_code": ${exit_code},
"error": "${error}",
"error_category": "${error_category}",
"install_duration": ${duration:-0},
"os_type": "${os_type}",
"os_version": "${os_version}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
curl -fsS -m "${TELEMETRY_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" &>/dev/null || true
}
# ------------------------------------------------------------------------------
# post_update_to_api_extended()
#
# - Extended version of post_update_to_api with duration, GPU, and error category
# - Same arguments as post_update_to_api:
# * $1: status ("done" or "failed")
# * $2: exit_code (numeric)
# - Automatically includes:
# * Install duration (if start_install_timer was called)
# * Error category (for failed status)
# * GPU info (if detect_gpu was called)
# ------------------------------------------------------------------------------
post_update_to_api_extended() {
# Silent fail - telemetry should never break scripts
command -v curl &>/dev/null || return 0
# Prevent duplicate submissions
POST_UPDATE_DONE=${POST_UPDATE_DONE:-false}
[[ "$POST_UPDATE_DONE" == "true" ]] && return 0
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
[[ -z "${RANDOM_UUID:-}" ]] && return 0
local status="${1:-failed}"
local raw_exit_code="${2:-1}"
local exit_code=0 error="" pb_status error_category=""
local duration gpu_vendor gpu_passthrough
# Get duration
duration=$(get_install_duration)
# Get GPU info (if detected)
gpu_vendor="${GPU_VENDOR:-}"
gpu_passthrough="${GPU_PASSTHROUGH:-}"
# Map status to telemetry values
case "$status" in
done | success)
pb_status="success"
exit_code=0
error=""
error_category=""
;;
failed)
pb_status="failed"
;;
*)
pb_status="unknown"
;;
esac
# For failed/unknown status, resolve exit code and error description
if [[ "$pb_status" == "failed" ]] || [[ "$pb_status" == "unknown" ]]; then
if [[ "$raw_exit_code" =~ ^[0-9]+$ ]]; then
exit_code="$raw_exit_code"
else
exit_code=1
fi
local error_text=""
error_text=$(get_error_text)
local full_error
full_error=$(build_error_string "$exit_code" "$error_text")
error=$(json_escape "$full_error")
error_category=$(categorize_error "$exit_code")
[[ -z "$error" ]] && error="Unknown error"
fi
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
{
"random_id": "${RANDOM_UUID}",
"execution_id": "${EXECUTION_ID:-${RANDOM_UUID}}",
"type": "${TELEMETRY_TYPE:-lxc}",
"nsapp": "${NSAPP:-unknown}",
"status": "${pb_status}",
"exit_code": ${exit_code},
"error": "${error}",
"error_category": "${error_category}",
"install_duration": ${duration:-0},
"gpu_vendor": "${gpu_vendor}",
"gpu_passthrough": "${gpu_passthrough}",
"repo_source": "${REPO_SOURCE}"
}
EOF
)
local http_code
http_code=$(curl -sS -w "%{http_code}" -m "${STATUS_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "$JSON_PAYLOAD" -o /dev/null 2>/dev/null) || http_code="000"
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
POST_UPDATE_DONE=true
return 0
fi
# Retry with minimal payload
sleep 1
http_code=$(curl -sS -w "%{http_code}" -m "${STATUS_TIMEOUT}" -X POST "${TELEMETRY_URL}" \
-H "Content-Type: application/json" \
-d "{\"random_id\":\"${RANDOM_UUID}\",\"execution_id\":\"${EXECUTION_ID:-${RANDOM_UUID}}\",\"type\":\"${TELEMETRY_TYPE:-lxc}\",\"nsapp\":\"${NSAPP:-unknown}\",\"status\":\"${pb_status}\",\"exit_code\":${exit_code},\"install_duration\":${duration:-0}}" \
-o /dev/null 2>/dev/null) || http_code="000"
if [[ "$http_code" =~ ^2[0-9]{2}$ ]]; then
POST_UPDATE_DONE=true
return 0
fi
# Do NOT set POST_UPDATE_DONE=true — let EXIT trap retry
}