Commit Graph

725 Commits

Author SHA1 Message Date
CanbiZ (MickLesk)
441158b195 Standardize mirror messaging; simplify apt update
Replace inconsistent msg_info calls with msg_warn for mirror-list warnings and msg_custom for "Attempting mirror" lines to unify output formatting in misc/build.func and misc/install.func. Simplify tools/pve/update-lxcs-cron.sh by removing the long mirror-probing block and using a straightforward apt-get update + noninteractive dist-upgrade (and cleanup of EXTERNALLY-MANAGED) in containers to reduce complexity and improve maintainability.
2026-03-26 16:32:25 +01:00
CanbiZ (MickLesk)
dd911670e1 Enhance apt/apk mirror fallback logic
Improve package mirror resiliency in misc/build.func and misc/install.func by adding fallback and detection logic for Debian/Ubuntu and Alpine CDNs. Changes include:

- Detect distro (debian vs ubuntu) and use appropriate mirror lists and primary mirror (ftp.debian.org vs archive.ubuntu.com).
- Add retry loops that try a shuffled list of regional mirrors, test connectivity, update repo files, and attempt package installs/updates again.
- For Alpine (apk) installs and updates, try alternate mirrors on failure, write /etc/apk/repositories, and surface clearer warnings/errors if all mirrors fail.
- Update interactive prompts and informational messages to reference the correct mirror list (Ubuntu vs Debian) and generalize prompt wording.
- Improve exit handling so failures are reported and abort the process when necessary.

These changes make automated installs more robust against CDN sync issues and unreachable primary mirrors.
2026-03-26 16:04:48 +01:00
CanbiZ (MickLesk)
937fd1cad4 Improve apt mirror logging and error messages
Standardize and clarify apt mirror diagnostics in misc/build.func and misc/install.func: update warning phrasing when apt-get update fails; change per-mirror failure messages from "Mirror X: <reason>" to "Mirror X failed (<reason>)"; change success messages to "CDN set to <mirror>: tests passed"; and add informational logs like "Attempting mirror: <mirror>" when trying alternate mirrors. These changes improve visibility into mirror selection and failure reasons during package update/installation.
2026-03-26 15:44:58 +01:00
CanbiZ (MickLesk)
5f25ce38b2 Report failing APT mirror on update failure
When apt-get update fails, capture and display the failing mirror host to aid debugging. Adds grep-based extraction of the upstream host (from /etc/apt/sources.list.d/debian.sources or /etc/apt/sources.list) with a fallback to "unknown", and includes the host in the warning/echo. Applied to misc/build.func (inside pct exec), misc/install.func (pkg_update), and tools/pve/update-lxcs-cron.sh.
2026-03-26 15:42:43 +01:00
CanbiZ (MickLesk)
92648bca13 Simplify mirror logs and use http for custom mirror
Reduce noisy mirror-related output and clarify messages across misc/build.func and misc/install.func. Reword various apt/mirror status lines (hash mismatch, SSL/certificate, apt-get update failed, package install failed) and standardize success to "Using mirror"/msg_ok. Remove verbose scan/try/skip/count logs and suppression of detailed apt output to make output cleaner for automated runs. Also change the custom_mirror sed replacement to use http:// instead of https:// to support non-HTTPS local mirrors.
2026-03-26 15:20:55 +01:00
CanbiZ (MickLesk)
f8ca562da6 Detect apt SSL errors and use http mirrors
Add explicit detection of SSL/certificate errors when running apt-get update and treat those mirrors as failed (logs a short SSL error snippet and returns failure). Change mirror URL rewrites to use http:// instead of https:// when substituting mirror hosts, and remove ftp.in.debian.org from the AP mirror lists. Changes applied to misc/build.func (try_mirrors) and misc/install.func (pkg_update/_try_apt_mirror) to improve mirror fallback behavior and error reporting.
2026-03-26 15:12:43 +01:00
CanbiZ (MickLesk)
cd1c412822 Try global mirrors first; add apt fallback
Reorder mirror probing in misc/build.func and misc/install.func to scan global (OTHERS) mirrors first and fall back to regional mirrors—pick up to 3 random reachable mirrors and try them. Rename related variables accordingly and keep the fallback logic intact. Update tools/pve/update-lxcs-cron.sh to define region-specific Debian mirror lists, detect container timezone to pick regional vs other mirrors, and implement a fallback loop that checks mirror connectivity, rewrites APT URIs to use a working mirror, and retries apt-get update before proceeding with dist-upgrade. These changes improve resilience against local CDN/local-regional outages by preferring globally reachable mirrors first and providing an automated mirror-switch fallback.
2026-03-26 14:52:03 +01:00
CanbiZ (MickLesk)
9db2169ee0 Improve apt mirror scanning and retry logic
Make APT mirror selection more robust by scanning for reachable mirrors and retrying installs with clearer failure reasons. In misc/build.func added mirror_exit, try_mirrors and scan_reachable helpers; prefer regional mirrors (pick up to 3), fall back to ftp.debian.org, then try global mirrors. Improved detection/reporting of hash mismatches vs apt-get errors and return codes, and use a short reachability TCP check before attempting updates. In misc/install.func added a reachable-mirror scan phase, count/report reachable mirrors, and provide clearer log messages when apt-get update/install fails. These changes aim to reduce failures caused by unreachable mirrors or CDN synchronization issues.
2026-03-26 14:46:27 +01:00
CanbiZ (MickLesk)
947a536351 Improve apt mirror selection and fallback
Refactor mirror logic in misc/build.func and misc/install.func to prefer regional Debian mirrors (detected from /etc/timezone), shuffle mirror lists, and include additional AP region hosts. Add quick TCP reachability checks, limit consecutive mirror failures, and surface clearer status messages. On repeated failures, prompt interactively for a custom mirror (with validation) and handle pct exec exit codes to present more informative errors and retry behavior.
2026-03-26 14:29:20 +01:00
CanbiZ (MickLesk)
76c79872e0 Fallback to alternate Debian mirrors on apt failure
When apt-get update fails, switch from disabling hash verification/insecure repos to trying a list of alternate Debian mirrors. Updated misc/build.func to iterate a curated mirror list, rewrite /etc/apt/sources.list* to point to each mirror, and attempt apt-get update/install until one succeeds (exiting with failure if all mirrors fail). Updated misc/install.func to perform a similar mirror-rotation loop and return an error if no mirror succeeds. Also adjusted warning/error messages to reflect the new behavior. This improves resilience against repo desyncs without enabling insecure apt settings.
2026-03-26 14:10:49 +01:00
CanbiZ (MickLesk)
6d213d511f Simplify APT retry logic and add insecure fallback
Replace the previous multi-step APT retry sequence (mirror swaps, sleeps, multiple retries) with a simpler fallback: on apt-get update failure disable Acquire::By-Hash, enable Acquire::AllowInsecureRepositories and attempt updates/installs using --allow-insecure-repositories/--allow-unauthenticated where needed. Restore secure settings and refresh lists afterwards, and preserve/propagate the original command exit status. Apply the same simplification in misc/build.func, misc/install.func and the Proxmox LXC cron updater (tools/pve/update-lxcs-cron.sh) to handle Debian repo desyncs more reliably and reduce complex retry logic.
2026-03-26 14:02:09 +01:00
CanbiZ (MickLesk)
a87cdd8b9e Update build.func 2026-03-26 13:50:55 +01:00
CanbiZ (MickLesk)
0f3a8c8406 Improve apt-get retry and mirror fallback
Add robust retry logic for APT operations in misc/build.func and misc/install.func. Introduces an apt_retry helper and multiple fallback steps: disable by-hash, switch to a country mirror (ftp.de.debian.org), wait and retry to allow mirror sync, and as a last resort temporarily relax APT verification to allow insecure repositories to complete updates. Ensures cleanup and restores secure settings where possible, with clearer failure handling and messages to increase resilience of package installation during container builds and installs.
2026-03-26 13:48:56 +01:00
CanbiZ (MickLesk)
66cd1fb05a Add APT by-hash bypass and mirror fallback
Improve APT retry logic to handle failures caused by by-hash/CDN issues. Both misc/build.func and misc/install.func now write an apt config to disable Acquire::By-Hash, remove /var/lib/apt/lists/* and retry apt-get update/install; if that still fails they substitute deb.debian.org with ftp.debian.org as a fallback. This makes container builds and package updates more robust against CDN/hash-related apt failures.
2026-03-26 13:33:58 +01:00
CanbiZ (MickLesk)
6614b65bc1 magicdns 2026-03-26 13:19:14 +01:00
CanbiZ (MickLesk)
785a215a50 base packages dns 2026-03-26 13:18:52 +01:00
MickLesk
96fc169f4b fix(build): prevent error handler crash from pipefail on grep
The missing_cmd grep pipeline in the install error handler runs
under set -o pipefail. When grep finds no match (exit 1), the
entire pipeline fails and triggers the ERR trap recursively,
crashing the recovery menu. Add || true to suppress this.
2026-03-21 19:16:33 +01:00
CanbiZ (MickLesk)
7ab160ff75 gitea sync 2026-03-18 14:58:47 +01:00
CanbiZ (MickLesk)
aaaf18de91 switch urls back to github (due slow gitea syncs) 2026-03-18 14:47:29 +01:00
CanbiZ (MickLesk)
fdc30949e3 feat(preflight): add enterprise repo subscription check
- New preflight_repo_access() warns if enterprise repos are active without subscription
- Scans /etc/apt/sources.list.d/ for enterprise.proxmox.com entries
- Tests HTTP access (detects 401/403 Unauthorized)
- Warning only — not a blocker (packages come from pve-no-subscription repo)
2026-03-03 14:42:59 +01:00
CanbiZ (MickLesk)
d242100531 refactor(preflight): move to install_script, clean UX flow
- Move run_preflight from build_container() into install_script() after header_info
- Shows: Header → preflight checks → 2s pause → clear (via next header_info call)
- On failure: show summary + exit cleanly (no ERR trap)
- Change all return 1 → return 0 in preflight functions (prevents ERR trap under set -Ee)
- Remove PREFLIGHT_DONE guard from build_container (no longer needed)
2026-03-03 14:10:03 +01:00
CanbiZ (MickLesk)
4254e15bba Update build.func 2026-03-03 14:06:26 +01:00
CanbiZ (MickLesk)
c35cfb59fb Update build.func 2026-03-03 13:51:20 +01:00
CanbiZ (MickLesk)
e29561e401 Update build.func 2026-03-03 13:40:40 +01:00
CanbiZ (MickLesk)
1ae3275248 Add template availability preflight check
Introduce preflight_template_available() in misc/build.func to verify a matching OS/version template exists locally or in the online pveam catalog, and to fail early with a helpful list of available versions when none is found (exit code 225). Integrate this check into run_preflight. Also move the run_preflight invocation inside install_script to run after header_info (so preflight output is visible) and remove the earlier preflight call.
2026-03-03 13:36:02 +01:00
CanbiZ (MickLesk)
ed0e6ed49c linting 2026-03-03 13:30:25 +01:00
CanbiZ (MickLesk)
f0c4843784 preflight dev 2026-03-03 13:27:42 +01:00
CanbiZ (MickLesk)
80cc18f9e1 Revert "[Linting] misc/build.func (first pass)" 2026-03-02 15:32:35 +01:00
CanbiZ (MickLesk)
c29b00c97e Merge branch 'main' into lint-build-func 2026-03-02 15:02:31 +01:00
CanbiZ (MickLesk)
d16008181d merge build.func 2026-02-24 09:47:12 +01:00
MickLesk
4b9121f41c feat: add configurable COMMUNITY_SCRIPTS_URL for dev sourcing (#1449)
- Replace all hardcoded source URLs with $COMMUNITY_SCRIPTS_URL variable
- Default: https://git.community-scripts.org/.../ProxmoxVED/raw/branch/main
- Override: export COMMUNITY_SCRIPTS_URL=https://.../your-fork/raw/branch/xyz
- Propagates into container via build.func exports
- Updated: 43 ct scripts, 14 deferred, 22 vm scripts, 5 func files
- Document in docs/DEV_MODE.md with usage examples
- Also normalizes legacy GitHub URLs to Gitea canonical URL
2026-02-22 14:36:31 +01:00
justin
a9188982ac Remove FUNCTIONS_FILE_PATH quotes 2026-02-14 12:30:27 -05:00
justin
bb4f7c17de $PCT_OPTIONS needs to be unquoted for now 2026-02-14 12:30:27 -05:00
justin
7ebfc41eab Co-Author 2026-02-14 12:30:27 -05:00
justin
c23e0086c4 Revert "Extract pct create cmd to var & call with command"
This reverts commit ec10ea8747.
2026-02-14 12:30:26 -05:00
justin
9b15034ddd Extract pct create cmd to var & call with command 2026-02-14 12:30:26 -05:00
justin
9a0a712dab Linting build.func first pass 2026-02-14 12:30:26 -05:00
CanbiZ (MickLesk)
82de09f19f fix: remove debug output, add duration tracking, cleanup duplicate log messages 2026-02-10 14:29:28 +01:00
CanbiZ (MickLesk)
c60ff2ae75 Update build.func 2026-02-10 14:18:42 +01:00
CanbiZ (MickLesk)
2c0c72b0e5 unified logging 2026-02-10 13:59:35 +01:00
Tobias
4cd72cc2ec Add APPLICATION_VERSION export to build function 2026-02-10 08:08:24 +01:00
CanbiZ (MickLesk)
5aa85ace6a Add telemetry debug/logging and failure report
Enhance post_update_to_api robustness and observability: add debug traces for entry, missing curl, duplicate submissions, DIAGNOSTICS/RANDOM_UUID checks, payload/URL output, and HTTP response code capture; make curl non-blocking and tolerant of failures. Also invoke post_update_to_api on installation failure so build/install errors are reported to telemetry. Includes a small comment glyph fix in build.func. Changes keep telemetry as a silent, best-effort path that won't break script execution.
2026-02-09 16:53:25 +01:00
CanbiZ (MickLesk)
ce375b02aa Refactor telemetry checks, relax method validation
Reformat telemetry pre-checks in misc/api.func for clearer multi-line condition handling and remove stray whitespace around the curl response logging. Tweak misc/build.func comment line (minor encoding/visual cleanup). In misc/data/service.go remove the strict allowedMethod map and its validation so the 'method' field is treated as optional/flexible; keep sanitization and numeric validations unchanged. These changes improve readability and allow custom method values without breaking behavior.
2026-02-09 16:51:51 +01:00
CanbiZ (MickLesk)
bcc6bb9f5f Update build.func 2026-02-09 16:46:25 +01:00
CanbiZ (MickLesk)
fafdf88e6a Update build.func 2026-02-09 16:12:35 +01:00
CanbiZ (MickLesk)
820d4551a1 Replace Go API with PocketBase; update docs
Remove the old Go/Mongo API (api/main.go, go.mod, go.sum, .env.example) and switch telemetry backend to PocketBase (http://db.community-scripts.org). Update documentation and flowcharts to reflect the PocketBase collection (_dev_telemetry_data), new REST endpoints (POST/PATCH/GET), field schema, and revised api.func integration (LXC/VM reporting and status updates). Misc scripts and helpers were adjusted (misc/api.func, misc/build.func, misc/error_handler.func) and a new misc/ingest.go was added. This consolidates telemetry to a hosted PocketBase instance and updates docs and integration points accordingly.
2026-02-09 15:34:17 +01:00
Finn Joshua Bartels
3232665b92 feat(core): dev mode configurator menu
- Implement interactive Whiptail checklist for dev_mode configuration
- Add logic to read and pre-populate menu from existing dev_mode enviorment var
2026-02-09 11:49:02 +01:00
CanbiZ (MickLesk)
dc5c1d3ef0 Improve locale setup and database user privileges
Add command to grant superuser, createrole, and createdb privileges to the postgres user during Databasus installation. Enhance LXC container customization by installing locales for Devuan templates and only configuring locale if locale.gen exists, improving compatibility with minimal templates.
2026-01-30 09:47:38 +01:00
justin
000a7a2701 Fix Error check and error message style 2026-01-29 10:21:41 +01:00
justin
8c4f1ce531 Make check_container_storage() POSIX compliant 2026-01-29 10:21:41 +01:00