Dawnline 0.6.0

- Access rules fail closed: when the table cannot load, no device is loaded
  into the tunnels (the firewall alone would let every loaded device reach
  everything); an older table that survived a failed rebuild keeps its rules
  and a revoked device still leaves. The agent re-syncs within a minute when
  the table is gone
- A change the router saved but could not put into effect (access rules that
  did not load, a server that refused its devices) is said in the admin UI
  instead of a plain "Saved"
- The guard judges every packet, not only a connection's first: a change of
  access, a pause or a limit now cuts open connections too
- Each server's home networks are its own: with several servers, "all home"
  on one no longer reaches the other's networks where firewall zones overlap
- People limited to some home networks reach the router only on the
  addresses in those networks (LuCI and SSH on the main LAN stay closed)
- Internet-only people can open their personal portal with the VPN on
- Tunnel DNS reaches the resolver and the LAN's DNS from servers that do not
  forward to those zones on their own (Home-only names on servers without
  "like a home device", LAN DNS on servers with no home network)
- A new personal link offers to remove the devices added through the old one,
  so a leaked link is really contained
- Import and restore refuse an inconsistent file (devices with addresses
  outside their server, shared addresses, malformed keys, broken references);
  Health reports one, and the sync leaves such a device out instead of
  failing its server. A config write that did not reach flash is an error,
  not a silent success
- Backups are kept per kind (nightly ones no longer push out manual or
  pre-delete ones), nightly and pre-change backups hold the config only,
  archives are written whole or not at all, and a failed nightly backup is
  logged. A restore no longer has its history overwritten by the agent's
  in-memory counters
- Settings defaults and ranges come from one place (dawnline.schema)
- Portal: anything not shaped like a personal link is answered by a shell
  before ucode starts (a scan on the root costs nothing); 4 portal processes
  at once instead of 16; a link may make 40 changes a quarter hour; over
  plain HTTP the page shows but configurations, new devices and new keys are
  refused with the HTTPS twin of the link; the connect sheet polls only while
  the first connection is awaited, with backoff, and stops once the link
  stopped working
- Updates: after Dawnline's own package changes, a shell checks that LuCI
  reaches it and the agent runs, and puts the kept previous version back when
  they do not (automatic runs always keep it); the AmneziaWG module reload
  verifies rmmod and modprobe; installs check free space; on apk a reinstall
  that changed nothing is an error; uninstall.sh hands the update lock to the
  removal it calls; an upgrade restarts the service only where it was
  enabled; the release workflow pins the SDK images by digest and the actions
  by commit
- AmneziaWG: features count only when both the tools and the kernel module
  have them; a server whose disguise this build cannot run (a 3.1 preset
  after a rollback to 2.0) is kept down and reported; Components tells 1.0
  builds from 2.0 ones; `awg show dump` is parsed by shape; Maximum's
  DNS-shaped signature packets are made for each server and its keep-alive
  is a 25-35 s range
- CI compiles every ucode file and runs the new tests/ucode unit tests (54
  tests on the real libuci) in OpenWrt 24.10 and 25.12 rootfs containers;
  scripts/compile-ucode.sh does the same wherever ucode runs and
  scripts/ucode-container.sh locally with docker; lint-ucode checks POSIX
  regexes and module layering; a tag builds only after CI passes
This commit is contained in:
Aemeath committed 2026-10-18 15:32:27 +03:00
1 parent 2b98abdba4
commit 31a2065f05
70 files changed
+2008 -307

No files matched your search

+2 -2
View File
@@ -19,9 +19,9 @@ grep -q "\"version\": \"$VERSION\"" "$ROOT/web/package.json" || { echo "web/pack
mkdir -p "$STAGE/www" "$OUT"
cp -R "$APP/root/." "$STAGE/"
cp -R "$APP/htdocs/." "$STAGE/www/"
chmod 755 "$STAGE/usr/bin/dawnline" "$STAGE/usr/libexec/dawnline/agent" "$STAGE/usr/libexec/dawnline/updater" "$STAGE/usr/libexec/dawnline/resolver" \
chmod 755 "$STAGE/usr/bin/dawnline" "$STAGE/usr/libexec/dawnline/agent" "$STAGE/usr/libexec/dawnline/updater" "$STAGE/usr/libexec/dawnline/resolver" "$STAGE/usr/libexec/dawnline/postupdate" \
"$STAGE/etc/init.d/dawnline" "$STAGE/etc/uci-defaults/90_dawnline" "$STAGE/etc/hotplug.d/iface/95-dawnline" \
"$STAGE/usr/share/dawnline/portal/www/cgi-bin/portal"
"$STAGE/usr/share/dawnline/portal/www/cgi-bin/portal" "$STAGE/usr/share/dawnline/portal/cgi.uc"
find "$STAGE" -name '.DS_Store' -delete
TAR="$OUT/luci-app-dawnline-$VERSION.tar.gz"
+5 -19
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
# Compile-check every ucode file on a router without installing anything:
# the tree is unpacked under /tmp/dlc and modules are resolved from there.
# the tree is unpacked under /tmp/dlc and scripts/compile-ucode.sh runs there
# (the same check CI runs in an OpenWrt rootfs container, scripts/ucode-container.sh).
# scripts/check-ucode.sh [root@host]
set -euo pipefail
HOST="${1:-root@outline.lan}"
@@ -12,21 +13,6 @@ TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
python3 "$(dirname "$0")/lint-ucode.py" "$ROOT"
( cd "$ROOT" && COPYFILE_DISABLE=1 tar -czf "$TMP/t.tgz" usr etc )
scp -q "$TMP/t.tgz" "$HOST:/tmp/dl-check.tgz"
ssh "$HOST" sh <<'REMOTE'
rm -rf /tmp/dlc && mkdir -p /tmp/dlc && tar -xzf /tmp/dl-check.tgz -C /tmp/dlc && rm /tmp/dl-check.tgz
cd /tmp/dlc
rc=0
# Modules by their import name: dawnline/x.uc is dawnline.x, dawnline/rpc/x.uc dawnline.rpc.x.
for f in usr/share/ucode/dawnline/*.uc usr/share/ucode/dawnline/rpc/*.uc; do
[ -f "$f" ] || continue
n=$(echo "${f#usr/share/ucode/}" | sed 's/\.uc$//; s#/#.#g')
out=$(ucode -L '/tmp/dlc/usr/share/ucode/*.uc' -e "import * as x from '$n';" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
done
for f in usr/share/rpcd/ucode/luci.dawnline usr/libexec/dawnline/agent usr/libexec/dawnline/updater usr/bin/dawnline usr/share/dawnline/portal/www/cgi-bin/portal; do
[ -f "$f" ] || continue
out=$(ucode -L '/tmp/dlc/usr/share/ucode/*.uc' -c -o /dev/null "$f" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
done
rm -rf /tmp/dlc
[ $rc = 0 ] && echo "all ucode files compile"
exit $rc
REMOTE
ssh "$HOST" 'rm -rf /tmp/dlc && mkdir -p /tmp/dlc && tar -xzf /tmp/dl-check.tgz -C /tmp/dlc && rm /tmp/dl-check.tgz'
# The script goes over stdin (`sh -s`), nothing of it stays on the router.
ssh "$HOST" 'sh -s /tmp/dlc; rc=$?; rm -rf /tmp/dlc; exit $rc' < "$(dirname "$0")/compile-ucode.sh"
+30
View File
@@ -0,0 +1,30 @@
#!/bin/sh
# Compile-check every ucode file of a Dawnline tree without installing it:
# modules are imported by name (dawnline/x.uc is dawnline.x, dawnline/rpc/x.uc
# dawnline.rpc.x) and entry points compiled with `ucode -c`. Needs ucode with
# its fs, uci, ubus and uloop modules, which is any OpenWrt box or an
# openwrt/rootfs container (scripts/ucode-container.sh); check-ucode.sh pipes
# this script to a router.
# scripts/compile-ucode.sh [ROOT] ROOT holds usr/ and etc/ (default: .)
#
# ucode resolves names at compile time and imports are static, so this catches
# a syntax error, a missing module and a named import that nothing exports.
set -eu
ROOT="$(cd "${1:-.}" && pwd)"
cd "$ROOT"
LIB="$ROOT/usr/share/ucode/*.uc"
rc=0
for f in usr/share/ucode/dawnline/*.uc usr/share/ucode/dawnline/rpc/*.uc; do
[ -f "$f" ] || continue
n=$(echo "${f#usr/share/ucode/}" | sed 's/\.uc$//; s#/#.#g')
out=$(ucode -L "$LIB" -e "import * as x from '$n';" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
done
# Entry points (ucode scripts outside the module tree); the shell ones live in
# the ShellCheck list of .github/workflows/ci.yml instead.
for f in usr/share/rpcd/ucode/luci.dawnline usr/libexec/dawnline/agent usr/libexec/dawnline/updater usr/bin/dawnline \
usr/share/dawnline/portal/cgi.uc; do
[ -f "$f" ] || { echo "FAIL $f: missing"; rc=1; continue; }
out=$(ucode -L "$LIB" -c -o /dev/null "$f" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
done
[ $rc = 0 ] && echo "all ucode files compile"
exit $rc
+43 -5
View File
@@ -3,8 +3,10 @@
ucode resolves names where they are used: calling a function that is declared
further down the same file fails at runtime ("access to undeclared variable").
Also: calls and named imports must match what a Dawnline module exports, and
the rpcd methods (dawnline.rpc.*) must match the plugin and its ACL.
Also: calls and named imports must match what a Dawnline module exports, the
rpcd methods (dawnline.rpc.*) must match the plugin and its ACL, and regexes
stay POSIX (rules/backend.md). Imports against the module layering are
reported as notes, not failures.
Run: scripts/lint-ucode.py [package root]
"""
import glob
@@ -22,9 +24,8 @@ files = modules + [
PLUGIN,
f"{root}/usr/libexec/dawnline/agent",
f"{root}/usr/libexec/dawnline/updater",
f"{root}/usr/libexec/dawnline/resolver",
f"{root}/usr/bin/dawnline",
f"{root}/usr/share/dawnline/portal/www/cgi-bin/portal",
f"{root}/usr/share/dawnline/portal/cgi.uc",
]
files = [f for f in files if os.path.exists(f)]
bad = 0
@@ -40,6 +41,16 @@ def strip_comments(text):
return re.sub(r'//[^\n]*|/\*.*?\*/', '', text, flags=re.S)
def blank_comments(text):
"""Comments replaced by their newlines: line numbers stay, their text goes."""
return re.sub(r'//[^\n]*|/\*.*?\*/', lambda m: '\n' * m.group(0).count('\n'), text, flags=re.S)
# A regex literal on one line: `/…/` with no slash inside, followed by flags or
# a delimiter (so a division keeps out). Good enough for how Dawnline writes them.
REGEX_LITERAL = re.compile(r'(?<![\w)\]])/((?:\\.|[^/\\\n])+)/[gis]*(?=[,);\s])')
def modname(path):
"""dawnline/rpc/people.uc → 'rpc.people' (as in import … from 'dawnline.rpc.people')."""
rel = os.path.relpath(path, MODDIR)[:-3]
@@ -55,7 +66,7 @@ for f in files:
problem(f"{f}:{line}: {name}() is used before its definition")
# In a module an exported function is an expression statement: its closing
# brace needs a semicolon ("Expecting ';'" breaks the whole module).
lines = open(f, encoding='utf-8').read().split('\n')
lines = blank_comments(open(f, encoding='utf-8').read()).split('\n')
inside = None
for n, text in enumerate(lines, 1):
if inside is None and re.match(r'^export function \w+\(', text):
@@ -68,6 +79,14 @@ for f in files:
# not preceded by valid expression"), and let [a, b] = … does not compile.
if re.search(r'/[^/\n]*\(\?[:=!][^/\n]*/', text):
problem(f"{f}:{n}: ucode regexes are POSIX: no (?: (?= (?! groups")
# Lazy quantifiers (*? +? }?) and the [\s\S] class are PCRE: the C
# library's regcomp does something else with them or rejects them.
for m in REGEX_LITERAL.finditer(text):
body = m.group(1)
if re.search(r'(?<!\\)[*+}]\?', body):
problem(f"{f}:{n}: ucode regexes are POSIX: no lazy quantifiers (*? +? }}?) in /{body}/")
if re.search(r'\[\\[sS]\\[sS]\]', body):
problem(f"{f}:{n}: ucode regexes are POSIX: no [\\s\\S] class in /{body}/ (use (.|\\n) or a negated class)")
if re.match(r'\s*(let|const)\s*[\[{]', text):
problem(f"{f}:{n}: ucode has no destructuring")
@@ -124,5 +143,24 @@ if methods and os.path.exists(ACL):
if 'uci' in acl.get(kind, {}) and 'dawnline' in acl[kind]['uci']:
problem(f"{ACL}: {kind} grants uci 'dawnline' (private keys, no validation): use the rpc methods")
# Layering (rules/backend.md): util, schema → core → store → domain modules →
# apply → entry points. A module importing one from a layer above it is
# reported, not failed, until the existing edges are untangled.
LAYER = {'util': 0, 'schema': 1, 'core': 2, 'store': 3, 'apply': 5}
LAYER_NAME = {0: 'util', 1: 'schema', 2: 'core', 3: 'store', 4: 'domain', 5: 'apply'}
notes = 0
for f in modules:
mod = modname(f)
if mod.startswith('rpc.'):
continue # an entry point: may import anything
mine = LAYER.get(mod, 4)
for dep in re.findall(r"from 'dawnline\.([\w.]+)'", strip_comments(open(f, encoding='utf-8').read())):
theirs = LAYER.get(dep, 4)
if theirs > mine or (theirs == mine and mine < 4 and dep != mod):
print(f"note: {f}: {mod} ({LAYER_NAME[mine]}) imports dawnline.{dep} ({LAYER_NAME[theirs]}), a layer above it")
notes += 1
if notes:
print(f"lint-ucode: {notes} layering notes (not failures)")
print(f"lint-ucode: {len(files)} files checked" + (", problems found" if bad else ", ok"))
sys.exit(bad)
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Run the ucode checks the way CI does: in an OpenWrt rootfs container with the
# release's own ucode and libuci — compile every file (compile-ucode.sh), then
# the unit tests (tests/ucode/run.sh). Needs docker; the repository is mounted
# read-only at /src, the container is thrown away.
# scripts/ucode-container.sh [24.10|25.12|all] [test name filter]
#
# Images are the x86-64 rootfs of the OpenWrt releases the packages are built
# for (build/Dockerfile, scripts/build-packages.sh), pinned by digest so a
# retagged image cannot change what CI runs. To move to a new release change
# the tag and the digest together (docker buildx imagetools inspect <image>).
set -euo pipefail
REPO="$(cd "$(dirname "$0")/.." && pwd)"
image_for() {
case "$1" in
24.10) echo 'openwrt/rootfs:x86-64-24.10.6@sha256:fd76f3afb9c390ef2915618b6e3d7187698c03ecc6c23d3a13f12b62d9c6209e' ;;
25.12) echo 'openwrt/rootfs:x86-64-25.12.3@sha256:af882e0583954fc2ceac6b081a9d214fc739cfea36a29b48795a5f15563aa3b5' ;;
*) echo "unknown release $1 (24.10 or 25.12)" >&2; exit 2 ;;
esac
}
releases="${1:-all}"
[ "$releases" = all ] && releases='24.10 25.12'
rc=0
for rel in $releases; do
image="$(image_for "$rel")"
echo "== OpenWrt $rel ($image)"
# The checks run from this stdin script (`sh -s`): ucode-mod-uloop is not in
# the 24.10 rootfs, so what is missing is installed from the release's feeds.
docker run --rm -i --platform linux/amd64 -v "$REPO:/src:ro" -e "FILTER=${2:-}" "$image" sh -s <<-'INNER' || rc=1
set -eu
missing=''
command -v ucode >/dev/null || missing='ucode'
for m in fs uci ubus uloop; do
[ -e "/usr/lib/ucode/$m.so" ] || missing="$missing ucode-mod-$m"
done
if [ -n "$missing" ]; then
echo "installing:$missing"
if command -v apk >/dev/null; then
# shellcheck disable=SC2086
apk add --quiet $missing
else
mkdir -p /var/lock
# shellcheck disable=SC2086
opkg update >/dev/null && opkg install $missing >/dev/null
fi
fi
sh /src/scripts/compile-ucode.sh /src/package/luci-app-dawnline/root
sh /src/tests/ucode/run.sh /src "$FILTER"
INNER
done
exit $rc