Dawnline 0.6.0
- Access rules fail closed: when the table cannot load, no device is loaded into the tunnels (the firewall alone would let every loaded device reach everything); an older table that survived a failed rebuild keeps its rules and a revoked device still leaves. The agent re-syncs within a minute when the table is gone - A change the router saved but could not put into effect (access rules that did not load, a server that refused its devices) is said in the admin UI instead of a plain "Saved" - The guard judges every packet, not only a connection's first: a change of access, a pause or a limit now cuts open connections too - Each server's home networks are its own: with several servers, "all home" on one no longer reaches the other's networks where firewall zones overlap - People limited to some home networks reach the router only on the addresses in those networks (LuCI and SSH on the main LAN stay closed) - Internet-only people can open their personal portal with the VPN on - Tunnel DNS reaches the resolver and the LAN's DNS from servers that do not forward to those zones on their own (Home-only names on servers without "like a home device", LAN DNS on servers with no home network) - A new personal link offers to remove the devices added through the old one, so a leaked link is really contained - Import and restore refuse an inconsistent file (devices with addresses outside their server, shared addresses, malformed keys, broken references); Health reports one, and the sync leaves such a device out instead of failing its server. A config write that did not reach flash is an error, not a silent success - Backups are kept per kind (nightly ones no longer push out manual or pre-delete ones), nightly and pre-change backups hold the config only, archives are written whole or not at all, and a failed nightly backup is logged. A restore no longer has its history overwritten by the agent's in-memory counters - Settings defaults and ranges come from one place (dawnline.schema) - Portal: anything not shaped like a personal link is answered by a shell before ucode starts (a scan on the root costs nothing); 4 portal processes at once instead of 16; a link may make 40 changes a quarter hour; over plain HTTP the page shows but configurations, new devices and new keys are refused with the HTTPS twin of the link; the connect sheet polls only while the first connection is awaited, with backoff, and stops once the link stopped working - Updates: after Dawnline's own package changes, a shell checks that LuCI reaches it and the agent runs, and puts the kept previous version back when they do not (automatic runs always keep it); the AmneziaWG module reload verifies rmmod and modprobe; installs check free space; on apk a reinstall that changed nothing is an error; uninstall.sh hands the update lock to the removal it calls; an upgrade restarts the service only where it was enabled; the release workflow pins the SDK images by digest and the actions by commit - AmneziaWG: features count only when both the tools and the kernel module have them; a server whose disguise this build cannot run (a 3.1 preset after a rollback to 2.0) is kept down and reported; Components tells 1.0 builds from 2.0 ones; `awg show dump` is parsed by shape; Maximum's DNS-shaped signature packets are made for each server and its keep-alive is a 25-35 s range - CI compiles every ucode file and runs the new tests/ucode unit tests (54 tests on the real libuci) in OpenWrt 24.10 and 25.12 rootfs containers; scripts/compile-ucode.sh does the same wherever ucode runs and scripts/ucode-container.sh locally with docker; lint-ucode checks POSIX regexes and module layering; a tag builds only after CI passes
This commit is contained in:
1 parent
2b98abdba4
commit
31a2065f05
70 files changed
+2008
-307
No files matched your search
@@ -19,9 +19,9 @@ grep -q "\"version\": \"$VERSION\"" "$ROOT/web/package.json" || { echo "web/pack
|
||||
mkdir -p "$STAGE/www" "$OUT"
|
||||
cp -R "$APP/root/." "$STAGE/"
|
||||
cp -R "$APP/htdocs/." "$STAGE/www/"
|
||||
chmod 755 "$STAGE/usr/bin/dawnline" "$STAGE/usr/libexec/dawnline/agent" "$STAGE/usr/libexec/dawnline/updater" "$STAGE/usr/libexec/dawnline/resolver" \
|
||||
chmod 755 "$STAGE/usr/bin/dawnline" "$STAGE/usr/libexec/dawnline/agent" "$STAGE/usr/libexec/dawnline/updater" "$STAGE/usr/libexec/dawnline/resolver" "$STAGE/usr/libexec/dawnline/postupdate" \
|
||||
"$STAGE/etc/init.d/dawnline" "$STAGE/etc/uci-defaults/90_dawnline" "$STAGE/etc/hotplug.d/iface/95-dawnline" \
|
||||
"$STAGE/usr/share/dawnline/portal/www/cgi-bin/portal"
|
||||
"$STAGE/usr/share/dawnline/portal/www/cgi-bin/portal" "$STAGE/usr/share/dawnline/portal/cgi.uc"
|
||||
find "$STAGE" -name '.DS_Store' -delete
|
||||
|
||||
TAR="$OUT/luci-app-dawnline-$VERSION.tar.gz"
|
||||
|
||||
+5
-19
@@ -1,6 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Compile-check every ucode file on a router without installing anything:
|
||||
# the tree is unpacked under /tmp/dlc and modules are resolved from there.
|
||||
# the tree is unpacked under /tmp/dlc and scripts/compile-ucode.sh runs there
|
||||
# (the same check CI runs in an OpenWrt rootfs container, scripts/ucode-container.sh).
|
||||
# scripts/check-ucode.sh [root@host]
|
||||
set -euo pipefail
|
||||
HOST="${1:-root@outline.lan}"
|
||||
@@ -12,21 +13,6 @@ TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
||||
python3 "$(dirname "$0")/lint-ucode.py" "$ROOT"
|
||||
( cd "$ROOT" && COPYFILE_DISABLE=1 tar -czf "$TMP/t.tgz" usr etc )
|
||||
scp -q "$TMP/t.tgz" "$HOST:/tmp/dl-check.tgz"
|
||||
ssh "$HOST" sh <<'REMOTE'
|
||||
rm -rf /tmp/dlc && mkdir -p /tmp/dlc && tar -xzf /tmp/dl-check.tgz -C /tmp/dlc && rm /tmp/dl-check.tgz
|
||||
cd /tmp/dlc
|
||||
rc=0
|
||||
# Modules by their import name: dawnline/x.uc is dawnline.x, dawnline/rpc/x.uc dawnline.rpc.x.
|
||||
for f in usr/share/ucode/dawnline/*.uc usr/share/ucode/dawnline/rpc/*.uc; do
|
||||
[ -f "$f" ] || continue
|
||||
n=$(echo "${f#usr/share/ucode/}" | sed 's/\.uc$//; s#/#.#g')
|
||||
out=$(ucode -L '/tmp/dlc/usr/share/ucode/*.uc' -e "import * as x from '$n';" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
|
||||
done
|
||||
for f in usr/share/rpcd/ucode/luci.dawnline usr/libexec/dawnline/agent usr/libexec/dawnline/updater usr/bin/dawnline usr/share/dawnline/portal/www/cgi-bin/portal; do
|
||||
[ -f "$f" ] || continue
|
||||
out=$(ucode -L '/tmp/dlc/usr/share/ucode/*.uc' -c -o /dev/null "$f" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
|
||||
done
|
||||
rm -rf /tmp/dlc
|
||||
[ $rc = 0 ] && echo "all ucode files compile"
|
||||
exit $rc
|
||||
REMOTE
|
||||
ssh "$HOST" 'rm -rf /tmp/dlc && mkdir -p /tmp/dlc && tar -xzf /tmp/dl-check.tgz -C /tmp/dlc && rm /tmp/dl-check.tgz'
|
||||
# The script goes over stdin (`sh -s`), nothing of it stays on the router.
|
||||
ssh "$HOST" 'sh -s /tmp/dlc; rc=$?; rm -rf /tmp/dlc; exit $rc' < "$(dirname "$0")/compile-ucode.sh"
|
||||
Executable
+30
@@ -0,0 +1,30 @@
|
||||
#!/bin/sh
|
||||
# Compile-check every ucode file of a Dawnline tree without installing it:
|
||||
# modules are imported by name (dawnline/x.uc is dawnline.x, dawnline/rpc/x.uc
|
||||
# dawnline.rpc.x) and entry points compiled with `ucode -c`. Needs ucode with
|
||||
# its fs, uci, ubus and uloop modules, which is any OpenWrt box or an
|
||||
# openwrt/rootfs container (scripts/ucode-container.sh); check-ucode.sh pipes
|
||||
# this script to a router.
|
||||
# scripts/compile-ucode.sh [ROOT] ROOT holds usr/ and etc/ (default: .)
|
||||
#
|
||||
# ucode resolves names at compile time and imports are static, so this catches
|
||||
# a syntax error, a missing module and a named import that nothing exports.
|
||||
set -eu
|
||||
ROOT="$(cd "${1:-.}" && pwd)"
|
||||
cd "$ROOT"
|
||||
LIB="$ROOT/usr/share/ucode/*.uc"
|
||||
rc=0
|
||||
for f in usr/share/ucode/dawnline/*.uc usr/share/ucode/dawnline/rpc/*.uc; do
|
||||
[ -f "$f" ] || continue
|
||||
n=$(echo "${f#usr/share/ucode/}" | sed 's/\.uc$//; s#/#.#g')
|
||||
out=$(ucode -L "$LIB" -e "import * as x from '$n';" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
|
||||
done
|
||||
# Entry points (ucode scripts outside the module tree); the shell ones live in
|
||||
# the ShellCheck list of .github/workflows/ci.yml instead.
|
||||
for f in usr/share/rpcd/ucode/luci.dawnline usr/libexec/dawnline/agent usr/libexec/dawnline/updater usr/bin/dawnline \
|
||||
usr/share/dawnline/portal/cgi.uc; do
|
||||
[ -f "$f" ] || { echo "FAIL $f: missing"; rc=1; continue; }
|
||||
out=$(ucode -L "$LIB" -c -o /dev/null "$f" 2>&1) || { echo "FAIL $f"; echo "$out" | head -8; rc=1; }
|
||||
done
|
||||
[ $rc = 0 ] && echo "all ucode files compile"
|
||||
exit $rc
|
||||
+43
-5
@@ -3,8 +3,10 @@
|
||||
|
||||
ucode resolves names where they are used: calling a function that is declared
|
||||
further down the same file fails at runtime ("access to undeclared variable").
|
||||
Also: calls and named imports must match what a Dawnline module exports, and
|
||||
the rpcd methods (dawnline.rpc.*) must match the plugin and its ACL.
|
||||
Also: calls and named imports must match what a Dawnline module exports, the
|
||||
rpcd methods (dawnline.rpc.*) must match the plugin and its ACL, and regexes
|
||||
stay POSIX (rules/backend.md). Imports against the module layering are
|
||||
reported as notes, not failures.
|
||||
Run: scripts/lint-ucode.py [package root]
|
||||
"""
|
||||
import glob
|
||||
@@ -22,9 +24,8 @@ files = modules + [
|
||||
PLUGIN,
|
||||
f"{root}/usr/libexec/dawnline/agent",
|
||||
f"{root}/usr/libexec/dawnline/updater",
|
||||
f"{root}/usr/libexec/dawnline/resolver",
|
||||
f"{root}/usr/bin/dawnline",
|
||||
f"{root}/usr/share/dawnline/portal/www/cgi-bin/portal",
|
||||
f"{root}/usr/share/dawnline/portal/cgi.uc",
|
||||
]
|
||||
files = [f for f in files if os.path.exists(f)]
|
||||
bad = 0
|
||||
@@ -40,6 +41,16 @@ def strip_comments(text):
|
||||
return re.sub(r'//[^\n]*|/\*.*?\*/', '', text, flags=re.S)
|
||||
|
||||
|
||||
def blank_comments(text):
|
||||
"""Comments replaced by their newlines: line numbers stay, their text goes."""
|
||||
return re.sub(r'//[^\n]*|/\*.*?\*/', lambda m: '\n' * m.group(0).count('\n'), text, flags=re.S)
|
||||
|
||||
|
||||
# A regex literal on one line: `/…/` with no slash inside, followed by flags or
|
||||
# a delimiter (so a division keeps out). Good enough for how Dawnline writes them.
|
||||
REGEX_LITERAL = re.compile(r'(?<![\w)\]])/((?:\\.|[^/\\\n])+)/[gis]*(?=[,);\s])')
|
||||
|
||||
|
||||
def modname(path):
|
||||
"""dawnline/rpc/people.uc → 'rpc.people' (as in import … from 'dawnline.rpc.people')."""
|
||||
rel = os.path.relpath(path, MODDIR)[:-3]
|
||||
@@ -55,7 +66,7 @@ for f in files:
|
||||
problem(f"{f}:{line}: {name}() is used before its definition")
|
||||
# In a module an exported function is an expression statement: its closing
|
||||
# brace needs a semicolon ("Expecting ';'" breaks the whole module).
|
||||
lines = open(f, encoding='utf-8').read().split('\n')
|
||||
lines = blank_comments(open(f, encoding='utf-8').read()).split('\n')
|
||||
inside = None
|
||||
for n, text in enumerate(lines, 1):
|
||||
if inside is None and re.match(r'^export function \w+\(', text):
|
||||
@@ -68,6 +79,14 @@ for f in files:
|
||||
# not preceded by valid expression"), and let [a, b] = … does not compile.
|
||||
if re.search(r'/[^/\n]*\(\?[:=!][^/\n]*/', text):
|
||||
problem(f"{f}:{n}: ucode regexes are POSIX: no (?: (?= (?! groups")
|
||||
# Lazy quantifiers (*? +? }?) and the [\s\S] class are PCRE: the C
|
||||
# library's regcomp does something else with them or rejects them.
|
||||
for m in REGEX_LITERAL.finditer(text):
|
||||
body = m.group(1)
|
||||
if re.search(r'(?<!\\)[*+}]\?', body):
|
||||
problem(f"{f}:{n}: ucode regexes are POSIX: no lazy quantifiers (*? +? }}?) in /{body}/")
|
||||
if re.search(r'\[\\[sS]\\[sS]\]', body):
|
||||
problem(f"{f}:{n}: ucode regexes are POSIX: no [\\s\\S] class in /{body}/ (use (.|\\n) or a negated class)")
|
||||
if re.match(r'\s*(let|const)\s*[\[{]', text):
|
||||
problem(f"{f}:{n}: ucode has no destructuring")
|
||||
|
||||
@@ -124,5 +143,24 @@ if methods and os.path.exists(ACL):
|
||||
if 'uci' in acl.get(kind, {}) and 'dawnline' in acl[kind]['uci']:
|
||||
problem(f"{ACL}: {kind} grants uci 'dawnline' (private keys, no validation): use the rpc methods")
|
||||
|
||||
# Layering (rules/backend.md): util, schema → core → store → domain modules →
|
||||
# apply → entry points. A module importing one from a layer above it is
|
||||
# reported, not failed, until the existing edges are untangled.
|
||||
LAYER = {'util': 0, 'schema': 1, 'core': 2, 'store': 3, 'apply': 5}
|
||||
LAYER_NAME = {0: 'util', 1: 'schema', 2: 'core', 3: 'store', 4: 'domain', 5: 'apply'}
|
||||
notes = 0
|
||||
for f in modules:
|
||||
mod = modname(f)
|
||||
if mod.startswith('rpc.'):
|
||||
continue # an entry point: may import anything
|
||||
mine = LAYER.get(mod, 4)
|
||||
for dep in re.findall(r"from 'dawnline\.([\w.]+)'", strip_comments(open(f, encoding='utf-8').read())):
|
||||
theirs = LAYER.get(dep, 4)
|
||||
if theirs > mine or (theirs == mine and mine < 4 and dep != mod):
|
||||
print(f"note: {f}: {mod} ({LAYER_NAME[mine]}) imports dawnline.{dep} ({LAYER_NAME[theirs]}), a layer above it")
|
||||
notes += 1
|
||||
if notes:
|
||||
print(f"lint-ucode: {notes} layering notes (not failures)")
|
||||
|
||||
print(f"lint-ucode: {len(files)} files checked" + (", problems found" if bad else ", ok"))
|
||||
sys.exit(bad)
|
||||
Executable
+51
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run the ucode checks the way CI does: in an OpenWrt rootfs container with the
|
||||
# release's own ucode and libuci — compile every file (compile-ucode.sh), then
|
||||
# the unit tests (tests/ucode/run.sh). Needs docker; the repository is mounted
|
||||
# read-only at /src, the container is thrown away.
|
||||
# scripts/ucode-container.sh [24.10|25.12|all] [test name filter]
|
||||
#
|
||||
# Images are the x86-64 rootfs of the OpenWrt releases the packages are built
|
||||
# for (build/Dockerfile, scripts/build-packages.sh), pinned by digest so a
|
||||
# retagged image cannot change what CI runs. To move to a new release change
|
||||
# the tag and the digest together (docker buildx imagetools inspect <image>).
|
||||
set -euo pipefail
|
||||
REPO="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
image_for() {
|
||||
case "$1" in
|
||||
24.10) echo 'openwrt/rootfs:x86-64-24.10.6@sha256:fd76f3afb9c390ef2915618b6e3d7187698c03ecc6c23d3a13f12b62d9c6209e' ;;
|
||||
25.12) echo 'openwrt/rootfs:x86-64-25.12.3@sha256:af882e0583954fc2ceac6b081a9d214fc739cfea36a29b48795a5f15563aa3b5' ;;
|
||||
*) echo "unknown release $1 (24.10 or 25.12)" >&2; exit 2 ;;
|
||||
esac
|
||||
}
|
||||
releases="${1:-all}"
|
||||
[ "$releases" = all ] && releases='24.10 25.12'
|
||||
rc=0
|
||||
for rel in $releases; do
|
||||
image="$(image_for "$rel")"
|
||||
echo "== OpenWrt $rel ($image)"
|
||||
# The checks run from this stdin script (`sh -s`): ucode-mod-uloop is not in
|
||||
# the 24.10 rootfs, so what is missing is installed from the release's feeds.
|
||||
docker run --rm -i --platform linux/amd64 -v "$REPO:/src:ro" -e "FILTER=${2:-}" "$image" sh -s <<-'INNER' || rc=1
|
||||
set -eu
|
||||
missing=''
|
||||
command -v ucode >/dev/null || missing='ucode'
|
||||
for m in fs uci ubus uloop; do
|
||||
[ -e "/usr/lib/ucode/$m.so" ] || missing="$missing ucode-mod-$m"
|
||||
done
|
||||
if [ -n "$missing" ]; then
|
||||
echo "installing:$missing"
|
||||
if command -v apk >/dev/null; then
|
||||
# shellcheck disable=SC2086
|
||||
apk add --quiet $missing
|
||||
else
|
||||
mkdir -p /var/lock
|
||||
# shellcheck disable=SC2086
|
||||
opkg update >/dev/null && opkg install $missing >/dev/null
|
||||
fi
|
||||
fi
|
||||
sh /src/scripts/compile-ucode.sh /src/package/luci-app-dawnline/root
|
||||
sh /src/tests/ucode/run.sh /src "$FILTER"
|
||||
INNER
|
||||
done
|
||||
exit $rc
|
||||
Reference in new issue
Block a user