Dawnline 0.20.0
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions
- **Backend rewritten in Go.** One daemon, `dawnlined`, replaces the whole ucode backend: the rpcd plugin, the agent, the CLI and the updater. The package no longer depends on ucode. - **Per-architecture builds.** The release carries the backend for arm64, ARMv7, mips, mipsle and amd64; the router fetches its own after the package is installed and checks its sha256. Dawnline does not run on other architectures (older ARM, 32-bit x86, RISC-V). - **Faster on the router.** A firewall sync takes 0.06 s instead of 0.9 s; a full apply takes about 300 ms. The portal is served by the daemon itself, without a CGI process per request. - **Live pages.** The admin and the portal follow a live stream from the router instead of re-fetching the whole state every 30 seconds. - **Admin redesigned in the portal's style.** Three sections (Overview · Log · Settings); a server is a page of its own with People, Places and Settings; a device opens as a dialog over its person. - **Changes wait for «Apply».** Every setting, and a person's access, is staged in one dock before it reaches the router. - **Configuration rollout.** After a change that makes devices fetch their configuration again, the admin shows how many have fetched it and who has not. Every setting shows beforehand (⟳N) how many devices it will touch. - **Waiting for a decision.** Overview and People put first the people who need one (access ended, limit reached, a device that does not connect), each with its one action a tap away. - **Bulk actions on people.** «Select» (or a long press on a card) picks several people to extend by a month, pause or resume at once. - **Undo.** An action done from a card or a request is undone from its toast. - **Update dialogs redesigned.** An update is one view from its question to its result; Dawnline's own update runs in it like every other, and the page reloads itself once the new version answers. Other versions are a list, each with its notes and Install on a page of its own. - **Site to site: networks behind a router.** A router connected as a device can have networks behind it (up to eight, /16 to /30): the server routes them and applies the router's access to them. A network that shares addresses with a tunnel, the router's own networks or another site is refused by name. - **Routers from the portal.** People can add a router and name its networks on their own page; the admin accepts or declines with a reason, or lets them through without asking. - **Check access and Health know the networks behind routers.** - **Usage limits reworked.** A monthly limit no longer follows the calendar. A person's month starts with their first traffic and ends a month later; the next one starts only when they connect again, so time away uses up none of it. Before, everyone's limit reset on the 1st. - Extra data lasts until that person's month ends; «Reset usage» starts the month anew. - A limit is for a month or for the whole term; the admin and the portal show when it resets. - Charts that said «this month» show the last 30 days. - After the update nobody loses what was counted: the month that was running becomes the person's first and ends on the 1st. - **Access by time of day removed.** The Hours limit is gone from the admin, the portal and the backend; saved hours are dropped. - **Expiry simplified.** An end date always stops access, with a heads-up three days before; the per-person choice is gone. - **No false disconnects from sleeping phones.** A phone or a tablet stays online through ten minutes of silence. - **Mobile Safari fixes.** Pressing a control no longer sends the page to its top; tab tracks that do not fit scroll; switches animate after a press.
This commit is contained in:
1 parent
f8130e39a8
commit
ac90575b0b
788 files changed
+139244
-41720
No files matched your search
@@ -0,0 +1,365 @@
|
||||
package methods
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/tickcount/luci-app-dawnline/backend/files"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/health"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/i18n"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/journal"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/model"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/uci"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/zone"
|
||||
)
|
||||
|
||||
// How many backups of each kind stay (the newest): the nightly ones must not push out what the admin made
|
||||
// by hand or what a removal left behind. A kind not named keeps three.
|
||||
var backupKeep = map[string]int{"manual": 10, "auto": 7, "pre-delete": 5, "pre-remove": 3, "pre-update": 3, "pre-restore": 3, "pre-import": 3}
|
||||
|
||||
// Kinds that hold the settings alone: made often, and never to bring history back.
|
||||
var configOnly = map[string]bool{"auto": true, "pre-delete": true, "pre-remove": true, "pre-update": true}
|
||||
|
||||
var backupKind = regexp.MustCompile(`^[a-z0-9-]{1,24}$`)
|
||||
|
||||
// backupCreate makes a backup now and returns its name (dawnline.backup backup_create): the settings, and
|
||||
// for the kinds that protect data about to be replaced the traffic history and the activity log too, as a
|
||||
// tar.gz with the paths ucode's restore unpacks at /. Written whole or not at all. Of each kind the oldest
|
||||
// beyond its number are removed.
|
||||
func backupCreate(d Deps, reason string, keep ...string) (string, error) {
|
||||
kind := "manual"
|
||||
if backupKind.MatchString(reason) {
|
||||
kind = reason
|
||||
}
|
||||
// the live counters and log first, so the backup has them as they are now
|
||||
if !configOnly[kind] {
|
||||
if raw, err := os.ReadFile(d.RunDir + "/usage.json"); err == nil {
|
||||
files.Write(d.DataDir+"/usage.json", raw, 0o600)
|
||||
}
|
||||
if d.Store != nil {
|
||||
if _, m, err := d.Store.Read(); err == nil {
|
||||
journal.Log{Run: d.RunDir, Data: d.DataDir}.Persist(journal.PolicyOf(m.Settings.Retention), clock())
|
||||
}
|
||||
}
|
||||
}
|
||||
dir := d.DataDir + "/backups"
|
||||
os.MkdirAll(dir, 0o700)
|
||||
name := "dawnline-" + time.Unix(clock(), 0).In(zone.Local()).Format("20060102-150405")
|
||||
if kind != "manual" {
|
||||
name += "-" + kind
|
||||
}
|
||||
name += ".tar.gz"
|
||||
want := [][2]string{{"etc/config/dawnline", d.Config}}
|
||||
if !configOnly[kind] {
|
||||
want = append(want, [2]string{"etc/dawnline/usage.json", d.DataDir + "/usage.json"}, [2]string{"etc/dawnline/events.json", d.DataDir + "/events.json"})
|
||||
}
|
||||
// not a backup's name until it is complete
|
||||
tmp := dir + "/." + name + ".tmp"
|
||||
err := func() error {
|
||||
f, err := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
zw := gzip.NewWriter(f)
|
||||
tw := tar.NewWriter(zw)
|
||||
for _, w := range want {
|
||||
raw, err := os.ReadFile(w[1])
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
st, _ := os.Stat(w[1])
|
||||
if err := tw.WriteHeader(&tar.Header{Name: w[0], Mode: 0o600, Size: int64(len(raw)), ModTime: st.ModTime(), Typeflag: tar.TypeReg, Uname: "root", Gname: "root"}); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tw.Write(raw); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := tw.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := zw.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
return f.Sync()
|
||||
}()
|
||||
if err == nil {
|
||||
err = os.Rename(tmp, dir+"/"+name)
|
||||
}
|
||||
if err != nil {
|
||||
os.Remove(tmp)
|
||||
return "", errors.New("Backup failed: the archive could not be written (is the flash full?)")
|
||||
}
|
||||
os.Chmod(dir+"/"+name, 0o600)
|
||||
seen := map[string]int{}
|
||||
for _, b := range backups(dir) {
|
||||
// the new one, and the backup being restored: it must outlive the backup made just before restoring it
|
||||
if b["name"] == name || (len(keep) > 0 && b["name"] == keep[0]) {
|
||||
continue
|
||||
}
|
||||
k := b["reason"].(string)
|
||||
keep, known := backupKeep[k]
|
||||
if !known {
|
||||
keep = 3
|
||||
}
|
||||
if seen[k]++; seen[k] > keep {
|
||||
os.Remove(dir + "/" + b["name"].(string))
|
||||
}
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
|
||||
// what a backup may hold, by its path in the archive
|
||||
var backupFiles = []string{"etc/config/dawnline", "etc/dawnline/usage.json", "etc/dawnline/events.json"}
|
||||
|
||||
// The unpacked size a backup may have (an upload itself is limited to 4 MB).
|
||||
const maxUnpacked = 16 * 1024 * 1024
|
||||
|
||||
// unpack reads a backup archive after checking it as dawnline.backup check_archive does: only Dawnline's own
|
||||
// files, all of them regular files (no links, devices or directories), within a sane unpacked size, the
|
||||
// settings among them.
|
||||
func unpack(raw []byte) (map[string][]byte, error) {
|
||||
invalid := errors.New("Not a valid backup archive.")
|
||||
zr, err := gzip.NewReader(bytes.NewReader(raw))
|
||||
if err != nil {
|
||||
return nil, invalid
|
||||
}
|
||||
tr := tar.NewReader(zr)
|
||||
out := map[string][]byte{}
|
||||
var total int64
|
||||
for {
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, invalid
|
||||
}
|
||||
name := strings.TrimPrefix(strings.TrimPrefix(h.Name, "."), "/")
|
||||
if h.Typeflag != tar.TypeReg {
|
||||
return nil, said{"Unexpected entry in backup: %s", []any{name}}
|
||||
}
|
||||
known := false
|
||||
for _, f := range backupFiles {
|
||||
known = known || f == name
|
||||
}
|
||||
if !known {
|
||||
return nil, said{"Unexpected file in backup: %s", []any{name}}
|
||||
}
|
||||
if total += h.Size; total > maxUnpacked {
|
||||
return nil, errors.New("Backup is too large.")
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(tr, maxUnpacked+1))
|
||||
if err != nil {
|
||||
return nil, invalid
|
||||
}
|
||||
out[name] = data
|
||||
}
|
||||
if out["etc/config/dawnline"] == nil {
|
||||
return nil, errors.New("Backup does not contain /etc/config/dawnline.")
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// usable refuses settings that cannot replace the current ones, listing what is wrong with them.
|
||||
func usable(d Deps, t *i18n.Catalog, text string) error {
|
||||
c, err := uci.Parse(text)
|
||||
if err != nil || len(c.Sections) == 0 {
|
||||
return errors.New("The configuration in this file cannot be read.")
|
||||
}
|
||||
// the router's networks that are not Dawnline's own tunnels' (the current settings say which those are)
|
||||
var foreign []string
|
||||
if d.Addresses != nil && d.Store != nil {
|
||||
ours := map[string]bool{}
|
||||
if _, now, err := d.Store.Read(); err == nil {
|
||||
for _, s := range now.Servers {
|
||||
ours[s.Ifname] = true
|
||||
}
|
||||
}
|
||||
for _, a := range d.Addresses() {
|
||||
if !ours[a.Iface] {
|
||||
foreign = append(foreign, a.CIDR)
|
||||
}
|
||||
}
|
||||
}
|
||||
if errs := health.ConfigErrors(model.Load(c, nil), t, parseFile("/etc/config/network"), foreign); len(errs) > 0 {
|
||||
return said{"This configuration can't be used: %s", []any{strings.Join(errs, " ")}}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// restore puts a backup's files in place: checked first, a backup of what is there made before, the
|
||||
// settings written first under the config lock (when they cannot be, nothing else is touched), then the
|
||||
// counters and the log, which the watcher takes up from flash; then ucode's full apply.
|
||||
func restore(d Deps, lang string, raw []byte, name string) (map[string]any, error) {
|
||||
parts, err := unpack(raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := usable(d, i18n.Use(d.Catalogs, lang), string(parts["etc/config/dawnline"])); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := backupCreate(d, "pre-restore", name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
os.MkdirAll(d.DataDir, 0o700)
|
||||
err = func() error {
|
||||
defer files.Lock(d.RunDir, "config")()
|
||||
for _, f := range [][2]string{{"etc/config/dawnline", d.Config}, {"etc/dawnline/usage.json", d.DataDir + "/usage.json"}, {"etc/dawnline/events.json", d.DataDir + "/events.json"}} {
|
||||
if data, has := parts[f[0]]; has {
|
||||
if files.Write(f[1], data, 0o600) != nil {
|
||||
return said{"Couldn't write %s. Is the flash full?", []any{"/" + f[0]}}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// the live counters and log must follow the restored files: the watcher loads them from flash anew
|
||||
os.Remove(d.RunDir + "/usage.json")
|
||||
os.Remove(d.RunDir + "/events.json")
|
||||
files.Write(d.RunDir+"/usage.reload", []byte(strconv.FormatInt(clock(), 10)), 0o600)
|
||||
applied := map[string]any{}
|
||||
if d.Apply != nil {
|
||||
applied = d.Apply(lang)
|
||||
}
|
||||
if d.Told != nil {
|
||||
d.Told()
|
||||
}
|
||||
return applied, nil
|
||||
}
|
||||
|
||||
// backupRestore brings a kept backup back.
|
||||
func backupRestore(d Deps, name, lang string) (map[string]any, error) {
|
||||
if !backupFile.MatchString(name) {
|
||||
return nil, errors.New("Invalid backup name.")
|
||||
}
|
||||
raw, err := os.ReadFile(d.DataDir + "/backups/" + name)
|
||||
if err != nil {
|
||||
return nil, errors.New("Backup not found.")
|
||||
}
|
||||
out, err := restore(d, lang, raw, name)
|
||||
if err == nil && d.Log != nil {
|
||||
d.Log.Admin("dawnline", "restored", "backup", name, "")
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
|
||||
// backupUpload brings back a backup the admin sends (base64, 4 MB at most).
|
||||
func backupUpload(d Deps, data, lang string) (map[string]any, error) {
|
||||
raw, err := base64.StdEncoding.DecodeString(strings.Trim(data, blank))
|
||||
if err != nil || len(raw) == 0 || len(raw) > 4*1024*1024 {
|
||||
return nil, errors.New("Upload is empty or too large.")
|
||||
}
|
||||
out, err := restore(d, lang, raw, "")
|
||||
if err == nil && d.Log != nil {
|
||||
d.Log.Admin("dawnline", "restored", "backup", "from a file", "")
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
|
||||
var (
|
||||
sectionName = regexp.MustCompile(`^[A-Za-z0-9_]{1,64}$`)
|
||||
optionName = regexp.MustCompile(`^[a-z0-9_]{1,32}$`)
|
||||
)
|
||||
|
||||
// importAll replaces the settings with an export (dawnline.backup import_json): only Dawnline's kinds of
|
||||
// section, options that are strings or lists of them; checked as a restore's, a backup made before. A
|
||||
// section's options are written in the order of their names (the bus hands them over as a map).
|
||||
func importAll(d Deps, data any, lang string) (map[string]any, error) {
|
||||
o, _ := data.(map[string]any)
|
||||
sections, isList := o["sections"].([]any)
|
||||
if o == nil || o["format"] != "dawnline" || !isList {
|
||||
return nil, errors.New("Not a Dawnline export.")
|
||||
}
|
||||
kinds := map[string]bool{"settings": true, "server": true, "person": true, "device": true, "service": true, "circle": true}
|
||||
c := &uci.Config{}
|
||||
for _, x := range sections {
|
||||
s, _ := x.(map[string]any)
|
||||
name, _ := s[".name"].(string)
|
||||
kind, _ := s[".type"].(string)
|
||||
if s == nil || !kinds[kind] || !sectionName.MatchString(name) {
|
||||
return nil, said{"Invalid section %s.", []any{name}}
|
||||
}
|
||||
c.Add(kind, name)
|
||||
keys := make([]string, 0, len(s))
|
||||
for k := range s {
|
||||
if !strings.HasPrefix(k, ".") {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, k := range keys {
|
||||
if !optionName.MatchString(k) {
|
||||
return nil, said{"Invalid option %s.", []any{k}}
|
||||
}
|
||||
switch v := s[k].(type) {
|
||||
case string:
|
||||
// an empty value is written as it is: a cursor would drop it, a file keeps it
|
||||
sec := c.Section(name)
|
||||
sec.Values[k] = uci.Value{Str: v}
|
||||
sec.Order = append(sec.Order, k)
|
||||
case []any:
|
||||
var list []string
|
||||
for _, e := range v {
|
||||
str, isString := e.(string)
|
||||
if !isString {
|
||||
return nil, said{"Invalid option %s.", []any{k}}
|
||||
}
|
||||
list = append(list, str)
|
||||
}
|
||||
if len(list) > 0 {
|
||||
sec := c.Section(name)
|
||||
sec.Values[k] = uci.Value{IsList: true, List: list}
|
||||
sec.Order = append(sec.Order, k)
|
||||
}
|
||||
default:
|
||||
return nil, said{"Invalid option %s.", []any{k}}
|
||||
}
|
||||
}
|
||||
}
|
||||
text := c.Format()
|
||||
if err := usable(d, i18n.Use(d.Catalogs, lang), text); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := backupCreate(d, "pre-import"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
err := func() error {
|
||||
defer files.Lock(d.RunDir, "config")()
|
||||
if files.Write(d.Config, []byte(text), 0o600) != nil {
|
||||
return said{"Couldn't write %s. Is the flash full?", []any{"/etc/config/dawnline"}}
|
||||
}
|
||||
return nil
|
||||
}()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
applied := map[string]any{}
|
||||
if d.Apply != nil {
|
||||
applied = d.Apply(lang)
|
||||
}
|
||||
if d.Log != nil {
|
||||
d.Log.Admin("dawnline", "imported", "", "settings from a file", "")
|
||||
}
|
||||
if d.Told != nil {
|
||||
d.Told()
|
||||
}
|
||||
return applied, nil
|
||||
}
|
||||
|
||||
// BackupCreate saves a backup of a kind ("auto": the nightly one) for who keeps the router on its own.
|
||||
func BackupCreate(d Deps, kind string) (string, error) { return backupCreate(d, kind) }
|
||||
Reference in new issue
Block a user