Dawnline 0.20.0
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions
- **Backend rewritten in Go.** One daemon, `dawnlined`, replaces the whole ucode backend: the rpcd plugin, the agent, the CLI and the updater. The package no longer depends on ucode. - **Per-architecture builds.** The release carries the backend for arm64, ARMv7, mips, mipsle and amd64; the router fetches its own after the package is installed and checks its sha256. Dawnline does not run on other architectures (older ARM, 32-bit x86, RISC-V). - **Faster on the router.** A firewall sync takes 0.06 s instead of 0.9 s; a full apply takes about 300 ms. The portal is served by the daemon itself, without a CGI process per request. - **Live pages.** The admin and the portal follow a live stream from the router instead of re-fetching the whole state every 30 seconds. - **Admin redesigned in the portal's style.** Three sections (Overview · Log · Settings); a server is a page of its own with People, Places and Settings; a device opens as a dialog over its person. - **Changes wait for «Apply».** Every setting, and a person's access, is staged in one dock before it reaches the router. - **Configuration rollout.** After a change that makes devices fetch their configuration again, the admin shows how many have fetched it and who has not. Every setting shows beforehand (⟳N) how many devices it will touch. - **Waiting for a decision.** Overview and People put first the people who need one (access ended, limit reached, a device that does not connect), each with its one action a tap away. - **Bulk actions on people.** «Select» (or a long press on a card) picks several people to extend by a month, pause or resume at once. - **Undo.** An action done from a card or a request is undone from its toast. - **Update dialogs redesigned.** An update is one view from its question to its result; Dawnline's own update runs in it like every other, and the page reloads itself once the new version answers. Other versions are a list, each with its notes and Install on a page of its own. - **Site to site: networks behind a router.** A router connected as a device can have networks behind it (up to eight, /16 to /30): the server routes them and applies the router's access to them. A network that shares addresses with a tunnel, the router's own networks or another site is refused by name. - **Routers from the portal.** People can add a router and name its networks on their own page; the admin accepts or declines with a reason, or lets them through without asking. - **Check access and Health know the networks behind routers.** - **Usage limits reworked.** A monthly limit no longer follows the calendar. A person's month starts with their first traffic and ends a month later; the next one starts only when they connect again, so time away uses up none of it. Before, everyone's limit reset on the 1st. - Extra data lasts until that person's month ends; «Reset usage» starts the month anew. - A limit is for a month or for the whole term; the admin and the portal show when it resets. - Charts that said «this month» show the last 30 days. - After the update nobody loses what was counted: the month that was running becomes the person's first and ends on the 1st. - **Access by time of day removed.** The Hours limit is gone from the admin, the portal and the backend; saved hours are dropped. - **Expiry simplified.** An end date always stops access, with a heads-up three days before; the per-person choice is gone. - **No false disconnects from sleeping phones.** A phone or a tablet stays online through ten minutes of silence. - **Mobile Safari fixes.** Pressing a control no longer sends the page to its top; tab tracks that do not fit scroll; switches animate after a press.
This commit is contained in:
1 parent
f8130e39a8
commit
ac90575b0b
788 files changed
+139244
-41720
No files matched your search
@@ -0,0 +1,708 @@
|
||||
// Package daemon is the running backend: it joins the parts and keeps them
|
||||
// going until it is told to stop. cmd/dawnlined gives it the router's paths;
|
||||
// a test gives it a stand's.
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
backend "github.com/tickcount/luci-app-dawnline/backend"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/api"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/apply"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/awg"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/bridge"
|
||||
minute "github.com/tickcount/luci-app-dawnline/backend/chores"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/files"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/geo"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/github"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/hub"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/i18n"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/journal"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/methods"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/migrate"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/model"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/netview"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/portal"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/resolve"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/roster"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/servicelog"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/store"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/ubus"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/uci"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/usage"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/view"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/watch"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/watchdog"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/zone"
|
||||
)
|
||||
|
||||
// Options is where the backend finds the router.
|
||||
type Options struct {
|
||||
// Socket is ubusd's.
|
||||
Socket string
|
||||
// RunDir holds runtime state (tmpfs), DataDir kept state (flash), Config the settings file.
|
||||
RunDir, DataDir, Config string
|
||||
// Stream is how pages follow: "sse" or "poll" (api.Info).
|
||||
Stream string
|
||||
// Watch: the backend looks at the tunnels and keeps the router to the settings (always, on a router; a
|
||||
// test stand that only wants the bus and the stream leaves it off).
|
||||
Watch bool
|
||||
// Tunnels reads the tunnels the watcher looks at; nil: the router's own (awg.System). A test stand's are
|
||||
// in its hands.
|
||||
Tunnels *awg.Reader
|
||||
// Portal: the backend serves the personal portal itself (package portal); nil: its own web server does.
|
||||
Portal *Portal
|
||||
// Log gets the lines for the system log.
|
||||
Log func(format string, args ...any)
|
||||
}
|
||||
|
||||
// Portal is what the portal's web server runs with (the init script works it out).
|
||||
type Portal struct {
|
||||
// Port is the HTTPS port, HTTP the plain one beside it (0: none); Cert and Key the certificate's files.
|
||||
Port, HTTP int
|
||||
Cert, Key string
|
||||
// Procs is how many of the ucode portal run at once.
|
||||
Procs int
|
||||
}
|
||||
|
||||
// Run works until ctx ends, then waits for its parts.
|
||||
func Run(ctx context.Context, o Options) {
|
||||
// The router's zone, before anything counts a day in it (OpenWrt keeps it where Go does not look).
|
||||
zone.Load(zone.File)
|
||||
h := hub.New()
|
||||
audience := api.NewAudience()
|
||||
var b *bridge.Bridge
|
||||
// The watcher, when this backend runs it: what the methods it answers itself read.
|
||||
var w *watch.Watcher
|
||||
var more []ubus.Method
|
||||
nets := &homes{}
|
||||
// the portal's own answers, where this backend keeps the router (set with the watcher)
|
||||
var native http.Handler
|
||||
// set when the watcher runs: the topics Go makes itself, told again as the router is at that moment
|
||||
// (after a change of the settings)
|
||||
var publish func()
|
||||
if o.Watch {
|
||||
tunnels := awg.System()
|
||||
if o.Tunnels != nil {
|
||||
tunnels = *o.Tunnels
|
||||
}
|
||||
w = &watch.Watcher{
|
||||
Config: o.Config, RunDir: o.RunDir, DataDir: o.DataDir,
|
||||
DHCP: "/etc/config/dhcp", Leases: "/tmp/dhcp.leases", Zone: zone.File,
|
||||
Tunnels: tunnels, Hub: h, Audience: audience, Log: o.Log,
|
||||
}
|
||||
names := &resolve.Resolver{}
|
||||
// The router is this backend's to keep: its own apply, sync, keeper, chores and watchdog. One file gives
|
||||
// all of it back to the ucode backend, for as long as that one is still shipped: /etc/dawnline/backend-off
|
||||
// (the init script then starts this process without its watcher, and the agent does the work).
|
||||
mine := func() bool {
|
||||
_, err := os.Stat(o.DataDir + "/backend-off")
|
||||
return err != nil
|
||||
}
|
||||
logs := &servicelog.Log{RunDir: o.RunDir, Level: func() string {
|
||||
if world := w.World(); world != nil {
|
||||
return world.M.Settings.LogLevel
|
||||
}
|
||||
return "info"
|
||||
}}
|
||||
started(o, logs)
|
||||
defer ended(o, logs)
|
||||
syncer := &apply.Router{RunDir: o.RunDir, HostsFile: "/tmp/hosts/dawnline", DHCP: "/etc/config/dhcp", Log: logs,
|
||||
Network: "/etc/config/network", Firewall: "/etc/config/firewall",
|
||||
Remember: func(address string) {
|
||||
(store.Store{Config: o.Config, RunDir: o.RunDir}).Transact(func(c *uci.Config, _ *model.Model) error {
|
||||
if c.Section("settings") == nil {
|
||||
c.Add("settings", "settings")
|
||||
}
|
||||
c.Set("settings", "lan_address", address)
|
||||
return nil
|
||||
})
|
||||
},
|
||||
Snapshot: func() *netview.Snapshot { return netview.Take(netview.System(nets.logical)) },
|
||||
Exec: func(name string, args ...string) (string, int) {
|
||||
out, err := exec.Command(name, args...).CombinedOutput()
|
||||
if err == nil {
|
||||
return string(out), 0
|
||||
}
|
||||
if ee, isExit := err.(*exec.ExitError); isExit {
|
||||
return string(out), ee.ExitCode()
|
||||
}
|
||||
return string(out), 127
|
||||
}}
|
||||
// The keeper: once a minute, unless the router is given back to the ucode backend (mine), the runtime
|
||||
// sync, the home-like path's upkeep and the watch on the router's networks. The ucode agent leaves
|
||||
// those steps alone while this process is alive (its beat).
|
||||
keeper := &apply.Keeper{R: syncer, Config: o.Config, Words: i18n.Use(i18n.Dir, ""),
|
||||
Load: func() (*model.Model, string, error) {
|
||||
c, m, err := (store.Store{Config: o.Config, RunDir: o.RunDir}).Read()
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
saved := ""
|
||||
if s := c.Section("settings"); s != nil {
|
||||
saved, _ = s.Get("lan_address")
|
||||
}
|
||||
return m, saved, nil
|
||||
},
|
||||
Usage: func() *usage.Usage {
|
||||
if world := w.World(); world != nil {
|
||||
return world.Usage
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Dump: func() (map[string]awg.Peer, map[string]string) {
|
||||
if world := w.World(); world != nil {
|
||||
return world.Peers, world.Reads
|
||||
}
|
||||
return nil, nil
|
||||
},
|
||||
Tools: func() bool {
|
||||
_, tool := os.Stat("/usr/bin/awg")
|
||||
_, proto := os.Stat("/lib/netifd/proto/amneziawg.sh")
|
||||
return tool == nil && proto == nil
|
||||
},
|
||||
Caps: func() awg.Caps { return awg.SystemCaps(o.DataDir) }}
|
||||
// The other things done once a minute (package chores), with the same marker: what happens to people on
|
||||
// its own goes to the activity log, and their pages are told.
|
||||
var chores struct {
|
||||
deps methods.Deps
|
||||
run *minute.Chores
|
||||
}
|
||||
chores.run = &minute.Chores{RunDir: o.RunDir, DataDir: o.DataDir, Config: o.Config, Log: logs,
|
||||
PortalCert: func(s model.Settings) (string, bool) { return methods.PortalCert(s, o.DataDir) },
|
||||
WanAddress: func(srv *model.Server) string { return netview.Take(netview.System(nets.logical)).WanAddress(srv) },
|
||||
Backup: func(kind string) (string, error) { return methods.BackupCreate(chores.deps, kind) },
|
||||
Reload: func() {
|
||||
cmd := exec.Command("sh", "-c", "/etc/init.d/dawnline reload >/dev/null 2>&1")
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
|
||||
cmd.Start()
|
||||
},
|
||||
Updating: func() bool { return methods.Updating(o.RunDir) },
|
||||
Update: func() {
|
||||
exec.Command("start-stop-daemon", "-S", "-b", "-x", "/usr/libexec/dawnline/updater", "--", "auto").Run()
|
||||
}}
|
||||
// The places for the map: lookups run by themselves, a few at a time; what they find goes onto the
|
||||
// connections logged before it was known, and those people's pages are told.
|
||||
poke := func(people []string) {
|
||||
if len(people) == 0 {
|
||||
return
|
||||
}
|
||||
var waiting []string
|
||||
files.ReadJSON(o.RunDir+"/presence.poke", &waiting)
|
||||
files.WriteJSON(o.RunDir+"/presence.poke", append(waiting, people...), 0o600)
|
||||
}
|
||||
places := &geo.Locator{Store: geo.Store{Run: o.RunDir, Data: o.DataDir}, Get: geo.HTTP,
|
||||
Usage: func() *usage.Usage {
|
||||
if world := w.World(); world != nil {
|
||||
return world.Usage
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Found: func(where map[string]string, devices []string) {
|
||||
journal.Log{Run: o.RunDir, Data: o.DataDir}.SetPlaces(where, time.Now().Unix()-3600)
|
||||
world := w.World()
|
||||
if world == nil {
|
||||
return
|
||||
}
|
||||
var people []string
|
||||
for _, id := range devices {
|
||||
if d := world.M.Devices[id]; d != nil {
|
||||
people = append(people, d.Person)
|
||||
}
|
||||
}
|
||||
poke(people)
|
||||
}}
|
||||
chores.run.FlushPlaces = func() { places.Store.Flush() }
|
||||
go func() {
|
||||
// the addresses the watcher saw connect or move: located at once
|
||||
for {
|
||||
time.Sleep(time.Second)
|
||||
if !mine() {
|
||||
continue
|
||||
}
|
||||
var ips []string
|
||||
if !files.ReadJSON(o.RunDir+"/geo.poke", &ips) {
|
||||
continue
|
||||
}
|
||||
os.Remove(o.RunDir + "/geo.poke")
|
||||
if world := w.World(); world != nil && world.M.Settings.GeoEnabled {
|
||||
places.Ask(world.M.Settings.GeoProvider, ips, nil)
|
||||
}
|
||||
}
|
||||
}()
|
||||
others := func() {
|
||||
world := w.World()
|
||||
if world == nil {
|
||||
return
|
||||
}
|
||||
_, m, err := (store.Store{Config: o.Config, RunDir: o.RunDir}).Read()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
events := chores.run.Tick(m, world.Usage)
|
||||
wan := ""
|
||||
if st := chores.deps.Bus; st != nil {
|
||||
list, _ := st("network.interface.wan", "status", nil)["ipv4-address"].([]any)
|
||||
if len(list) > 0 {
|
||||
first, _ := list[0].(map[string]any)
|
||||
wan, _ = first["address"].(string)
|
||||
}
|
||||
}
|
||||
places.Minute(m.Settings.GeoEnabled, m.Settings.GeoProvider, wan)
|
||||
if len(events) == 0 {
|
||||
return
|
||||
}
|
||||
log := journal.Log{Run: o.RunDir, Data: o.DataDir}
|
||||
if err := log.Add(events, journal.PolicyOf(m.Settings.Retention), journal.NamesOf(m), time.Now().Unix()); err != nil {
|
||||
logs.Say("error", "chores", "the activity log was not written · "+err.Error())
|
||||
}
|
||||
// the watcher moves these people's revisions with its next look, so their pages follow
|
||||
var people []string
|
||||
for _, e := range events {
|
||||
if p, _ := e["person"].(string); p != "" {
|
||||
people = append(people, p)
|
||||
}
|
||||
}
|
||||
poke(people)
|
||||
}
|
||||
go func() {
|
||||
// the first look once the watcher has its own, then every minute
|
||||
for wait := 15 * time.Second; ; wait = time.Minute {
|
||||
time.Sleep(wait)
|
||||
if mine() {
|
||||
beatNow(o)
|
||||
keeper.Tick()
|
||||
others()
|
||||
// the watchdog: what keeps people connected, looked at and repaired
|
||||
func() {
|
||||
defer func() {
|
||||
if e := recover(); e != nil {
|
||||
logs.Say("error", "watchdog", fmt.Sprint(e))
|
||||
}
|
||||
}()
|
||||
m, _, err := keeper.Load()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
facts := methods.WatchdogFacts(chores.deps, m, syncer, func() {
|
||||
if world := w.World(); world != nil {
|
||||
syncer.Apply(keeper.Load, world.Usage, keeper.Words, keeper.Tools(), keeper.Caps())
|
||||
}
|
||||
})
|
||||
watchdog.Pass(o.RunDir+"/watchdog.json", watchdog.Probes(m, facts), m.Settings.Watchdog, methods.Updating(o.RunDir), time.Now().Unix(), logs)
|
||||
}()
|
||||
}
|
||||
}
|
||||
}()
|
||||
deps := methods.Deps{World: func() *view.World { return roster.Complete(w.World(), nets.of, o.DataDir) }, Events: func() []map[string]any { return w.Recent(0) }, Catalogs: i18n.Dir,
|
||||
Home: func(srv *model.Server, p *model.Person) []string { return nets.of(srv, p) }, HostIP: names.HostIP, RunDir: o.RunDir, DataDir: o.DataDir, Config: o.Config, Addresses: nets.addresses,
|
||||
Snapshot: func() *netview.Snapshot { return netview.Take(netview.System(nets.logical)) },
|
||||
Handler: nets.handler,
|
||||
Resolve: names.Addresses,
|
||||
ResetUsage: w.ResetUsage,
|
||||
Teardown: func(m *model.Model) {
|
||||
syncer.Teardown(m)
|
||||
migrate.Router{Store: store.Store{Config: o.Config, RunDir: o.RunDir}, Uhttpd: "/etc/config/uhttpd", CGI: "/www/cgi-bin/dawnline-portal", Log: logs,
|
||||
ReloadUhttpd: func() { exec.Command("/etc/init.d/uhttpd", "reload").Run() }}.UhttpdOnly()
|
||||
// the agent's heartbeat check
|
||||
exec.Command("sh", "-c", "grep -qs /usr/libexec/dawnline/heartbeat /etc/crontabs/root && { sed -i '\\#/usr/libexec/dawnline/heartbeat#d' /etc/crontabs/root; /etc/init.d/cron restart >/dev/null 2>&1; }").Run()
|
||||
},
|
||||
// after the answer is out: this process is one of the service's own
|
||||
Stop: func() {
|
||||
cmd := exec.Command("sh", "-c", "sleep 1; /etc/init.d/dawnline stop >/dev/null 2>&1; /etc/init.d/dawnline disable >/dev/null 2>&1")
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
|
||||
cmd.Start()
|
||||
},
|
||||
ApplyPlan: func() any {
|
||||
c, m, err := (store.Store{Config: o.Config, RunDir: o.RunDir}).Read()
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
saved := ""
|
||||
if s := c.Section("settings"); s != nil {
|
||||
saved, _ = s.Get("lan_address")
|
||||
}
|
||||
_, tool := os.Stat("/usr/bin/awg")
|
||||
_, proto := os.Stat("/lib/netifd/proto/amneziawg.sh")
|
||||
return syncer.ApplyPlan(m, saved, tool == nil && proto == nil, awg.SystemCaps(o.DataDir))
|
||||
},
|
||||
SyncPlan: func() any {
|
||||
world := w.World()
|
||||
_, m, err := (store.Store{Config: o.Config, RunDir: o.RunDir}).Read()
|
||||
if world == nil || err != nil {
|
||||
return nil
|
||||
}
|
||||
return syncer.Plan(m, world.Usage, time.Now().Unix())
|
||||
},
|
||||
Background: func(script string) {
|
||||
cmd := exec.Command("sh", "-c", "( "+script+" ) >/dev/null 2>&1 </dev/null")
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
|
||||
cmd.Start()
|
||||
},
|
||||
Ifup: func(name string) { exec.Command("ifup", name).Run() },
|
||||
Updater: func(lang string, args []string) {
|
||||
cmd := exec.Command("start-stop-daemon", append([]string{"-S", "-b", "-x", "/usr/libexec/dawnline/updater", "--"}, args...)...)
|
||||
if lang != "" {
|
||||
cmd.Env = append(os.Environ(), "DAWNLINE_LANG="+lang)
|
||||
}
|
||||
cmd.Run()
|
||||
},
|
||||
Restart: func(network bool) {
|
||||
script := "/etc/init.d/dawnline enabled || /etc/init.d/dawnline enable; sleep 1; "
|
||||
if network {
|
||||
script += "/etc/init.d/network restart; "
|
||||
}
|
||||
cmd := exec.Command("sh", "-c", script+"/etc/init.d/dawnline restart >/dev/null 2>&1")
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
|
||||
cmd.Start()
|
||||
},
|
||||
// the full apply
|
||||
Apply: func(lang string) map[string]any {
|
||||
// Go's own full apply (unless the router is given back to ucode): the one-time steps,
|
||||
// then network, firewall, the home-like path, the resolver, the reply table and the sync
|
||||
if mine() {
|
||||
if world := w.World(); world != nil {
|
||||
st := store.Store{Config: o.Config, RunDir: o.RunDir}
|
||||
migrate.Router{Store: st, NewCode: methods.NewCode, Uhttpd: "/etc/config/uhttpd", CGI: "/www/cgi-bin/dawnline-portal", Log: logs,
|
||||
ReloadUhttpd: func() { exec.Command("/etc/init.d/uhttpd", "reload").Run() }}.Run()
|
||||
_, tool := os.Stat("/usr/bin/awg")
|
||||
_, proto := os.Stat("/lib/netifd/proto/amneziawg.sh")
|
||||
full := syncer.Apply(func() (*model.Model, string, error) {
|
||||
c, m, err := st.Read()
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
saved := ""
|
||||
if s := c.Section("settings"); s != nil {
|
||||
saved, _ = s.Get("lan_address")
|
||||
}
|
||||
return m, saved, nil
|
||||
}, world.Usage, i18n.Use(i18n.Dir, lang), tool == nil && proto == nil, awg.SystemCaps(o.DataDir))
|
||||
out := map[string]any{"network": full.Network, "firewall": full.Firewall, "like_home": full.LikeHome, "peers": full.Peers, "warning": nil}
|
||||
if full.Warning != "" {
|
||||
out["warning"] = full.Warning
|
||||
}
|
||||
return out
|
||||
}
|
||||
}
|
||||
// switched off while it runs (/etc/dawnline/backend-off without a restart): the ucode backend's
|
||||
// own service applies when it starts
|
||||
return map[string]any{"warning": "The change is saved. Dawnline's backend is switched off: restart the service to put it into effect (/etc/init.d/dawnline restart)."}
|
||||
},
|
||||
// after the answer is out: the reload may restart this very process (its portal's port is in its arguments)
|
||||
Reload: func() {
|
||||
cmd := exec.Command("sh", "-c", "sleep 1; /etc/init.d/dawnline reload >/dev/null 2>&1")
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
|
||||
cmd.Start()
|
||||
},
|
||||
AWG: func() map[string]any { return roster.AwgState(nets.handler) },
|
||||
Store: &store.Store{Config: o.Config, RunDir: o.RunDir},
|
||||
Told: func() {
|
||||
if publish != nil {
|
||||
publish()
|
||||
}
|
||||
},
|
||||
// the runtime sync, and its warning for the admin
|
||||
Sync: func(lang string) any {
|
||||
// Go's own sync (unless the router is given back to ucode): the access sets, the devices
|
||||
// into the tunnels and the hostnames, with no call to ucode
|
||||
if mine() {
|
||||
if world := w.World(); world != nil {
|
||||
if _, m, err := (store.Store{Config: o.Config, RunDir: o.RunDir}).Read(); err == nil {
|
||||
res := syncer.Sync(m, world.Usage, time.Now().Unix(), false, i18n.Use(i18n.Dir, lang))
|
||||
if res.Warning == "" {
|
||||
return nil
|
||||
}
|
||||
return res.Warning
|
||||
}
|
||||
}
|
||||
}
|
||||
return "The change is saved. Dawnline's backend is switched off: restart the service to put it into effect (/etc/init.d/dawnline restart)."
|
||||
},
|
||||
Log: &servicelog.Log{RunDir: o.RunDir, Level: func() string {
|
||||
if world := w.World(); world != nil {
|
||||
return world.M.Settings.LogLevel
|
||||
}
|
||||
return "info"
|
||||
}},
|
||||
RouterWAN: nets.routerWAN,
|
||||
Bus: func(object, method string, args map[string]any) map[string]any {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, err := nets.bus.Call(ctx, object, method, args)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return res
|
||||
},
|
||||
WANOf: nets.wanOf,
|
||||
GitHub: &github.Client{RunDir: o.RunDir},
|
||||
Board: func() map[string]any {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, _ := nets.bus.Call(ctx, "system", "board", nil)
|
||||
return res
|
||||
}}
|
||||
more = methods.List(deps)
|
||||
chores.deps = deps
|
||||
native = methods.PortalHandler(deps, true)
|
||||
}
|
||||
object := api.Object(h, audience, api.Info{Version: backend.Version, Stream: o.Stream, Refresh: func(ctx context.Context) { b.Look(ctx) }, More: more})
|
||||
bus := ubus.NewBus(o.Socket, object)
|
||||
bus.Log = o.Log
|
||||
b = bridge.New(bridge.Bridge{
|
||||
Hub: h, Audience: audience, Log: o.Log,
|
||||
// what the topics are made of besides what the watcher tells of itself: the settings, the
|
||||
// configurations handed out, the servers kept down, the places found
|
||||
Watch: []string{o.Config, o.DataDir + "/issued.json", o.RunDir + "/netifd-blocked.json", o.RunDir + "/geo.json"},
|
||||
})
|
||||
run := []func(){
|
||||
func() { bus.Run(ctx) },
|
||||
func() { api.Broadcast(ctx, h, bus, object) },
|
||||
func() { b.Run(ctx) },
|
||||
}
|
||||
if o.Watch {
|
||||
run = append(run, func() { w.Run(ctx) })
|
||||
// The people as Go makes them, held against each status ucode answers (package roster).
|
||||
nets.bus = bus
|
||||
ro := &roster.Roster{World: w.World, Home: nets.of, DataDir: o.DataDir, RunDir: o.RunDir, Hub: h, Log: o.Log, Keyed: bridge.Split, Recent: w.Recent, Own: nets.own, Iface: nets.ifaceState, Homes: nets.serverHomes, Handler: nets.handler, WAN: nets.routerWAN}
|
||||
// The topics as the router is now: the watcher looks first, so what is told is never what it knew a
|
||||
// second ago. Without that look a page that asked in the backend's first second was told nothing
|
||||
// until Refresh (the watcher had no world yet), and settings changed on disk were told as they were
|
||||
// before (the bridge's second comes before the watcher's).
|
||||
tell := func() bool {
|
||||
w.Look()
|
||||
return ro.Publish()
|
||||
}
|
||||
publish = func() { tell() }
|
||||
// told again when the settings change, when somebody starts to look, and with the minute
|
||||
b.Tell = tell
|
||||
// the moment the watcher saw someone come, go or move, the pages have Go's own people
|
||||
w.Changed = func() { go ro.Publish() }
|
||||
}
|
||||
if p := o.Portal; p != nil {
|
||||
bell := &portal.Bell{}
|
||||
srv := &portal.Server{
|
||||
TLS: []string{fmt.Sprintf(":%d", p.Port)}, Cert: p.Cert, Key: p.Key,
|
||||
People: &portal.Settings{Path: o.Config}, Bell: bell, Log: o.Log,
|
||||
// the portal's answers (methods.Portal)
|
||||
Native: native,
|
||||
}
|
||||
if p.HTTP > 0 {
|
||||
srv.Plain = []string{fmt.Sprintf(":%d", p.HTTP)}
|
||||
}
|
||||
run = append(run, func() { srv.Run(ctx) }, func() { portal.Watch(ctx, bell, o.RunDir, o.DataDir, o.Config) })
|
||||
}
|
||||
var parts sync.WaitGroup
|
||||
for _, part := range run {
|
||||
parts.Add(1)
|
||||
go func() {
|
||||
defer parts.Done()
|
||||
part()
|
||||
}()
|
||||
}
|
||||
parts.Wait()
|
||||
}
|
||||
|
||||
// homes answers which home networks a person reaches, from a look at the router's interfaces that is kept
|
||||
// for a few seconds (every person of a status asks).
|
||||
type homes struct {
|
||||
bus *ubus.Bus
|
||||
mu sync.Mutex
|
||||
snap *netview.Snapshot
|
||||
at time.Time
|
||||
}
|
||||
|
||||
func (h *homes) of(srv *model.Server, p *model.Person) []string {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
if h.snap == nil || time.Since(h.at) > 5*time.Second {
|
||||
h.snap = netview.Take(netview.System(h.logical))
|
||||
h.at = time.Now()
|
||||
}
|
||||
return h.snap.PersonHome(srv, p)
|
||||
}
|
||||
|
||||
// own is the router's own public-side addresses: each server's WAN address, and the wan interface's.
|
||||
func (h *homes) own(m *model.Model) map[string]bool {
|
||||
h.mu.Lock()
|
||||
if h.snap == nil || time.Since(h.at) > 5*time.Second {
|
||||
h.snap = netview.Take(netview.System(h.logical))
|
||||
h.at = time.Now()
|
||||
}
|
||||
snap := h.snap
|
||||
h.mu.Unlock()
|
||||
out := map[string]bool{}
|
||||
for _, srv := range m.Servers {
|
||||
a := snap.WanAddress(srv)
|
||||
if a == "" {
|
||||
a = h.ifaceAddr(srv.WanZone)
|
||||
}
|
||||
if a != "" {
|
||||
out[a] = true
|
||||
}
|
||||
}
|
||||
if a := h.ifaceAddr("wan"); a != "" {
|
||||
out[a] = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// addresses is every IPv4 address of netifd's interfaces, in its order.
|
||||
func (h *homes) addresses() []methods.Address {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, err := h.bus.Call(ctx, "network.interface", "dump", nil)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var out []methods.Address
|
||||
list, _ := res["interface"].([]any)
|
||||
for _, i := range list {
|
||||
m, _ := i.(map[string]any)
|
||||
name, _ := m["interface"].(string)
|
||||
addrs, _ := m["ipv4-address"].([]any)
|
||||
for _, a := range addrs {
|
||||
am, _ := a.(map[string]any)
|
||||
out = append(out, methods.Address{Iface: name, CIDR: fmt.Sprintf("%v/%v", am["address"], am["mask"])})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// wanOf is the IPv4 address of a server's WAN (its interface's, else its zone's netifd interface's); of
|
||||
// netifd's wan for no server.
|
||||
func (h *homes) wanOf(srv *model.Server) string {
|
||||
if srv == nil {
|
||||
return h.ifaceAddr("wan")
|
||||
}
|
||||
if a := netview.Take(netview.System(h.logical)).WanAddress(srv); a != "" {
|
||||
return a
|
||||
}
|
||||
return h.ifaceAddr(srv.WanZone)
|
||||
}
|
||||
|
||||
// handler tells whether netifd lists the amneziawg protocol.
|
||||
func (h *homes) handler() bool {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, err := h.bus.Call(ctx, "network", "get_proto_handlers", nil)
|
||||
_, has := res["amneziawg"]
|
||||
return err == nil && has
|
||||
}
|
||||
|
||||
// routerWAN is the router's WAN address: the first server's with keys, else its zone's interface's.
|
||||
func (h *homes) routerWAN(m *model.Model) string {
|
||||
h.mu.Lock()
|
||||
if h.snap == nil || time.Since(h.at) > 5*time.Second {
|
||||
h.snap = netview.Take(netview.System(h.logical))
|
||||
h.at = time.Now()
|
||||
}
|
||||
snap := h.snap
|
||||
h.mu.Unlock()
|
||||
for _, id := range m.ServerIDs() {
|
||||
srv := m.Servers[id]
|
||||
if srv.PrivateKey == "" {
|
||||
continue
|
||||
}
|
||||
if a := snap.WanAddress(srv); a != "" {
|
||||
return a
|
||||
}
|
||||
if a := h.ifaceAddr(srv.WanZone); a != "" {
|
||||
return a
|
||||
}
|
||||
}
|
||||
return h.ifaceAddr("wan")
|
||||
}
|
||||
|
||||
// ifaceState is whether a netifd interface is up, and for how long.
|
||||
func (h *homes) ifaceState(name string) (bool, int64) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, err := h.bus.Call(ctx, "network.interface."+name, "status", nil)
|
||||
if err != nil {
|
||||
return false, 0
|
||||
}
|
||||
up, _ := res["up"].(bool)
|
||||
var uptime int64
|
||||
switch n := res["uptime"].(type) {
|
||||
case int64:
|
||||
uptime = n
|
||||
case int32:
|
||||
uptime = int64(n)
|
||||
case float64:
|
||||
uptime = int64(n)
|
||||
case int:
|
||||
uptime = int64(n)
|
||||
}
|
||||
return up, uptime
|
||||
}
|
||||
|
||||
// serverHomes is a server's home interfaces with what each reaches.
|
||||
func (h *homes) serverHomes(srv *model.Server) []view.HomeNet {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
if h.snap == nil || time.Since(h.at) > 5*time.Second {
|
||||
h.snap = netview.Take(netview.System(h.logical))
|
||||
h.at = time.Now()
|
||||
}
|
||||
home, _ := h.snap.ServerHome(srv)
|
||||
out := []view.HomeNet{}
|
||||
for _, x := range home {
|
||||
out = append(out, view.HomeNet{Dev: x.Dev, Nets: x.Nets})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ifaceAddr is a netifd interface's first IPv4 address, or "".
|
||||
func (h *homes) ifaceAddr(name string) string {
|
||||
if name == "" {
|
||||
return ""
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, err := h.bus.Call(ctx, "network.interface."+name, "status", nil)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
list, _ := res["ipv4-address"].([]any)
|
||||
if len(list) == 0 {
|
||||
return ""
|
||||
}
|
||||
first, _ := list[0].(map[string]any)
|
||||
a, _ := first["address"].(string)
|
||||
return a
|
||||
}
|
||||
|
||||
// logical asks netifd which interface stands on which device.
|
||||
func (h *homes) logical() map[string]netview.Logical {
|
||||
out := map[string]netview.Logical{}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
res, err := h.bus.Call(ctx, "network.interface", "dump", nil)
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
list, _ := res["interface"].([]any)
|
||||
for _, i := range list {
|
||||
m, _ := i.(map[string]any)
|
||||
dev, _ := m["l3_device"].(string)
|
||||
if dev == "" {
|
||||
dev, _ = m["device"].(string)
|
||||
}
|
||||
if _, seen := out[dev]; dev == "" || seen {
|
||||
continue
|
||||
}
|
||||
name, _ := m["interface"].(string)
|
||||
proto, _ := m["proto"].(string)
|
||||
out[dev] = netview.Logical{Name: name, Proto: proto}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,368 @@
|
||||
//go:build e2e
|
||||
|
||||
// The whole path a page takes, on a real OpenWrt userland: uhttpd's /ubus,
|
||||
// rpcd's sessions and access lists (the package's own ACL file), ubusd and the
|
||||
// backend with its own watcher and topics, over a stand's settings and tunnel.
|
||||
// scripts/backend-container.sh starts the daemons and runs this.
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/tickcount/luci-app-dawnline/backend/awg"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/hub"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/ubus"
|
||||
)
|
||||
|
||||
const web = "http://127.0.0.1:8099/ubus"
|
||||
|
||||
// rpc is a JSON-RPC call through uhttpd, as LuCI's rpc.js makes it. It
|
||||
// returns ubus's status and the answer.
|
||||
func rpc(t *testing.T, sid, object, method string, args map[string]any) (int, map[string]any) {
|
||||
t.Helper()
|
||||
body, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "call", "params": []any{sid, object, method, args}})
|
||||
res, err := http.Post(web, "application/json", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
var out struct {
|
||||
Result []json.RawMessage
|
||||
Error *struct {
|
||||
Code int
|
||||
Message string
|
||||
}
|
||||
}
|
||||
if err := json.NewDecoder(res.Body).Decode(&out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if out.Error != nil {
|
||||
return out.Error.Code, nil
|
||||
}
|
||||
var code int
|
||||
json.Unmarshal(out.Result[0], &code)
|
||||
var data map[string]any
|
||||
if len(out.Result) > 1 {
|
||||
json.Unmarshal(out.Result[1], &data)
|
||||
}
|
||||
return code, data
|
||||
}
|
||||
|
||||
func at(v any, path ...string) any {
|
||||
for _, k := range path {
|
||||
m, _ := v.(map[string]any)
|
||||
v = m[k]
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// The stand's settings: one server, one person, one device.
|
||||
const (
|
||||
standServerKey = "yAnz5TF+lXXJte14tji3zlMNq+hd2rYUIgJBgB3fBmk="
|
||||
standServerPub = "xTIBA5rboUvnH4htodjb6e697QjLERt1NAB4mZqp8Dg="
|
||||
standDeviceKey = "HIgo9xNzJMWLKASShiTqIybxZ0U3wGLiUeJ1PKf8ykw="
|
||||
)
|
||||
|
||||
func standSettings(person string) string {
|
||||
return `
|
||||
config settings 'settings'
|
||||
config server 's1'
|
||||
option name 'Home'
|
||||
option ifname 'dawn0'
|
||||
option subnet '10.77.0.0/24'
|
||||
option private_key '` + standServerKey + `'
|
||||
option public_key '` + standServerPub + `'
|
||||
config person 'p1'
|
||||
option server 's1'
|
||||
option name '` + person + `'
|
||||
config device 'd1'
|
||||
option person 'p1'
|
||||
option name 'Phone'
|
||||
option kind 'phone'
|
||||
option address '10.77.0.2'
|
||||
option public_key '` + standDeviceKey + `'
|
||||
`
|
||||
}
|
||||
|
||||
// tunnel is the stand's one tunnel: its device is there, with the handshake the test gave it.
|
||||
type tunnel struct {
|
||||
mu sync.Mutex
|
||||
handshake int64
|
||||
}
|
||||
|
||||
func (tn *tunnel) reader() *awg.Reader {
|
||||
return &awg.Reader{
|
||||
// as `awg show <ifname> dump` prints: the interface's line, then a peer a line
|
||||
Show: func(string) (string, bool) {
|
||||
tn.mu.Lock()
|
||||
defer tn.mu.Unlock()
|
||||
return standServerKey + "\t" + standServerPub + "\t4500\toff\n" +
|
||||
fmt.Sprintf("%s\t(none)\t203.0.113.7:41821\t10.77.0.2/32\t%d\t0\t0\toff\n", standDeviceKey, tn.handshake), true
|
||||
},
|
||||
Index: func(string) (int, bool) { return 7, true },
|
||||
}
|
||||
}
|
||||
|
||||
func (tn *tunnel) shake(at int64) {
|
||||
tn.mu.Lock()
|
||||
tn.handshake = at
|
||||
tn.mu.Unlock()
|
||||
}
|
||||
|
||||
// follow asks with wait until the documents satisfy `done`, taking the events into them, and returns the
|
||||
// number the page is at and the events of the last answer. The topics are the page's whole snapshot.
|
||||
func follow(t *testing.T, sid, epoch string, seq float64, docs map[string]any, topics []any, what string, done func() bool) (float64, []any) {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(25 * time.Second)
|
||||
for {
|
||||
args := map[string]any{"epoch": epoch, "seq": seq, "timeout": 10}
|
||||
if topics != nil {
|
||||
args["topics"] = topics
|
||||
}
|
||||
code, got := rpc(t, sid, "dawnline", "wait", args)
|
||||
events, _ := got["events"].([]any)
|
||||
if code != 0 || got["docs"] != nil || (len(events) == 0 && time.Now().After(deadline)) {
|
||||
t.Fatalf("wait for %s: %d %v", what, code, got)
|
||||
}
|
||||
take(docs, events)
|
||||
seq = got["seq"].(float64)
|
||||
if done() {
|
||||
return seq, events
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// take applies events to a page's documents, as a page does.
|
||||
func take(docs map[string]any, events []any) {
|
||||
for _, e := range events {
|
||||
topic, _ := at(e, "topic").(string)
|
||||
switch at(e, "op") {
|
||||
case "put":
|
||||
docs[topic] = at(e, "data")
|
||||
case "patch":
|
||||
docs[topic] = hub.Merge(docs[topic], at(e, "data"))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestThroughUhttpd(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
defer cancel()
|
||||
dir := t.TempDir()
|
||||
config := filepath.Join(dir, "config")
|
||||
settings := func(person string) {
|
||||
if err := os.WriteFile(config, []byte(standSettings(person)), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
settings("Myata")
|
||||
// The stand is no router to keep: the minute's work (the access rules, the watchdog) stays away, the
|
||||
// watcher and the topics are the backend's own.
|
||||
if err := os.WriteFile(filepath.Join(dir, "backend-off"), nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A backend that was restarted: its earlier run left who was online, and nobody came or went since, so
|
||||
// the watcher's first look finds nothing to tell of by itself.
|
||||
run := filepath.Join(dir, "run")
|
||||
os.MkdirAll(run, 0o755)
|
||||
if err := os.WriteFile(filepath.Join(run, "presence.json"), []byte(`{"online":{"d1":false},"endpoint":{"d1":"203.0.113.7"},"rev":1,"people":{"p1":1}}`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tn := &tunnel{}
|
||||
began := time.Now()
|
||||
// The backend is stopped, and has stopped, before the test ends: it writes into the stand's folders (its beat,
|
||||
// its presence) and logs through t, and a test that ended under it failed at removing a folder still written to.
|
||||
stopped := make(chan struct{})
|
||||
go func() {
|
||||
defer close(stopped)
|
||||
Run(ctx, Options{Socket: ubus.Socket, RunDir: run, DataDir: dir, Config: config, Stream: "sse", Watch: true, Tunnels: tn.reader(), Log: t.Logf})
|
||||
}()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case <-stopped:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Error("the backend did not stop")
|
||||
}
|
||||
}()
|
||||
for exec.CommandContext(ctx, "ubus", "list", "dawnline").Run() != nil {
|
||||
if ctx.Err() != nil {
|
||||
t.Fatal("the object never appeared on the bus")
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
|
||||
// Without a session rpcd refuses; with one, the read ACL lets the page in.
|
||||
if code, _ := rpc(t, "00000000000000000000000000000000", "dawnline", "hello", nil); code == 0 {
|
||||
t.Fatal("answered without a session")
|
||||
}
|
||||
code, login := rpc(t, "00000000000000000000000000000000", "session", "login", map[string]any{"username": "root", "password": ""})
|
||||
if code != 0 {
|
||||
t.Fatalf("login: %d", code)
|
||||
}
|
||||
sid := login["ubus_rpc_session"].(string)
|
||||
|
||||
code, hello := rpc(t, sid, "dawnline", "hello", map[string]any{})
|
||||
if code != 0 || hello["stream"] != "sse" || hello["epoch"] == "" {
|
||||
t.Fatalf("hello: %d %v", code, hello)
|
||||
}
|
||||
|
||||
// A page that was open while the backend restarted asks at once, before the watcher's first look: its
|
||||
// first question starts the bridge, and the documents come with the bridge's next second. (They once
|
||||
// came only with the next Refresh, half a minute later: the bridge took topics nobody could make yet
|
||||
// for told.)
|
||||
asked := time.Now()
|
||||
code, snap := rpc(t, sid, "dawnline", "sync", map[string]any{})
|
||||
if code != 0 {
|
||||
t.Fatalf("sync: %d", code)
|
||||
}
|
||||
epoch, seq := snap["epoch"].(string), snap["seq"].(float64)
|
||||
docs, _ := snap["docs"].(map[string]any)
|
||||
if docs == nil {
|
||||
docs = map[string]any{}
|
||||
}
|
||||
whole := func() bool {
|
||||
return at(docs, "people", "items", "p1", "name") == "Myata" && at(docs, "servers", "items", "s1") != nil && at(docs, "backend", "ok") == true
|
||||
}
|
||||
if !whole() {
|
||||
seq, _ = follow(t, sid, epoch, seq, docs, nil, "the first documents", whole)
|
||||
}
|
||||
took := time.Since(asked)
|
||||
t.Logf("the page asked %v after the backend started and had its first documents %v later", asked.Sub(began).Round(time.Millisecond), took.Round(time.Millisecond))
|
||||
if took > 3*time.Second {
|
||||
t.Fatalf("the first documents took %v", took)
|
||||
}
|
||||
if at(docs, "people", "items", "p1", "devices", "items", "d1", "online") != false || at(docs, "servers", "order").([]any)[0] != "s1" {
|
||||
t.Fatalf("documents: %v", docs)
|
||||
}
|
||||
|
||||
// An event stream, as a page opens it; on a web server that cannot hold one this part is skipped.
|
||||
stream, lines := openStream(t, ctx, sid)
|
||||
|
||||
// A device comes online: the long poll returns with a patch that says so, and not the people whole.
|
||||
online := func() bool { return at(docs, "people", "items", "p1", "devices", "items", "d1", "online") == true }
|
||||
type result struct {
|
||||
seq float64
|
||||
events []any
|
||||
}
|
||||
waited := make(chan result, 1)
|
||||
go func() {
|
||||
s, events := follow(t, sid, epoch, seq, docs, []any{"people"}, "the device to come online", online)
|
||||
waited <- result{s, events}
|
||||
}()
|
||||
time.Sleep(300 * time.Millisecond)
|
||||
changed := time.Now()
|
||||
tn.shake(time.Now().Unix())
|
||||
var r result
|
||||
select {
|
||||
case r = <-waited:
|
||||
case <-time.After(40 * time.Second):
|
||||
t.Fatal("the device never came online for the page")
|
||||
}
|
||||
var patch any
|
||||
for _, e := range r.events {
|
||||
if at(e, "op") == "patch" && at(e, "data", "items", "p1", "devices", "items", "d1", "online") == true {
|
||||
patch = e
|
||||
}
|
||||
}
|
||||
if patch == nil || at(patch, "data", "items", "p1", "name") != nil || at(patch, "data", "order") != nil {
|
||||
t.Fatalf("patch: %v", r.events)
|
||||
}
|
||||
t.Logf("a device that came online reached the page %v later, as %d bytes", time.Since(changed).Round(time.Millisecond), len(fmt.Sprint(at(patch, "data"))))
|
||||
|
||||
if stream {
|
||||
// The same event came down the stream, with the same number.
|
||||
deadline := time.After(10 * time.Second)
|
||||
for found := false; !found; {
|
||||
select {
|
||||
case l := <-lines:
|
||||
var e map[string]any
|
||||
if strings.HasPrefix(l, "data: ") && json.Unmarshal([]byte(l[6:]), &e) == nil && e["seq"] == at(patch, "seq") {
|
||||
if e["topic"] != "people" || e["op"] != "patch" || e["epoch"] != epoch || !reflect.DeepEqual(e["data"], at(patch, "data")) {
|
||||
t.Fatalf("the stream's event differs: %v", e)
|
||||
}
|
||||
found = true
|
||||
}
|
||||
case <-deadline:
|
||||
t.Fatal("the event never came down the stream")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The meter's points come as they are made, to a page that names it.
|
||||
if code, _ := rpc(t, sid, "dawnline", "watch", map[string]any{"topics": []any{"meter"}}); code != 0 {
|
||||
t.Fatalf("watch: %d", code)
|
||||
}
|
||||
code, got := rpc(t, sid, "dawnline", "wait", map[string]any{"epoch": epoch, "seq": r.seq, "topics": []any{"meter"}, "timeout": 10})
|
||||
events, _ := got["events"].([]any)
|
||||
appended := false
|
||||
for _, e := range events {
|
||||
appended = appended || at(e, "op") == "append"
|
||||
}
|
||||
if code != 0 || !appended {
|
||||
t.Fatalf("meter: %d %v", code, got)
|
||||
}
|
||||
|
||||
// A page reads its own edit: once refresh answers, the stream holds what is on disk, with no waiting
|
||||
// for the watcher's second.
|
||||
settings("Mint")
|
||||
code, stands := rpc(t, sid, "dawnline", "refresh", map[string]any{})
|
||||
if code != 0 || stands["epoch"] != epoch {
|
||||
t.Fatalf("refresh: %d %v", code, stands)
|
||||
}
|
||||
code, got = rpc(t, sid, "dawnline", "sync", map[string]any{"epoch": epoch, "seq": r.seq, "topics": []any{"people"}})
|
||||
events, _ = got["events"].([]any)
|
||||
take(docs, events)
|
||||
if code != 0 || at(docs, "people", "items", "p1", "name") != "Mint" || got["seq"].(float64) < stands["seq"].(float64) {
|
||||
t.Fatalf("after refresh: %d %v (the stream stands at %v)", code, got, stands["seq"])
|
||||
}
|
||||
|
||||
// Arguments are checked at the door.
|
||||
if code, _ := rpc(t, sid, "dawnline", "sync", map[string]any{"seq": -1}); code != int(ubus.StatusInvalidArgument) {
|
||||
t.Fatalf("a negative number was accepted: %d", code)
|
||||
}
|
||||
if code, _ := rpc(t, sid, "dawnline", "sync", map[string]any{"topics": "people"}); code != int(ubus.StatusInvalidArgument) {
|
||||
t.Fatalf("topics that are no list were accepted: %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// openStream subscribes through uhttpd and returns the stream's lines.
|
||||
func openStream(t *testing.T, ctx context.Context, sid string) (bool, <-chan string) {
|
||||
req, _ := http.NewRequestWithContext(ctx, "GET", web+"/subscribe/dawnline", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+sid)
|
||||
res, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ct := res.Header.Get("Content-Type"); !strings.HasPrefix(ct, "text/event-stream") {
|
||||
t.Logf("no event stream from this web server (%s): %s", res.Status, ct)
|
||||
res.Body.Close()
|
||||
return false, nil
|
||||
}
|
||||
lines := make(chan string, 64)
|
||||
go func() {
|
||||
defer res.Body.Close()
|
||||
sc := bufio.NewScanner(res.Body)
|
||||
sc.Buffer(make([]byte, 0, 64<<10), 2<<20)
|
||||
for sc.Scan() {
|
||||
select {
|
||||
case lines <- sc.Text():
|
||||
default:
|
||||
}
|
||||
}
|
||||
}()
|
||||
return true, lines
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
backend "github.com/tickcount/luci-app-dawnline/backend"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/geo"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/journal"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/servicelog"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/store"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/updater"
|
||||
)
|
||||
|
||||
// The beat is the time of the backend's last minute: the Health page calls it fine while it is fresh, and
|
||||
// a beat left behind by a backend that is gone tells the next start that the last one did not end well.
|
||||
const beatFile = "/agent.beat"
|
||||
|
||||
func readTrim(path string) string {
|
||||
raw, _ := os.ReadFile(path)
|
||||
return strings.Trim(string(raw), " \t\r\n\v\f")
|
||||
}
|
||||
|
||||
// started says in the service log why this start: an update that was rolled back, a new version, a crash
|
||||
// (the beat outlives a backend that died), a restart (the init script notes a stop), a boot, or a plain
|
||||
// start. It also puts back the service log a reboot took from memory, and remembers the kernel an
|
||||
// AmneziaWG module Dawnline did not install is for.
|
||||
func started(o Options, logs *servicelog.Log) {
|
||||
os.MkdirAll(o.RunDir, 0o755)
|
||||
_, beat := os.Stat(o.RunDir + beatFile)
|
||||
crashed := beat == nil
|
||||
stopped, _ := strconv.ParseInt(readTrim(o.RunDir+"/agent.stopped"), 10, 64)
|
||||
os.Remove(o.RunDir + "/agent.stopped")
|
||||
was := readTrim(o.DataDir + "/version")
|
||||
v := backend.Version
|
||||
if was != v {
|
||||
os.MkdirAll(o.DataDir, 0o700)
|
||||
os.WriteFile(o.DataDir+"/version", []byte(v), 0o644)
|
||||
}
|
||||
// the service log lives in memory: after a reboot, what flash kept of it
|
||||
if _, err := os.Stat(o.RunDir + "/service.log"); err != nil {
|
||||
if raw, err := os.ReadFile(o.DataDir + "/service.log"); err == nil {
|
||||
os.WriteFile(o.RunDir+"/service.log", raw, 0o600)
|
||||
}
|
||||
}
|
||||
say := func(level, msg, detail string) {
|
||||
logs.Write("dawnline", level, msg, detail, "Dawnline", servicelog.Extra{})
|
||||
}
|
||||
rolled := readTrim(o.RunDir + "/update-rolled-back")
|
||||
os.Remove(o.RunDir + "/update-rolled-back")
|
||||
uptime := 0.0
|
||||
fmt.Sscan(readTrim("/proc/uptime"), &uptime)
|
||||
switch {
|
||||
// postupdate put this (kept) version back because the new one did not come up
|
||||
case rolled != "":
|
||||
say("error", fmt.Sprintf("update to %s failed its check - rolled back to %s", rolled, v),
|
||||
"The new version did not answer in LuCI or its agent did not start. Try again from Components, or install it with install.sh to see why.")
|
||||
case was != "" && was != v:
|
||||
say("up", fmt.Sprintf("updated · %s → %s", was, v), "")
|
||||
case crashed:
|
||||
// why, when syslog knows: procd, the kernel (out of memory, a fault)
|
||||
why, _ := exec.Command("sh", "-c", `logread -l 400 2>/dev/null | grep -iE "kernel:.*(out of memory|killed process|oom-kill|segfault)|procd:.*dawnline|dawnlined" | tail -n 6`).Output()
|
||||
say("repair", "stopped unexpectedly - started again", strings.Trim(string(why), " \t\r\n\v\f"))
|
||||
case stopped != 0 && time.Now().Unix()-stopped < 300:
|
||||
say("info", "restarted · version "+v, "")
|
||||
case uptime < 600:
|
||||
say("info", "started with the router · version "+v, "")
|
||||
default:
|
||||
say("info", "started · version "+v, "")
|
||||
}
|
||||
updater.Adopt(o.DataDir)
|
||||
beatNow(o)
|
||||
}
|
||||
|
||||
func beatNow(o Options) {
|
||||
os.WriteFile(o.RunDir+beatFile, []byte(strconv.FormatInt(time.Now().Unix(), 10)), 0o644)
|
||||
}
|
||||
|
||||
// ended is a clean stop: what lives in memory goes to flash (the activity log, the service log, the places
|
||||
// found), and the beat goes, so the next start is not taken for a crash. The counters are the watcher's.
|
||||
func ended(o Options, logs *servicelog.Log) {
|
||||
now := time.Now().Unix()
|
||||
if _, m, err := (store.Store{Config: o.Config, RunDir: o.RunDir}).Read(); err == nil {
|
||||
journal.Log{Run: o.RunDir, Data: o.DataDir}.Persist(journal.PolicyOf(m.Settings.Retention), now)
|
||||
}
|
||||
logs.Trim(o.DataDir, true)
|
||||
geo.Store{Run: o.RunDir, Data: o.DataDir}.Flush()
|
||||
os.Remove(o.RunDir + beatFile)
|
||||
}
|
||||
Reference in new issue
Block a user