Dawnline 0.20.0
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions
- **Backend rewritten in Go.** One daemon, `dawnlined`, replaces the whole ucode backend: the rpcd plugin, the agent, the CLI and the updater. The package no longer depends on ucode. - **Per-architecture builds.** The release carries the backend for arm64, ARMv7, mips, mipsle and amd64; the router fetches its own after the package is installed and checks its sha256. Dawnline does not run on other architectures (older ARM, 32-bit x86, RISC-V). - **Faster on the router.** A firewall sync takes 0.06 s instead of 0.9 s; a full apply takes about 300 ms. The portal is served by the daemon itself, without a CGI process per request. - **Live pages.** The admin and the portal follow a live stream from the router instead of re-fetching the whole state every 30 seconds. - **Admin redesigned in the portal's style.** Three sections (Overview · Log · Settings); a server is a page of its own with People, Places and Settings; a device opens as a dialog over its person. - **Changes wait for «Apply».** Every setting, and a person's access, is staged in one dock before it reaches the router. - **Configuration rollout.** After a change that makes devices fetch their configuration again, the admin shows how many have fetched it and who has not. Every setting shows beforehand (⟳N) how many devices it will touch. - **Waiting for a decision.** Overview and People put first the people who need one (access ended, limit reached, a device that does not connect), each with its one action a tap away. - **Bulk actions on people.** «Select» (or a long press on a card) picks several people to extend by a month, pause or resume at once. - **Undo.** An action done from a card or a request is undone from its toast. - **Update dialogs redesigned.** An update is one view from its question to its result; Dawnline's own update runs in it like every other, and the page reloads itself once the new version answers. Other versions are a list, each with its notes and Install on a page of its own. - **Site to site: networks behind a router.** A router connected as a device can have networks behind it (up to eight, /16 to /30): the server routes them and applies the router's access to them. A network that shares addresses with a tunnel, the router's own networks or another site is refused by name. - **Routers from the portal.** People can add a router and name its networks on their own page; the admin accepts or declines with a reason, or lets them through without asking. - **Check access and Health know the networks behind routers.** - **Usage limits reworked.** A monthly limit no longer follows the calendar. A person's month starts with their first traffic and ends a month later; the next one starts only when they connect again, so time away uses up none of it. Before, everyone's limit reset on the 1st. - Extra data lasts until that person's month ends; «Reset usage» starts the month anew. - A limit is for a month or for the whole term; the admin and the portal show when it resets. - Charts that said «this month» show the last 30 days. - After the update nobody loses what was counted: the month that was running becomes the person's first and ends on the 1st. - **Access by time of day removed.** The Hours limit is gone from the admin, the portal and the backend; saved hours are dropped. - **Expiry simplified.** An end date always stops access, with a heads-up three days before; the per-person choice is gone. - **No false disconnects from sleeping phones.** A phone or a tablet stays online through ten minutes of silence. - **Mobile Safari fixes.** Pressing a control no longer sends the page to its top; tab tracks that do not fit scroll; switches animate after a press.
This commit is contained in:
1 parent
f8130e39a8
commit
ac90575b0b
788 files changed
+139244
-41720
No files matched your search
@@ -0,0 +1,375 @@
|
||||
// Package chores is what Dawnline does on its own once a minute besides keeping the router to the
|
||||
// settings (the ucode agent's steps "People", "Personal portal", "Router", "Backup" and "updates"):
|
||||
// people's states that change on their own go to the activity log, an end date is announced when its last
|
||||
// days begin, a renewed or expiring certificate and a changed public address are told in the service log,
|
||||
// the settings are backed up at night, and an automatic update is started when one is due.
|
||||
package chores
|
||||
|
||||
import (
|
||||
"crypto/md5"
|
||||
"crypto/x509"
|
||||
"encoding/hex"
|
||||
"encoding/pem"
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/tickcount/luci-app-dawnline/backend/files"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/journal"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/model"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/servicelog"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/state"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/usage"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/zone"
|
||||
)
|
||||
|
||||
// ExpireWarn is how many days before an end date it is announced.
|
||||
const ExpireWarn = 3
|
||||
|
||||
// Chores does one router's. What it remembers between ticks starts as "nothing seen yet".
|
||||
type Chores struct {
|
||||
RunDir, DataDir, Config string
|
||||
Log *servicelog.Log
|
||||
// PortalCert is the certificate the portal uses and whether it is Dawnline's own self-signed one.
|
||||
PortalCert func(s model.Settings) (path string, selfSigned bool)
|
||||
// WanAddress is the router's public IPv4 as a server sees it ("": none).
|
||||
WanAddress func(srv *model.Server) string
|
||||
// Backup saves the settings (a kind of backup: "auto"); Reload has the service read its settings again.
|
||||
Backup func(kind string) (string, error)
|
||||
Reload func()
|
||||
// Updating tells whether an update job runs; Update starts the updater on its own.
|
||||
Updating func() bool
|
||||
Update func()
|
||||
// FlushPlaces puts the places' cache on flash (nil: there is none to keep).
|
||||
FlushPlaces func()
|
||||
// Now is the moment (nil: the clock).
|
||||
Now func() time.Time
|
||||
|
||||
states map[string]string
|
||||
certTime int64
|
||||
certSeen bool
|
||||
certCheck int64
|
||||
backupDay int
|
||||
notes map[string]any
|
||||
|
||||
rateAt int64
|
||||
rateServers map[string]usage.Pair
|
||||
ratePeople map[string]usage.Pair
|
||||
flushed, logFlushed int64
|
||||
failed map[string]string
|
||||
}
|
||||
|
||||
func (c *Chores) now() time.Time {
|
||||
if c.Now != nil {
|
||||
return c.Now()
|
||||
}
|
||||
return time.Now()
|
||||
}
|
||||
|
||||
func (c *Chores) slog(src, level, msg, detail, who string) {
|
||||
if c.Log != nil {
|
||||
c.Log.Write(src, level, msg, detail, who, servicelog.Extra{})
|
||||
}
|
||||
}
|
||||
|
||||
// What the service log was already told. On flash: the public address often changes with a reboot
|
||||
// (PPPoE), and that is when it matters. Written rarely.
|
||||
func (c *Chores) note(k string, v any) {
|
||||
c.notes[k] = v
|
||||
os.MkdirAll(c.DataDir, 0o700)
|
||||
files.WriteJSON(c.DataDir+"/notes.json", c.notes, 0o600)
|
||||
}
|
||||
|
||||
var stateReason = map[string]string{"paused": "paused by admin", "expired": "end date reached", "quota": "data limit reached",
|
||||
"no_server": "no server", "server_off": "server turned off", "active": "active"}
|
||||
|
||||
// StateEvent is the event for a person's state moving on its own from prev to st, or nil. The admin's own
|
||||
// pauses and a server turned off or on are logged when they happen, so nothing here; nothing either
|
||||
// without a state before. The data limit has its own events; everything else is paused or resumed with
|
||||
// the reason.
|
||||
func StateEvent(person, prev, st string) journal.Event {
|
||||
byAdmin := func(x string) bool { return x == "paused" || x == "server_off" }
|
||||
if prev == "" || prev == st || byAdmin(prev) || byAdmin(st) {
|
||||
return nil
|
||||
}
|
||||
if st == "quota" {
|
||||
return journal.Event{"type": "quota_reached", "person": person}
|
||||
}
|
||||
if prev == "quota" && st == "active" {
|
||||
return journal.Event{"type": "quota_restored", "person": person}
|
||||
}
|
||||
kind := "paused"
|
||||
if st == "active" {
|
||||
kind = "resumed"
|
||||
}
|
||||
reason := stateReason[st]
|
||||
if reason == "" {
|
||||
reason = st
|
||||
}
|
||||
return journal.Event{"type": kind, "person": person, "detail": reason}
|
||||
}
|
||||
|
||||
// people is the states that changed on their own, and the end dates whose last days begin (once an end
|
||||
// date; not on a clock that is not set).
|
||||
func (c *Chores) people(m *model.Model, u *usage.Usage) []journal.Event {
|
||||
now := c.now()
|
||||
ts := now.Unix()
|
||||
var events []journal.Event
|
||||
states := map[string]string{}
|
||||
for _, id := range m.PersonIDs() {
|
||||
states[id] = state.Of(m, m.People[id], u, ts)
|
||||
if e := StateEvent(id, c.states[id], states[id]); e != nil && c.states != nil {
|
||||
events = append(events, e)
|
||||
}
|
||||
}
|
||||
c.states = states
|
||||
if now.In(zone.Local()).Year() < 2025 {
|
||||
return events
|
||||
}
|
||||
for _, id := range m.PersonIDs() {
|
||||
p := m.People[id]
|
||||
end, ok := state.DateEnd(p.Expires)
|
||||
if p.Expires == "" || !ok || end <= ts {
|
||||
continue
|
||||
}
|
||||
left := (end-ts)/86400 + 1
|
||||
if left <= ExpireWarn && c.notes["warned:"+id] != p.Expires {
|
||||
c.note("warned:"+id, p.Expires)
|
||||
days := "days"
|
||||
if left == 1 {
|
||||
days = "day"
|
||||
}
|
||||
events = append(events, journal.Event{"type": "expiring", "person": id, "detail": fmt.Sprintf("%d %s", left, days)})
|
||||
}
|
||||
}
|
||||
return events
|
||||
}
|
||||
|
||||
// certEnd is when a certificate ends ("Jan 1 2027", and the moment), from its file.
|
||||
func certEnd(path string) (string, time.Time, bool) {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return "", time.Time{}, false
|
||||
}
|
||||
block, _ := pem.Decode(raw)
|
||||
if block == nil {
|
||||
return "", time.Time{}, false
|
||||
}
|
||||
cert, err := x509.ParseCertificate(block.Bytes)
|
||||
if err != nil {
|
||||
return "", time.Time{}, false
|
||||
}
|
||||
end := cert.NotAfter.UTC()
|
||||
return fmt.Sprintf("%s %d %d", end.Format("Jan"), end.Day(), end.Year()), end, true
|
||||
}
|
||||
|
||||
// cert tells a renewed certificate (and has the service take it), and one that runs out within a week
|
||||
// (ACME did not renew it): once a certificate, looked at hourly.
|
||||
func (c *Chores) cert(m *model.Model) {
|
||||
if !m.Settings.PortalEnabled || c.PortalCert == nil {
|
||||
return
|
||||
}
|
||||
path, self := c.PortalCert(m.Settings)
|
||||
st, err := os.Stat(path)
|
||||
var mt int64
|
||||
if err == nil {
|
||||
mt = st.ModTime().Unix()
|
||||
}
|
||||
if c.certSeen && c.certTime != 0 && mt != 0 && mt != c.certTime {
|
||||
until := "?"
|
||||
if s, _, ok := certEnd(path); ok {
|
||||
until = s
|
||||
}
|
||||
c.slog("portal", "up", "certificate renewed · valid until "+until, "", "Personal portal")
|
||||
if c.Reload != nil {
|
||||
c.Reload()
|
||||
}
|
||||
}
|
||||
c.certTime, c.certSeen = mt, true
|
||||
now := c.now().Unix()
|
||||
if mt == 0 || self || now-c.certCheck < 3600 {
|
||||
return
|
||||
}
|
||||
c.certCheck = now
|
||||
warned, _ := c.notes["cert_warned"].(float64)
|
||||
s, end, ok := certEnd(path)
|
||||
if int64(warned) != mt && (!ok || end.Unix()-now < 604800) {
|
||||
msg := "certificate expires within 7 days"
|
||||
if ok {
|
||||
msg = "certificate expires " + s
|
||||
}
|
||||
c.slog("portal", "error", msg, "Browsers will refuse the portal after that. Renew it with ACME, or choose another certificate in Settings.", "Personal portal")
|
||||
c.note("cert_warned", float64(mt))
|
||||
}
|
||||
}
|
||||
|
||||
// wan tells that the router's public IPv4 changed. Devices whose configuration holds the address itself
|
||||
// (a server's public address is an IP) cannot connect until they get a new one; with a hostname only the
|
||||
// DNS record must follow.
|
||||
func (c *Chores) wan(m *model.Model) {
|
||||
if c.WanAddress == nil {
|
||||
return
|
||||
}
|
||||
ip := ""
|
||||
for _, id := range m.ServerIDs() {
|
||||
if srv := m.Servers[id]; srv.Enabled && srv.PrivateKey != "" {
|
||||
ip = c.WanAddress(srv)
|
||||
break
|
||||
}
|
||||
}
|
||||
was, _ := c.notes["wan"].(string)
|
||||
if ip == "" || ip == was {
|
||||
return
|
||||
}
|
||||
c.note("wan", ip)
|
||||
if was == "" {
|
||||
return
|
||||
}
|
||||
var stuck []string
|
||||
for _, id := range m.ServerIDs() {
|
||||
if srv := m.Servers[id]; m.EndpointHost(srv) == was {
|
||||
stuck = append(stuck, srv.Name)
|
||||
}
|
||||
}
|
||||
msg := fmt.Sprintf("public address changed · %s → %s", was, ip)
|
||||
switch len(stuck) {
|
||||
case 0:
|
||||
c.slog("dawnline", "info", msg, "", "Router")
|
||||
case 1:
|
||||
c.slog("dawnline", "error", msg, stuck[0]+" still gives devices the old address: set the public address in the server settings (a DDNS name keeps working), then devices need new configurations.", "Router")
|
||||
default:
|
||||
c.slog("dawnline", "error", msg, strings.Join(stuck, ", ")+" still give devices the old address: set the public address in the server settings (a DDNS name keeps working), then devices need new configurations.", "Router")
|
||||
}
|
||||
}
|
||||
|
||||
// backup saves the settings in the night's third hour, once a day, when they changed since the last one.
|
||||
func (c *Chores) backup() {
|
||||
t := c.now().In(zone.Local())
|
||||
if t.Hour() != 3 || c.backupDay == t.Day() || c.Backup == nil {
|
||||
return
|
||||
}
|
||||
c.backupDay = t.Day()
|
||||
raw, err := os.ReadFile(c.Config)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
sum := md5.Sum(raw)
|
||||
now := hex.EncodeToString(sum[:])
|
||||
var st struct {
|
||||
Auto string `json:"auto"`
|
||||
}
|
||||
files.ReadJSON(c.DataDir+"/backup-state.json", &st)
|
||||
if now == st.Auto {
|
||||
return
|
||||
}
|
||||
name, err := c.Backup("auto")
|
||||
if err != nil {
|
||||
// tried again tomorrow; meanwhile the admin should know (a full flash, mostly)
|
||||
c.slog("dawnline", "error", "nightly backup failed: "+err.Error(), "", "Backup")
|
||||
return
|
||||
}
|
||||
files.WriteJSON(c.DataDir+"/backup-state.json", map[string]any{"auto": now}, 0o600)
|
||||
if c.Log != nil {
|
||||
if name != "" {
|
||||
name = ": " + name
|
||||
}
|
||||
c.Log.Say("info", "backup", "nightly backup saved"+name)
|
||||
}
|
||||
}
|
||||
|
||||
var intervals = map[string]int64{"6h": 6 * 3600, "12h": 12 * 3600, "1d": 86400, "7d": 7 * 86400}
|
||||
var clock = regexp.MustCompile(`^([0-9]{1,2}):([0-9]{2})$`)
|
||||
|
||||
// slot is the scheduled slot now is in (up to an hour after the set time, also across midnight).
|
||||
func slot(pol model.Updates, now time.Time) (time.Time, bool) {
|
||||
hm := clock.FindStringSubmatch(pol.Time)
|
||||
if hm == nil {
|
||||
return time.Time{}, false
|
||||
}
|
||||
h, _ := strconv.Atoi(hm[1])
|
||||
mi, _ := strconv.Atoi(hm[2])
|
||||
for _, day := range []time.Time{now, now.Add(-24 * time.Hour)} {
|
||||
d := day.In(zone.Local())
|
||||
// as ucode's: the day's midnight by the clock's own reading, then the set time
|
||||
at := time.Unix(day.Unix()-int64(d.Hour()*3600+d.Minute()*60+d.Second())+int64(h*3600+mi*60), 0)
|
||||
if !now.Before(at) && now.Sub(at) < time.Hour {
|
||||
return at, true
|
||||
}
|
||||
}
|
||||
return time.Time{}, false
|
||||
}
|
||||
|
||||
// Due is what an automatic run should do now (dawnline.updates due): "install" (the scheduled window, at
|
||||
// most an hour after the set time, once a slot), "check" (the interval has passed), or "".
|
||||
func Due(pol model.Updates, installDay string, lastCheck int64, now time.Time) string {
|
||||
if pol.Auto && pol.Mode == "scheduled" && len(pol.Targets) > 0 {
|
||||
if at, in := slot(pol, now); in && installDay != at.In(zone.Local()).Format("2006-01-02") {
|
||||
return "install"
|
||||
}
|
||||
}
|
||||
every := intervals[pol.Interval]
|
||||
if every == 0 {
|
||||
every = 86400
|
||||
}
|
||||
if pol.Check && now.Unix()-lastCheck >= every {
|
||||
return "check"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// updates starts an automatic check or update when one is due. The updater does the work in its own
|
||||
// process; this only decides when.
|
||||
func (c *Chores) updates(m *model.Model) {
|
||||
if c.Update == nil {
|
||||
return
|
||||
}
|
||||
var st struct {
|
||||
InstallDay string `json:"install_day"`
|
||||
Check float64 `json:"check"`
|
||||
}
|
||||
files.ReadJSON(c.DataDir+"/update-auto.json", &st)
|
||||
if Due(m.Settings.Updates, st.InstallDay, int64(st.Check), c.now()) != "" && !(c.Updating != nil && c.Updating()) {
|
||||
c.Update()
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Chores) step(who string, quiet bool, fn func()) {
|
||||
defer func() {
|
||||
e := recover()
|
||||
if e == nil {
|
||||
delete(c.failed, who)
|
||||
return
|
||||
}
|
||||
msg := fmt.Sprint(e)
|
||||
// a new failure in the service log, a repeat or a quiet step's in syslog alone
|
||||
if !quiet && c.failed[who] != msg {
|
||||
c.slog("dawnline", "error", "failed: "+msg, "", who)
|
||||
} else if c.Log != nil {
|
||||
c.Log.Say("error", "chores", who+": "+msg)
|
||||
}
|
||||
c.failed[who] = msg
|
||||
}()
|
||||
fn()
|
||||
}
|
||||
|
||||
// Tick is one minute's chores: the events for the activity log (the caller writes them and tells the
|
||||
// pages), and everything else done. Each step on its own: a failing one is logged and the others still run.
|
||||
func (c *Chores) Tick(m *model.Model, u *usage.Usage) []journal.Event {
|
||||
if c.notes == nil {
|
||||
c.notes = map[string]any{}
|
||||
files.ReadJSON(c.DataDir+"/notes.json", &c.notes)
|
||||
c.failed = map[string]string{}
|
||||
}
|
||||
var events []journal.Event
|
||||
c.step("Speed", true, func() { c.rate(m, u) })
|
||||
c.step("People", false, func() { events = c.people(m, u) })
|
||||
c.step("Personal portal", false, func() { c.cert(m) })
|
||||
c.step("Router", true, func() { c.wan(m) })
|
||||
c.step("Backup", false, c.backup)
|
||||
c.step("updates", true, func() { c.updates(m) })
|
||||
c.step("Logs", true, func() { c.keepLogs(m) })
|
||||
return events
|
||||
}
|
||||
@@ -0,0 +1,216 @@
|
||||
package chores
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/tickcount/luci-app-dawnline/backend/model"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/servicelog"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/uci"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/usage"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/zone"
|
||||
)
|
||||
|
||||
func TestStateEvent(t *testing.T) {
|
||||
for _, c := range []struct{ prev, st, kind, detail string }{
|
||||
{"", "expired", "", ""}, {"active", "active", "", ""}, {"active", "paused", "", ""}, {"paused", "active", "", ""}, {"server_off", "active", "", ""},
|
||||
{"active", "quota", "quota_reached", ""}, {"quota", "active", "quota_restored", ""}, {"quota", "expired", "paused", "end date reached"},
|
||||
{"active", "odd", "paused", "odd"},
|
||||
} {
|
||||
e := StateEvent("p", c.prev, c.st)
|
||||
kind, _ := e["type"].(string)
|
||||
detail, _ := e["detail"].(string)
|
||||
if kind != c.kind || detail != c.detail {
|
||||
t.Errorf("%s → %s: %v", c.prev, c.st, e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The schedule of automatic updates: the hour after the set time, once a slot, also across midnight; a
|
||||
// check when its interval passed.
|
||||
func TestDue(t *testing.T) {
|
||||
loc := zone.Local()
|
||||
at := func(s string) time.Time {
|
||||
x, err := time.ParseInLocation("2006-01-02 15:04", s, loc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return x
|
||||
}
|
||||
pol := model.Updates{Check: true, Interval: "1d", Auto: true, Mode: "scheduled", Time: "23:30", Targets: []string{"app"}}
|
||||
fresh := at("2026-11-12 23:00").Unix()
|
||||
for _, c := range []struct {
|
||||
now, day string
|
||||
check int64
|
||||
want string
|
||||
}{
|
||||
{"2026-11-12 23:29", "", fresh, ""}, {"2026-11-12 23:30", "", fresh, "install"}, {"2026-11-13 00:29", "", fresh, "install"},
|
||||
{"2026-11-13 00:30", "", fresh, ""}, {"2026-11-13 00:10", "2026-11-12", fresh, ""}, {"2026-11-13 23:40", "2026-11-12", at("2026-11-13 23:00").Unix(), "install"},
|
||||
{"2026-11-12 12:00", "", at("2026-11-11 12:00").Unix(), "check"}, {"2026-11-12 12:00", "", at("2026-11-11 12:01").Unix(), ""},
|
||||
} {
|
||||
if got := Due(pol, c.day, c.check, at(c.now)); got != c.want {
|
||||
t.Errorf("at %s (installed %q): %q, not %q", c.now, c.day, got, c.want)
|
||||
}
|
||||
}
|
||||
off := pol
|
||||
off.Auto, off.Check = false, false
|
||||
if Due(off, "", 0, at("2026-11-12 23:40")) != "" {
|
||||
t.Error("due with everything off")
|
||||
}
|
||||
now := pol
|
||||
now.Mode = "immediate"
|
||||
if Due(now, "", fresh, at("2026-11-12 23:40")) != "" {
|
||||
t.Error("a scheduled install without a schedule")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTick(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
text, _ := os.ReadFile("../access/testdata/access.uci")
|
||||
c, err := uci.Parse(string(text))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
os.WriteFile(dir+"/dawnline", text, 0o600)
|
||||
os.WriteFile(dir+"/cert", []byte("not a certificate"), 0o644)
|
||||
clock := time.Date(2026, 11, 12, 2, 59, 0, 0, zone.Local())
|
||||
wan := "203.0.113.7"
|
||||
var said []string
|
||||
backups, reloads, updates := 0, 0, 0
|
||||
failing := false
|
||||
ch := &Chores{RunDir: dir + "/run", DataDir: dir + "/data", Config: dir + "/dawnline", Now: func() time.Time { return clock },
|
||||
Log: &servicelog.Log{RunDir: dir + "/run", Level: func() string { return "info" }, Syslog: func(_, line string) { said = append(said, line) }},
|
||||
PortalCert: func(model.Settings) (string, bool) { return dir + "/cert", false },
|
||||
WanAddress: func(*model.Server) string { return wan },
|
||||
Backup: func(kind string) (string, error) {
|
||||
if failing {
|
||||
return "", errors.New("the flash is full")
|
||||
}
|
||||
backups++
|
||||
return kind + "-1.tar.gz", nil
|
||||
},
|
||||
Reload: func() { reloads++ }, Update: func() { updates++ }, Updating: func() bool { return false }}
|
||||
people := c.OfType("person")
|
||||
if len(people) == 0 {
|
||||
t.Fatal("the household has nobody")
|
||||
}
|
||||
who := people[0].Name
|
||||
u := usage.Empty()
|
||||
load := func() *model.Model { return model.Load(c, nil) }
|
||||
told := func(part string) bool {
|
||||
for _, l := range said {
|
||||
if strings.Contains(l, part) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
// the first tick: nothing to compare with, the address noted, the certificate seen; an update check is due
|
||||
if ev := ch.Tick(load(), u); len(ev) != 0 || backups != 0 || reloads != 0 || updates != 1 {
|
||||
t.Fatalf("the first tick: %v, %d backups, %d reloads, %d updates", ev, backups, reloads, updates)
|
||||
}
|
||||
if !told("certificate expires within 7 days") {
|
||||
t.Errorf("a certificate that cannot be read is not told: %v", said)
|
||||
}
|
||||
// an end date three days off: announced once; passed: a pause for that reason
|
||||
c.Set(who, "expires", "2026-11-14")
|
||||
ev := ch.Tick(load(), u)
|
||||
if len(ev) != 1 || ev[0]["type"] != "expiring" || ev[0]["detail"] != "3 days" || ev[0]["person"] != who {
|
||||
t.Errorf("the end date's last days: %v", ev)
|
||||
}
|
||||
if ev := ch.Tick(load(), u); len(ev) != 0 {
|
||||
t.Errorf("announced again: %v", ev)
|
||||
}
|
||||
c.Set(who, "expires", "2026-11-11")
|
||||
ev = ch.Tick(load(), u)
|
||||
if len(ev) != 1 || ev[0]["type"] != "paused" || ev[0]["detail"] != "end date reached" {
|
||||
t.Errorf("the end date passed: %v", ev)
|
||||
}
|
||||
c.Delete(who, "expires")
|
||||
if ev := ch.Tick(load(), u); len(ev) != 1 || ev[0]["type"] != "resumed" {
|
||||
t.Errorf("the end date taken away: %v", ev)
|
||||
}
|
||||
// the night's third hour: one backup a day, and only of settings that changed
|
||||
clock = time.Date(2026, 11, 12, 3, 1, 0, 0, zone.Local())
|
||||
failing = true
|
||||
ch.Tick(load(), u)
|
||||
if !told("nightly backup failed: the flash is full") || backups != 0 {
|
||||
t.Errorf("a failed backup is not told: %v", said)
|
||||
}
|
||||
failing = false
|
||||
clock = clock.Add(24 * time.Hour)
|
||||
ch.Tick(load(), u)
|
||||
ch.Tick(load(), u)
|
||||
clock = clock.Add(24 * time.Hour)
|
||||
ch.Tick(load(), u)
|
||||
if backups != 1 || !told("nightly backup saved: auto-1.tar.gz") {
|
||||
t.Errorf("%d backups: %v", backups, said)
|
||||
}
|
||||
// the certificate's file replaced: told, and the service takes it
|
||||
os.Chtimes(dir+"/cert", clock, clock.Add(-time.Hour))
|
||||
ch.Tick(load(), u)
|
||||
if reloads != 1 || !told("certificate renewed") {
|
||||
t.Errorf("a renewed certificate: %d reloads, %v", reloads, said)
|
||||
}
|
||||
// the public address moved, and a server hands out the old one itself
|
||||
c.Set("home", "endpoint", "203.0.113.7")
|
||||
wan = "198.51.100.9"
|
||||
said = nil
|
||||
ch.Tick(load(), u)
|
||||
if !told("public address changed · 203.0.113.7 → 198.51.100.9") {
|
||||
t.Errorf("a moved address is not told: %v", said)
|
||||
}
|
||||
notes, _ := os.ReadFile(dir + "/data/notes.json")
|
||||
if !strings.Contains(string(notes), `"wan":"198.51.100.9"`) || !strings.Contains(string(notes), "warned:"+who) {
|
||||
t.Errorf("what was told is not kept: %s", notes)
|
||||
}
|
||||
if st, _ := os.Stat(dir + "/data/notes.json"); st.Mode().Perm() != 0o600 {
|
||||
t.Error("the notes are not private")
|
||||
}
|
||||
}
|
||||
|
||||
// The speed record: a point a minute from the totals, the last thirty a server and fifteen a person; who
|
||||
// is gone goes from the files.
|
||||
func TestRate(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
text, _ := os.ReadFile("../access/testdata/access.uci")
|
||||
c, _ := uci.Parse(string(text))
|
||||
m := model.Load(c, nil)
|
||||
who := m.PersonIDs()[0]
|
||||
srv := m.People[who].Server
|
||||
clock := time.Unix(1_800_000_000, 0)
|
||||
ch := &Chores{RunDir: dir, DataDir: dir + "/data", Now: func() time.Time { return clock }}
|
||||
u := usage.Empty()
|
||||
u.People[who] = &usage.Record{Total: &usage.Pair{1000, 500}}
|
||||
os.WriteFile(dir+"/rate.json", []byte(`{"gone":[[1,2,3]]}`), 0o644)
|
||||
ch.rate(m, u)
|
||||
for i := 1; i <= 40; i++ {
|
||||
clock = clock.Add(time.Minute)
|
||||
u.People[who].Total = &usage.Pair{1000 + float64(i)*6000, 500 + float64(i)*600}
|
||||
ch.rate(m, u)
|
||||
}
|
||||
var hist, people map[string][][3]int64
|
||||
raw, _ := os.ReadFile(dir + "/rate.json")
|
||||
json.Unmarshal(raw, &hist)
|
||||
raw, _ = os.ReadFile(dir + "/rate_people.json")
|
||||
json.Unmarshal(raw, &people)
|
||||
last := [3]int64{clock.Unix(), 100, 10}
|
||||
if len(hist[srv]) != 30 || hist[srv][29] != last || len(people[who]) != 15 || people[who][14] != last || hist["gone"] != nil {
|
||||
t.Errorf("the record: %d points a server, %d a person, the last %v", len(hist[srv]), len(people[who]), hist[srv])
|
||||
}
|
||||
// a counter that went back (a reset) is no negative speed; a long gap is no point
|
||||
clock = clock.Add(time.Minute)
|
||||
u.People[who].Total = &usage.Pair{0, 0}
|
||||
ch.rate(m, u)
|
||||
clock = clock.Add(time.Hour)
|
||||
ch.rate(m, u)
|
||||
raw, _ = os.ReadFile(dir + "/rate.json")
|
||||
json.Unmarshal(raw, &hist)
|
||||
if n := len(hist[srv]); n != 30 || hist[srv][n-1] != [3]int64{clock.Add(-time.Hour).Unix(), 0, 0} {
|
||||
t.Errorf("after a reset and a gap: %v", hist[srv][len(hist[srv])-2:])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
package chores
|
||||
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/tickcount/luci-app-dawnline/backend/files"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/journal"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/model"
|
||||
"github.com/tickcount/luci-app-dawnline/backend/usage"
|
||||
)
|
||||
|
||||
// point is a minute's speed: the moment, bytes a second down and up.
|
||||
type point [3]int64
|
||||
|
||||
// rate keeps the last half hour of each server's speed (rate.json) and the last quarter of each person's
|
||||
// (rate_people.json), a point a minute, from the counters' totals.
|
||||
func (c *Chores) rate(m *model.Model, u *usage.Usage) {
|
||||
ts := c.now().Unix()
|
||||
servers := map[string]usage.Pair{}
|
||||
totals := map[string]usage.Pair{}
|
||||
for _, id := range m.PersonIDs() {
|
||||
rec := u.People[id]
|
||||
if rec == nil || rec.Total == nil {
|
||||
continue
|
||||
}
|
||||
totals[id] = *rec.Total
|
||||
s := servers[m.People[id].Server]
|
||||
s[0] += rec.Total[0]
|
||||
s[1] += rec.Total[1]
|
||||
servers[m.People[id].Server] = s
|
||||
}
|
||||
hist := map[string][]point{}
|
||||
people := map[string][]point{}
|
||||
files.ReadJSON(c.RunDir+"/rate.json", &hist)
|
||||
files.ReadJSON(c.RunDir+"/rate_people.json", &people)
|
||||
per := func(now, was float64, dt int64) int64 {
|
||||
if now < was {
|
||||
return 0
|
||||
}
|
||||
return int64((now - was) / float64(dt))
|
||||
}
|
||||
if dt := ts - c.rateAt; c.rateAt != 0 && dt > 0 && dt < 600 {
|
||||
for s, v := range servers {
|
||||
p, had := c.rateServers[s]
|
||||
if !had {
|
||||
p = v
|
||||
}
|
||||
hist[s] = append(hist[s], point{ts, per(v[0], p[0], dt), per(v[1], p[1], dt)})
|
||||
if n := len(hist[s]); n > 30 {
|
||||
hist[s] = hist[s][n-30:]
|
||||
}
|
||||
}
|
||||
for _, id := range m.PersonIDs() {
|
||||
t, has := totals[id]
|
||||
p, had := c.ratePeople[id]
|
||||
if !has || !had {
|
||||
continue
|
||||
}
|
||||
people[id] = append(people[id], point{ts, per(t[0], p[0], dt), per(t[1], p[1], dt)})
|
||||
if n := len(people[id]); n > 15 {
|
||||
people[id] = people[id][n-15:]
|
||||
}
|
||||
}
|
||||
}
|
||||
for s := range hist {
|
||||
if m.Servers[s] == nil {
|
||||
delete(hist, s)
|
||||
}
|
||||
}
|
||||
for id := range people {
|
||||
if m.People[id] == nil {
|
||||
delete(people, id)
|
||||
}
|
||||
}
|
||||
c.rateAt, c.rateServers, c.ratePeople = ts, servers, totals
|
||||
os.MkdirAll(c.RunDir, 0o755)
|
||||
files.WriteJSON(c.RunDir+"/rate.json", hist, 0o644)
|
||||
files.WriteJSON(c.RunDir+"/rate_people.json", people, 0o644)
|
||||
}
|
||||
|
||||
// flushEvery is how often what lives in memory is put on flash.
|
||||
const flushEvery = 3600
|
||||
|
||||
var logFlush = map[string]int64{"10min": 600, "hourly": 3600}
|
||||
|
||||
// keepLogs trims the service log, and puts it, the places and the activity log on flash as often as each
|
||||
// is meant to be: flash is written rarely.
|
||||
func (c *Chores) keepLogs(m *model.Model) {
|
||||
now := c.now().Unix()
|
||||
if c.flushed == 0 {
|
||||
c.flushed, c.logFlushed = now, now
|
||||
}
|
||||
persist := now-c.flushed >= flushEvery
|
||||
if persist {
|
||||
c.flushed = now
|
||||
if c.FlushPlaces != nil {
|
||||
c.FlushPlaces()
|
||||
}
|
||||
}
|
||||
if c.Log != nil {
|
||||
c.Log.Trim(c.DataDir, persist)
|
||||
}
|
||||
pol := journal.PolicyOf(m.Settings.Retention)
|
||||
if pol.Persist == "off" || now-c.logFlushed >= logFlush[pol.Persist] {
|
||||
journal.Log{Run: c.RunDir, Data: c.DataDir}.Persist(pol, now)
|
||||
c.logFlushed = now
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user