Dawnline 0.20.0
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions

- **Backend rewritten in Go.** One daemon, `dawnlined`, replaces the whole ucode backend: the rpcd plugin, the agent, the CLI and the updater. The package no longer depends on ucode.
- **Per-architecture builds.** The release carries the backend for arm64, ARMv7, mips, mipsle and amd64; the router fetches its own after the package is installed and checks its sha256. Dawnline does not run on other architectures (older ARM, 32-bit x86, RISC-V).
- **Faster on the router.** A firewall sync takes 0.06 s instead of 0.9 s; a full apply takes about 300 ms. The portal is served by the daemon itself, without a CGI process per request.
- **Live pages.** The admin and the portal follow a live stream from the router instead of re-fetching the whole state every 30 seconds.
- **Admin redesigned in the portal's style.** Three sections (Overview · Log · Settings); a server is a page of its own with People, Places and Settings; a device opens as a dialog over its person.
- **Changes wait for «Apply».** Every setting, and a person's access, is staged in one dock before it reaches the router.
- **Configuration rollout.** After a change that makes devices fetch their configuration again, the admin shows how many have fetched it and who has not. Every setting shows beforehand (⟳N) how many devices it will touch.
- **Waiting for a decision.** Overview and People put first the people who need one (access ended, limit reached, a device that does not connect), each with its one action a tap away.
- **Bulk actions on people.** «Select» (or a long press on a card) picks several people to extend by a month, pause or resume at once.
- **Undo.** An action done from a card or a request is undone from its toast.
- **Update dialogs redesigned.** An update is one view from its question to its result; Dawnline's own update runs in it like every other, and the page reloads itself once the new version answers. Other versions are a list, each with its notes and Install on a page of its own.
- **Site to site: networks behind a router.** A router connected as a device can have networks behind it (up to eight, /16 to /30): the server routes them and applies the router's access to them. A network that shares addresses with a tunnel, the router's own networks or another site is refused by name.
- **Routers from the portal.** People can add a router and name its networks on their own page; the admin accepts or declines with a reason, or lets them through without asking.
- **Check access and Health know the networks behind routers.**
- **Usage limits reworked.** A monthly limit no longer follows the calendar. A person's month starts with their first traffic and ends a month later; the next one starts only when they connect again, so time away uses up none of it. Before, everyone's limit reset on the 1st.
- Extra data lasts until that person's month ends; «Reset usage» starts the month anew.
- A limit is for a month or for the whole term; the admin and the portal show when it resets.
- Charts that said «this month» show the last 30 days.
- After the update nobody loses what was counted: the month that was running becomes the person's first and ends on the 1st.
- **Access by time of day removed.** The Hours limit is gone from the admin, the portal and the backend; saved hours are dropped.
- **Expiry simplified.** An end date always stops access, with a heads-up three days before; the per-person choice is gone.
- **No false disconnects from sleeping phones.** A phone or a tablet stays online through ten minutes of silence.
- **Mobile Safari fixes.** Pressing a control no longer sends the page to its top; tab tracks that do not fit scroll; switches animate after a press.
This commit is contained in:
Aemeath committed 2026-11-13 09:43:50 +03:00
1 parent f8130e39a8
commit ac90575b0b
788 files changed
+139244 -41720

No files matched your search

+375
View File
@@ -0,0 +1,375 @@
// Package chores is what Dawnline does on its own once a minute besides keeping the router to the
// settings (the ucode agent's steps "People", "Personal portal", "Router", "Backup" and "updates"):
// people's states that change on their own go to the activity log, an end date is announced when its last
// days begin, a renewed or expiring certificate and a changed public address are told in the service log,
// the settings are backed up at night, and an automatic update is started when one is due.
package chores
import (
"crypto/md5"
"crypto/x509"
"encoding/hex"
"encoding/pem"
"fmt"
"os"
"regexp"
"strconv"
"strings"
"time"
"github.com/tickcount/luci-app-dawnline/backend/files"
"github.com/tickcount/luci-app-dawnline/backend/journal"
"github.com/tickcount/luci-app-dawnline/backend/model"
"github.com/tickcount/luci-app-dawnline/backend/servicelog"
"github.com/tickcount/luci-app-dawnline/backend/state"
"github.com/tickcount/luci-app-dawnline/backend/usage"
"github.com/tickcount/luci-app-dawnline/backend/zone"
)
// ExpireWarn is how many days before an end date it is announced.
const ExpireWarn = 3
// Chores does one router's. What it remembers between ticks starts as "nothing seen yet".
type Chores struct {
RunDir, DataDir, Config string
Log *servicelog.Log
// PortalCert is the certificate the portal uses and whether it is Dawnline's own self-signed one.
PortalCert func(s model.Settings) (path string, selfSigned bool)
// WanAddress is the router's public IPv4 as a server sees it ("": none).
WanAddress func(srv *model.Server) string
// Backup saves the settings (a kind of backup: "auto"); Reload has the service read its settings again.
Backup func(kind string) (string, error)
Reload func()
// Updating tells whether an update job runs; Update starts the updater on its own.
Updating func() bool
Update func()
// FlushPlaces puts the places' cache on flash (nil: there is none to keep).
FlushPlaces func()
// Now is the moment (nil: the clock).
Now func() time.Time
states map[string]string
certTime int64
certSeen bool
certCheck int64
backupDay int
notes map[string]any
rateAt int64
rateServers map[string]usage.Pair
ratePeople map[string]usage.Pair
flushed, logFlushed int64
failed map[string]string
}
func (c *Chores) now() time.Time {
if c.Now != nil {
return c.Now()
}
return time.Now()
}
func (c *Chores) slog(src, level, msg, detail, who string) {
if c.Log != nil {
c.Log.Write(src, level, msg, detail, who, servicelog.Extra{})
}
}
// What the service log was already told. On flash: the public address often changes with a reboot
// (PPPoE), and that is when it matters. Written rarely.
func (c *Chores) note(k string, v any) {
c.notes[k] = v
os.MkdirAll(c.DataDir, 0o700)
files.WriteJSON(c.DataDir+"/notes.json", c.notes, 0o600)
}
var stateReason = map[string]string{"paused": "paused by admin", "expired": "end date reached", "quota": "data limit reached",
"no_server": "no server", "server_off": "server turned off", "active": "active"}
// StateEvent is the event for a person's state moving on its own from prev to st, or nil. The admin's own
// pauses and a server turned off or on are logged when they happen, so nothing here; nothing either
// without a state before. The data limit has its own events; everything else is paused or resumed with
// the reason.
func StateEvent(person, prev, st string) journal.Event {
byAdmin := func(x string) bool { return x == "paused" || x == "server_off" }
if prev == "" || prev == st || byAdmin(prev) || byAdmin(st) {
return nil
}
if st == "quota" {
return journal.Event{"type": "quota_reached", "person": person}
}
if prev == "quota" && st == "active" {
return journal.Event{"type": "quota_restored", "person": person}
}
kind := "paused"
if st == "active" {
kind = "resumed"
}
reason := stateReason[st]
if reason == "" {
reason = st
}
return journal.Event{"type": kind, "person": person, "detail": reason}
}
// people is the states that changed on their own, and the end dates whose last days begin (once an end
// date; not on a clock that is not set).
func (c *Chores) people(m *model.Model, u *usage.Usage) []journal.Event {
now := c.now()
ts := now.Unix()
var events []journal.Event
states := map[string]string{}
for _, id := range m.PersonIDs() {
states[id] = state.Of(m, m.People[id], u, ts)
if e := StateEvent(id, c.states[id], states[id]); e != nil && c.states != nil {
events = append(events, e)
}
}
c.states = states
if now.In(zone.Local()).Year() < 2025 {
return events
}
for _, id := range m.PersonIDs() {
p := m.People[id]
end, ok := state.DateEnd(p.Expires)
if p.Expires == "" || !ok || end <= ts {
continue
}
left := (end-ts)/86400 + 1
if left <= ExpireWarn && c.notes["warned:"+id] != p.Expires {
c.note("warned:"+id, p.Expires)
days := "days"
if left == 1 {
days = "day"
}
events = append(events, journal.Event{"type": "expiring", "person": id, "detail": fmt.Sprintf("%d %s", left, days)})
}
}
return events
}
// certEnd is when a certificate ends ("Jan 1 2027", and the moment), from its file.
func certEnd(path string) (string, time.Time, bool) {
raw, err := os.ReadFile(path)
if err != nil {
return "", time.Time{}, false
}
block, _ := pem.Decode(raw)
if block == nil {
return "", time.Time{}, false
}
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return "", time.Time{}, false
}
end := cert.NotAfter.UTC()
return fmt.Sprintf("%s %d %d", end.Format("Jan"), end.Day(), end.Year()), end, true
}
// cert tells a renewed certificate (and has the service take it), and one that runs out within a week
// (ACME did not renew it): once a certificate, looked at hourly.
func (c *Chores) cert(m *model.Model) {
if !m.Settings.PortalEnabled || c.PortalCert == nil {
return
}
path, self := c.PortalCert(m.Settings)
st, err := os.Stat(path)
var mt int64
if err == nil {
mt = st.ModTime().Unix()
}
if c.certSeen && c.certTime != 0 && mt != 0 && mt != c.certTime {
until := "?"
if s, _, ok := certEnd(path); ok {
until = s
}
c.slog("portal", "up", "certificate renewed · valid until "+until, "", "Personal portal")
if c.Reload != nil {
c.Reload()
}
}
c.certTime, c.certSeen = mt, true
now := c.now().Unix()
if mt == 0 || self || now-c.certCheck < 3600 {
return
}
c.certCheck = now
warned, _ := c.notes["cert_warned"].(float64)
s, end, ok := certEnd(path)
if int64(warned) != mt && (!ok || end.Unix()-now < 604800) {
msg := "certificate expires within 7 days"
if ok {
msg = "certificate expires " + s
}
c.slog("portal", "error", msg, "Browsers will refuse the portal after that. Renew it with ACME, or choose another certificate in Settings.", "Personal portal")
c.note("cert_warned", float64(mt))
}
}
// wan tells that the router's public IPv4 changed. Devices whose configuration holds the address itself
// (a server's public address is an IP) cannot connect until they get a new one; with a hostname only the
// DNS record must follow.
func (c *Chores) wan(m *model.Model) {
if c.WanAddress == nil {
return
}
ip := ""
for _, id := range m.ServerIDs() {
if srv := m.Servers[id]; srv.Enabled && srv.PrivateKey != "" {
ip = c.WanAddress(srv)
break
}
}
was, _ := c.notes["wan"].(string)
if ip == "" || ip == was {
return
}
c.note("wan", ip)
if was == "" {
return
}
var stuck []string
for _, id := range m.ServerIDs() {
if srv := m.Servers[id]; m.EndpointHost(srv) == was {
stuck = append(stuck, srv.Name)
}
}
msg := fmt.Sprintf("public address changed · %s → %s", was, ip)
switch len(stuck) {
case 0:
c.slog("dawnline", "info", msg, "", "Router")
case 1:
c.slog("dawnline", "error", msg, stuck[0]+" still gives devices the old address: set the public address in the server settings (a DDNS name keeps working), then devices need new configurations.", "Router")
default:
c.slog("dawnline", "error", msg, strings.Join(stuck, ", ")+" still give devices the old address: set the public address in the server settings (a DDNS name keeps working), then devices need new configurations.", "Router")
}
}
// backup saves the settings in the night's third hour, once a day, when they changed since the last one.
func (c *Chores) backup() {
t := c.now().In(zone.Local())
if t.Hour() != 3 || c.backupDay == t.Day() || c.Backup == nil {
return
}
c.backupDay = t.Day()
raw, err := os.ReadFile(c.Config)
if err != nil {
return
}
sum := md5.Sum(raw)
now := hex.EncodeToString(sum[:])
var st struct {
Auto string `json:"auto"`
}
files.ReadJSON(c.DataDir+"/backup-state.json", &st)
if now == st.Auto {
return
}
name, err := c.Backup("auto")
if err != nil {
// tried again tomorrow; meanwhile the admin should know (a full flash, mostly)
c.slog("dawnline", "error", "nightly backup failed: "+err.Error(), "", "Backup")
return
}
files.WriteJSON(c.DataDir+"/backup-state.json", map[string]any{"auto": now}, 0o600)
if c.Log != nil {
if name != "" {
name = ": " + name
}
c.Log.Say("info", "backup", "nightly backup saved"+name)
}
}
var intervals = map[string]int64{"6h": 6 * 3600, "12h": 12 * 3600, "1d": 86400, "7d": 7 * 86400}
var clock = regexp.MustCompile(`^([0-9]{1,2}):([0-9]{2})$`)
// slot is the scheduled slot now is in (up to an hour after the set time, also across midnight).
func slot(pol model.Updates, now time.Time) (time.Time, bool) {
hm := clock.FindStringSubmatch(pol.Time)
if hm == nil {
return time.Time{}, false
}
h, _ := strconv.Atoi(hm[1])
mi, _ := strconv.Atoi(hm[2])
for _, day := range []time.Time{now, now.Add(-24 * time.Hour)} {
d := day.In(zone.Local())
// as ucode's: the day's midnight by the clock's own reading, then the set time
at := time.Unix(day.Unix()-int64(d.Hour()*3600+d.Minute()*60+d.Second())+int64(h*3600+mi*60), 0)
if !now.Before(at) && now.Sub(at) < time.Hour {
return at, true
}
}
return time.Time{}, false
}
// Due is what an automatic run should do now (dawnline.updates due): "install" (the scheduled window, at
// most an hour after the set time, once a slot), "check" (the interval has passed), or "".
func Due(pol model.Updates, installDay string, lastCheck int64, now time.Time) string {
if pol.Auto && pol.Mode == "scheduled" && len(pol.Targets) > 0 {
if at, in := slot(pol, now); in && installDay != at.In(zone.Local()).Format("2006-01-02") {
return "install"
}
}
every := intervals[pol.Interval]
if every == 0 {
every = 86400
}
if pol.Check && now.Unix()-lastCheck >= every {
return "check"
}
return ""
}
// updates starts an automatic check or update when one is due. The updater does the work in its own
// process; this only decides when.
func (c *Chores) updates(m *model.Model) {
if c.Update == nil {
return
}
var st struct {
InstallDay string `json:"install_day"`
Check float64 `json:"check"`
}
files.ReadJSON(c.DataDir+"/update-auto.json", &st)
if Due(m.Settings.Updates, st.InstallDay, int64(st.Check), c.now()) != "" && !(c.Updating != nil && c.Updating()) {
c.Update()
}
}
func (c *Chores) step(who string, quiet bool, fn func()) {
defer func() {
e := recover()
if e == nil {
delete(c.failed, who)
return
}
msg := fmt.Sprint(e)
// a new failure in the service log, a repeat or a quiet step's in syslog alone
if !quiet && c.failed[who] != msg {
c.slog("dawnline", "error", "failed: "+msg, "", who)
} else if c.Log != nil {
c.Log.Say("error", "chores", who+": "+msg)
}
c.failed[who] = msg
}()
fn()
}
// Tick is one minute's chores: the events for the activity log (the caller writes them and tells the
// pages), and everything else done. Each step on its own: a failing one is logged and the others still run.
func (c *Chores) Tick(m *model.Model, u *usage.Usage) []journal.Event {
if c.notes == nil {
c.notes = map[string]any{}
files.ReadJSON(c.DataDir+"/notes.json", &c.notes)
c.failed = map[string]string{}
}
var events []journal.Event
c.step("Speed", true, func() { c.rate(m, u) })
c.step("People", false, func() { events = c.people(m, u) })
c.step("Personal portal", false, func() { c.cert(m) })
c.step("Router", true, func() { c.wan(m) })
c.step("Backup", false, c.backup)
c.step("updates", true, func() { c.updates(m) })
c.step("Logs", true, func() { c.keepLogs(m) })
return events
}
+216
View File
@@ -0,0 +1,216 @@
package chores
import (
"encoding/json"
"errors"
"os"
"strings"
"testing"
"time"
"github.com/tickcount/luci-app-dawnline/backend/model"
"github.com/tickcount/luci-app-dawnline/backend/servicelog"
"github.com/tickcount/luci-app-dawnline/backend/uci"
"github.com/tickcount/luci-app-dawnline/backend/usage"
"github.com/tickcount/luci-app-dawnline/backend/zone"
)
func TestStateEvent(t *testing.T) {
for _, c := range []struct{ prev, st, kind, detail string }{
{"", "expired", "", ""}, {"active", "active", "", ""}, {"active", "paused", "", ""}, {"paused", "active", "", ""}, {"server_off", "active", "", ""},
{"active", "quota", "quota_reached", ""}, {"quota", "active", "quota_restored", ""}, {"quota", "expired", "paused", "end date reached"},
{"active", "odd", "paused", "odd"},
} {
e := StateEvent("p", c.prev, c.st)
kind, _ := e["type"].(string)
detail, _ := e["detail"].(string)
if kind != c.kind || detail != c.detail {
t.Errorf("%s → %s: %v", c.prev, c.st, e)
}
}
}
// The schedule of automatic updates: the hour after the set time, once a slot, also across midnight; a
// check when its interval passed.
func TestDue(t *testing.T) {
loc := zone.Local()
at := func(s string) time.Time {
x, err := time.ParseInLocation("2006-01-02 15:04", s, loc)
if err != nil {
t.Fatal(err)
}
return x
}
pol := model.Updates{Check: true, Interval: "1d", Auto: true, Mode: "scheduled", Time: "23:30", Targets: []string{"app"}}
fresh := at("2026-11-12 23:00").Unix()
for _, c := range []struct {
now, day string
check int64
want string
}{
{"2026-11-12 23:29", "", fresh, ""}, {"2026-11-12 23:30", "", fresh, "install"}, {"2026-11-13 00:29", "", fresh, "install"},
{"2026-11-13 00:30", "", fresh, ""}, {"2026-11-13 00:10", "2026-11-12", fresh, ""}, {"2026-11-13 23:40", "2026-11-12", at("2026-11-13 23:00").Unix(), "install"},
{"2026-11-12 12:00", "", at("2026-11-11 12:00").Unix(), "check"}, {"2026-11-12 12:00", "", at("2026-11-11 12:01").Unix(), ""},
} {
if got := Due(pol, c.day, c.check, at(c.now)); got != c.want {
t.Errorf("at %s (installed %q): %q, not %q", c.now, c.day, got, c.want)
}
}
off := pol
off.Auto, off.Check = false, false
if Due(off, "", 0, at("2026-11-12 23:40")) != "" {
t.Error("due with everything off")
}
now := pol
now.Mode = "immediate"
if Due(now, "", fresh, at("2026-11-12 23:40")) != "" {
t.Error("a scheduled install without a schedule")
}
}
func TestTick(t *testing.T) {
dir := t.TempDir()
text, _ := os.ReadFile("../access/testdata/access.uci")
c, err := uci.Parse(string(text))
if err != nil {
t.Fatal(err)
}
os.WriteFile(dir+"/dawnline", text, 0o600)
os.WriteFile(dir+"/cert", []byte("not a certificate"), 0o644)
clock := time.Date(2026, 11, 12, 2, 59, 0, 0, zone.Local())
wan := "203.0.113.7"
var said []string
backups, reloads, updates := 0, 0, 0
failing := false
ch := &Chores{RunDir: dir + "/run", DataDir: dir + "/data", Config: dir + "/dawnline", Now: func() time.Time { return clock },
Log: &servicelog.Log{RunDir: dir + "/run", Level: func() string { return "info" }, Syslog: func(_, line string) { said = append(said, line) }},
PortalCert: func(model.Settings) (string, bool) { return dir + "/cert", false },
WanAddress: func(*model.Server) string { return wan },
Backup: func(kind string) (string, error) {
if failing {
return "", errors.New("the flash is full")
}
backups++
return kind + "-1.tar.gz", nil
},
Reload: func() { reloads++ }, Update: func() { updates++ }, Updating: func() bool { return false }}
people := c.OfType("person")
if len(people) == 0 {
t.Fatal("the household has nobody")
}
who := people[0].Name
u := usage.Empty()
load := func() *model.Model { return model.Load(c, nil) }
told := func(part string) bool {
for _, l := range said {
if strings.Contains(l, part) {
return true
}
}
return false
}
// the first tick: nothing to compare with, the address noted, the certificate seen; an update check is due
if ev := ch.Tick(load(), u); len(ev) != 0 || backups != 0 || reloads != 0 || updates != 1 {
t.Fatalf("the first tick: %v, %d backups, %d reloads, %d updates", ev, backups, reloads, updates)
}
if !told("certificate expires within 7 days") {
t.Errorf("a certificate that cannot be read is not told: %v", said)
}
// an end date three days off: announced once; passed: a pause for that reason
c.Set(who, "expires", "2026-11-14")
ev := ch.Tick(load(), u)
if len(ev) != 1 || ev[0]["type"] != "expiring" || ev[0]["detail"] != "3 days" || ev[0]["person"] != who {
t.Errorf("the end date's last days: %v", ev)
}
if ev := ch.Tick(load(), u); len(ev) != 0 {
t.Errorf("announced again: %v", ev)
}
c.Set(who, "expires", "2026-11-11")
ev = ch.Tick(load(), u)
if len(ev) != 1 || ev[0]["type"] != "paused" || ev[0]["detail"] != "end date reached" {
t.Errorf("the end date passed: %v", ev)
}
c.Delete(who, "expires")
if ev := ch.Tick(load(), u); len(ev) != 1 || ev[0]["type"] != "resumed" {
t.Errorf("the end date taken away: %v", ev)
}
// the night's third hour: one backup a day, and only of settings that changed
clock = time.Date(2026, 11, 12, 3, 1, 0, 0, zone.Local())
failing = true
ch.Tick(load(), u)
if !told("nightly backup failed: the flash is full") || backups != 0 {
t.Errorf("a failed backup is not told: %v", said)
}
failing = false
clock = clock.Add(24 * time.Hour)
ch.Tick(load(), u)
ch.Tick(load(), u)
clock = clock.Add(24 * time.Hour)
ch.Tick(load(), u)
if backups != 1 || !told("nightly backup saved: auto-1.tar.gz") {
t.Errorf("%d backups: %v", backups, said)
}
// the certificate's file replaced: told, and the service takes it
os.Chtimes(dir+"/cert", clock, clock.Add(-time.Hour))
ch.Tick(load(), u)
if reloads != 1 || !told("certificate renewed") {
t.Errorf("a renewed certificate: %d reloads, %v", reloads, said)
}
// the public address moved, and a server hands out the old one itself
c.Set("home", "endpoint", "203.0.113.7")
wan = "198.51.100.9"
said = nil
ch.Tick(load(), u)
if !told("public address changed · 203.0.113.7 → 198.51.100.9") {
t.Errorf("a moved address is not told: %v", said)
}
notes, _ := os.ReadFile(dir + "/data/notes.json")
if !strings.Contains(string(notes), `"wan":"198.51.100.9"`) || !strings.Contains(string(notes), "warned:"+who) {
t.Errorf("what was told is not kept: %s", notes)
}
if st, _ := os.Stat(dir + "/data/notes.json"); st.Mode().Perm() != 0o600 {
t.Error("the notes are not private")
}
}
// The speed record: a point a minute from the totals, the last thirty a server and fifteen a person; who
// is gone goes from the files.
func TestRate(t *testing.T) {
dir := t.TempDir()
text, _ := os.ReadFile("../access/testdata/access.uci")
c, _ := uci.Parse(string(text))
m := model.Load(c, nil)
who := m.PersonIDs()[0]
srv := m.People[who].Server
clock := time.Unix(1_800_000_000, 0)
ch := &Chores{RunDir: dir, DataDir: dir + "/data", Now: func() time.Time { return clock }}
u := usage.Empty()
u.People[who] = &usage.Record{Total: &usage.Pair{1000, 500}}
os.WriteFile(dir+"/rate.json", []byte(`{"gone":[[1,2,3]]}`), 0o644)
ch.rate(m, u)
for i := 1; i <= 40; i++ {
clock = clock.Add(time.Minute)
u.People[who].Total = &usage.Pair{1000 + float64(i)*6000, 500 + float64(i)*600}
ch.rate(m, u)
}
var hist, people map[string][][3]int64
raw, _ := os.ReadFile(dir + "/rate.json")
json.Unmarshal(raw, &hist)
raw, _ = os.ReadFile(dir + "/rate_people.json")
json.Unmarshal(raw, &people)
last := [3]int64{clock.Unix(), 100, 10}
if len(hist[srv]) != 30 || hist[srv][29] != last || len(people[who]) != 15 || people[who][14] != last || hist["gone"] != nil {
t.Errorf("the record: %d points a server, %d a person, the last %v", len(hist[srv]), len(people[who]), hist[srv])
}
// a counter that went back (a reset) is no negative speed; a long gap is no point
clock = clock.Add(time.Minute)
u.People[who].Total = &usage.Pair{0, 0}
ch.rate(m, u)
clock = clock.Add(time.Hour)
ch.rate(m, u)
raw, _ = os.ReadFile(dir + "/rate.json")
json.Unmarshal(raw, &hist)
if n := len(hist[srv]); n != 30 || hist[srv][n-1] != [3]int64{clock.Add(-time.Hour).Unix(), 0, 0} {
t.Errorf("after a reset and a gap: %v", hist[srv][len(hist[srv])-2:])
}
}
+108
View File
@@ -0,0 +1,108 @@
package chores
import (
"os"
"github.com/tickcount/luci-app-dawnline/backend/files"
"github.com/tickcount/luci-app-dawnline/backend/journal"
"github.com/tickcount/luci-app-dawnline/backend/model"
"github.com/tickcount/luci-app-dawnline/backend/usage"
)
// point is a minute's speed: the moment, bytes a second down and up.
type point [3]int64
// rate keeps the last half hour of each server's speed (rate.json) and the last quarter of each person's
// (rate_people.json), a point a minute, from the counters' totals.
func (c *Chores) rate(m *model.Model, u *usage.Usage) {
ts := c.now().Unix()
servers := map[string]usage.Pair{}
totals := map[string]usage.Pair{}
for _, id := range m.PersonIDs() {
rec := u.People[id]
if rec == nil || rec.Total == nil {
continue
}
totals[id] = *rec.Total
s := servers[m.People[id].Server]
s[0] += rec.Total[0]
s[1] += rec.Total[1]
servers[m.People[id].Server] = s
}
hist := map[string][]point{}
people := map[string][]point{}
files.ReadJSON(c.RunDir+"/rate.json", &hist)
files.ReadJSON(c.RunDir+"/rate_people.json", &people)
per := func(now, was float64, dt int64) int64 {
if now < was {
return 0
}
return int64((now - was) / float64(dt))
}
if dt := ts - c.rateAt; c.rateAt != 0 && dt > 0 && dt < 600 {
for s, v := range servers {
p, had := c.rateServers[s]
if !had {
p = v
}
hist[s] = append(hist[s], point{ts, per(v[0], p[0], dt), per(v[1], p[1], dt)})
if n := len(hist[s]); n > 30 {
hist[s] = hist[s][n-30:]
}
}
for _, id := range m.PersonIDs() {
t, has := totals[id]
p, had := c.ratePeople[id]
if !has || !had {
continue
}
people[id] = append(people[id], point{ts, per(t[0], p[0], dt), per(t[1], p[1], dt)})
if n := len(people[id]); n > 15 {
people[id] = people[id][n-15:]
}
}
}
for s := range hist {
if m.Servers[s] == nil {
delete(hist, s)
}
}
for id := range people {
if m.People[id] == nil {
delete(people, id)
}
}
c.rateAt, c.rateServers, c.ratePeople = ts, servers, totals
os.MkdirAll(c.RunDir, 0o755)
files.WriteJSON(c.RunDir+"/rate.json", hist, 0o644)
files.WriteJSON(c.RunDir+"/rate_people.json", people, 0o644)
}
// flushEvery is how often what lives in memory is put on flash.
const flushEvery = 3600
var logFlush = map[string]int64{"10min": 600, "hourly": 3600}
// keepLogs trims the service log, and puts it, the places and the activity log on flash as often as each
// is meant to be: flash is written rarely.
func (c *Chores) keepLogs(m *model.Model) {
now := c.now().Unix()
if c.flushed == 0 {
c.flushed, c.logFlushed = now, now
}
persist := now-c.flushed >= flushEvery
if persist {
c.flushed = now
if c.FlushPlaces != nil {
c.FlushPlaces()
}
}
if c.Log != nil {
c.Log.Trim(c.DataDir, persist)
}
pol := journal.PolicyOf(m.Settings.Retention)
if pol.Persist == "off" || now-c.logFlushed >= logFlush[pol.Persist] {
journal.Log{Run: c.RunDir, Data: c.DataDir}.Persist(pol, now)
c.logFlushed = now
}
}