Dawnline 0.20.0
Build packages / CI (push) Blocked by required conditions
Build packages / Version (push) Waiting to run
CI / Web (format, types, tests, bundles) (push) Blocked by required conditions
CI / ShellCheck (push) Blocked by required conditions
CI / Backend (Go: vet, tests, a real ubusd and uhttpd) (push) Blocked by required conditions
CI / Package files (push) Blocked by required conditions

- **Backend rewritten in Go.** One daemon, `dawnlined`, replaces the whole ucode backend: the rpcd plugin, the agent, the CLI and the updater. The package no longer depends on ucode.
- **Per-architecture builds.** The release carries the backend for arm64, ARMv7, mips, mipsle and amd64; the router fetches its own after the package is installed and checks its sha256. Dawnline does not run on other architectures (older ARM, 32-bit x86, RISC-V).
- **Faster on the router.** A firewall sync takes 0.06 s instead of 0.9 s; a full apply takes about 300 ms. The portal is served by the daemon itself, without a CGI process per request.
- **Live pages.** The admin and the portal follow a live stream from the router instead of re-fetching the whole state every 30 seconds.
- **Admin redesigned in the portal's style.** Three sections (Overview · Log · Settings); a server is a page of its own with People, Places and Settings; a device opens as a dialog over its person.
- **Changes wait for «Apply».** Every setting, and a person's access, is staged in one dock before it reaches the router.
- **Configuration rollout.** After a change that makes devices fetch their configuration again, the admin shows how many have fetched it and who has not. Every setting shows beforehand (⟳N) how many devices it will touch.
- **Waiting for a decision.** Overview and People put first the people who need one (access ended, limit reached, a device that does not connect), each with its one action a tap away.
- **Bulk actions on people.** «Select» (or a long press on a card) picks several people to extend by a month, pause or resume at once.
- **Undo.** An action done from a card or a request is undone from its toast.
- **Update dialogs redesigned.** An update is one view from its question to its result; Dawnline's own update runs in it like every other, and the page reloads itself once the new version answers. Other versions are a list, each with its notes and Install on a page of its own.
- **Site to site: networks behind a router.** A router connected as a device can have networks behind it (up to eight, /16 to /30): the server routes them and applies the router's access to them. A network that shares addresses with a tunnel, the router's own networks or another site is refused by name.
- **Routers from the portal.** People can add a router and name its networks on their own page; the admin accepts or declines with a reason, or lets them through without asking.
- **Check access and Health know the networks behind routers.**
- **Usage limits reworked.** A monthly limit no longer follows the calendar. A person's month starts with their first traffic and ends a month later; the next one starts only when they connect again, so time away uses up none of it. Before, everyone's limit reset on the 1st.
- Extra data lasts until that person's month ends; «Reset usage» starts the month anew.
- A limit is for a month or for the whole term; the admin and the portal show when it resets.
- Charts that said «this month» show the last 30 days.
- After the update nobody loses what was counted: the month that was running becomes the person's first and ends on the 1st.
- **Access by time of day removed.** The Hours limit is gone from the admin, the portal and the backend; saved hours are dropped.
- **Expiry simplified.** An end date always stops access, with a heads-up three days before; the per-person choice is gone.
- **No false disconnects from sleeping phones.** A phone or a tablet stays online through ten minutes of silence.
- **Mobile Safari fixes.** Pressing a control no longer sends the page to its top; tab tracks that do not fit scroll; switches animate after a press.
This commit is contained in:
Aemeath committed 2026-11-13 09:43:50 +03:00
1 parent f8130e39a8
commit ac90575b0b
788 files changed
+139244 -41720

No files matched your search

+887
View File
@@ -0,0 +1,887 @@
{
"ts": 1794460039,
"steps": 400,
"seed": {
"devices": {
"d1": {
"raw": [
500,
700
],
"total": [
9000,
9000
],
"days": {
"2026-11-11": [
1,
2
],
"2026-11-10": [
2,
4
],
"2026-11-09": [
3,
6
],
"2026-11-08": [
4,
8
],
"2026-11-07": [
5,
10
],
"2026-11-06": [
6,
12
],
"2026-11-05": [
7,
14
],
"2026-11-04": [
8,
16
],
"2026-11-03": [
9,
18
],
"2026-11-02": [
10,
20
],
"2026-11-01": [
11,
22
],
"2026-10-31": [
12,
24
],
"2026-10-30": [
13,
26
],
"2026-10-29": [
14,
28
],
"2026-10-28": [
15,
30
],
"2026-10-27": [
16,
32
],
"2026-10-26": [
17,
34
],
"2026-10-25": [
18,
36
],
"2026-10-24": [
19,
38
],
"2026-10-23": [
20,
40
],
"2026-10-22": [
21,
42
],
"2026-10-21": [
22,
44
],
"2026-10-20": [
23,
46
],
"2026-10-19": [
24,
48
],
"2026-10-18": [
25,
50
],
"2026-10-17": [
26,
52
],
"2026-10-16": [
27,
54
],
"2026-10-15": [
28,
56
],
"2026-10-14": [
29,
58
],
"2026-10-13": [
30,
60
],
"2026-10-12": [
31,
62
],
"2026-10-11": [
32,
64
],
"2026-10-10": [
33,
66
],
"2026-10-09": [
34,
68
],
"2026-10-08": [
35,
70
],
"2026-10-07": [
36,
72
],
"2026-10-06": [
37,
74
],
"2026-10-05": [
38,
76
],
"2026-10-04": [
39,
78
],
"2026-10-03": [
40,
80
],
"2026-10-02": [
41,
82
],
"2026-10-01": [
42,
84
],
"2026-09-30": [
43,
86
],
"2026-09-29": [
44,
88
],
"2026-09-28": [
45,
90
],
"2026-09-27": [
46,
92
],
"2026-09-26": [
47,
94
],
"2026-09-25": [
48,
96
],
"2026-09-24": [
49,
98
],
"2026-09-23": [
50,
100
],
"2026-09-22": [
51,
102
],
"2026-09-21": [
52,
104
],
"2026-09-20": [
53,
106
],
"2026-09-19": [
54,
108
],
"2026-09-18": [
55,
110
],
"2026-09-17": [
56,
112
],
"2026-09-16": [
57,
114
],
"2026-09-15": [
58,
116
],
"2026-09-14": [
59,
118
],
"2026-09-13": [
60,
120
],
"2026-09-12": [
61,
122
],
"2026-09-11": [
62,
124
],
"2026-09-10": [
63,
126
],
"2026-09-09": [
64,
128
],
"2026-09-08": [
65,
130
],
"2026-09-07": [
66,
132
],
"2026-09-06": [
67,
134
],
"2026-09-05": [
68,
136
],
"2026-09-04": [
69,
138
],
"2026-09-03": [
70,
140
]
},
"months": {
"2026-10": [
10,
20
],
"2026-09": [
20,
40
],
"2026-08": [
30,
60
],
"2026-07": [
40,
80
],
"2026-06": [
50,
100
],
"2026-05": [
60,
120
],
"2026-04": [
70,
140
],
"2026-03": [
80,
160
],
"2026-02": [
90,
180
],
"2026-01": [
100,
200
],
"2025-12": [
110,
220
],
"2025-11": [
120,
240
],
"2025-10": [
130,
260
],
"2025-09": [
140,
280
],
"2025-08": [
150,
300
],
"2025-07": [
160,
320
],
"2025-06": [
170,
340
],
"2025-05": [
180,
360
],
"2025-04": [
190,
380
],
"2025-03": [
200,
400
],
"2025-02": [
210,
420
],
"2025-01": [
220,
440
],
"2024-12": [
230,
460
],
"2024-11": [
240,
480
],
"2024-10": [
250,
500
],
"2024-09": [
260,
520
],
"2024-08": [
270,
540
],
"2024-07": [
280,
560
],
"2024-06": [
290,
580
],
"2024-05": [
300,
600
]
}
},
"gone": {
"raw": [
1,
1
],
"total": [
5,
5
]
}
},
"people": {
"anna": {
"total": [
5000,
6000
],
"months": {
"2026-11": [
1000,
2000
]
},
"days": {
}
},
"boris": {
"period": {
"start": 1791004039,
"rx": 11,
"tx": 22
},
"total": [
11,
22
]
},
"vera": {
"period": {
"start": 1794028039,
"rx": 100,
"tx": 200
},
"total": [
100,
200
]
}
},
"updated": 0,
"flushed": 0
},
"final": {
"devices": {
"d1": {
"raw": [
0,
0
],
"total": [
1408856,
22080989
],
"days": {
"2026-11-11": [
1,
2
],
"2026-11-10": [
2,
4
],
"2026-11-09": [
3,
6
],
"2026-11-08": [
4,
8
],
"2026-11-07": [
5,
10
],
"2026-11-06": [
6,
12
],
"2026-11-05": [
7,
14
],
"2026-11-04": [
8,
16
],
"2026-11-03": [
9,
18
],
"2026-11-02": [
10,
20
],
"2026-11-01": [
11,
22
],
"2026-10-31": [
12,
24
],
"2026-10-30": [
13,
26
],
"2026-10-29": [
14,
28
],
"2026-10-28": [
15,
30
],
"2026-10-27": [
16,
32
],
"2026-10-26": [
17,
34
],
"2026-10-25": [
18,
36
],
"2026-10-24": [
19,
38
],
"2026-10-23": [
20,
40
],
"2026-10-22": [
21,
42
],
"2026-10-21": [
22,
44
],
"2026-10-20": [
23,
46
],
"2026-10-19": [
24,
48
],
"2026-10-18": [
25,
50
],
"2026-10-17": [
26,
52
],
"2026-10-16": [
27,
54
],
"2026-10-15": [
28,
56
],
"2026-10-14": [
29,
58
],
"2026-11-12": [
1399856,
22071989
]
},
"months": {
"2026-10": [
10,
20
],
"2026-09": [
20,
40
],
"2026-08": [
30,
60
],
"2026-07": [
40,
80
],
"2026-06": [
50,
100
],
"2026-05": [
60,
120
],
"2026-04": [
70,
140
],
"2026-03": [
80,
160
],
"2026-02": [
90,
180
],
"2026-01": [
100,
200
],
"2025-12": [
110,
220
],
"2026-11": [
1399856,
22071989
]
},
"ifindex": 13,
"handshake": 1794457860,
"endpoint": "203.0.113.9",
"ips": {
"203.0.113.1": 1794457514,
"203.0.113.2": 1794457704,
"203.0.113.10": 1794457860,
"203.0.113.4": 1794457591,
"203.0.113.5": 1794457643,
"203.0.113.12": 1794457665,
"203.0.113.11": 1794457704,
"203.0.113.9": 1794457860
}
},
"d2": {
"raw": [
22072,
342972
],
"ifindex": 17,
"total": [
702411,
12930847
],
"months": {
"2026-11": [
702411,
12930847
]
},
"days": {
"2026-11-12": [
702411,
12930847
]
},
"handshake": 1794457781,
"endpoint": "203.0.113.10",
"ips": {
"203.0.113.12": 1794457744,
"203.0.113.11": 1794457118,
"203.0.113.6": 1794457166,
"203.0.113.4": 1794457702,
"203.0.113.9": 1794457336,
"203.0.113.8": 1794457660,
"203.0.113.1": 1794457375,
"203.0.113.10": 1794457781
}
},
"d3": {
"raw": [
7040,
168363
],
"ifindex": 17,
"total": [
247234,
4021581
],
"months": {
"2026-11": [
247234,
4021581
]
},
"days": {
"2026-11-12": [
247234,
4021581
]
},
"handshake": 1794457562,
"endpoint": "203.0.113.1",
"ips": {
"203.0.113.10": 1794457265,
"203.0.113.8": 1794457303,
"203.0.113.6": 1794457462,
"203.0.113.1": 1794457562
}
},
"d4": {
"raw": [
0,
0
],
"ifindex": 18,
"total": [
305250,
6415449
],
"months": {
"2026-11": [
305250,
6415449
]
},
"days": {
"2026-11-12": [
305250,
6415449
]
},
"handshake": 1794458057,
"endpoint": "203.0.113.11",
"ips": {
"203.0.113.2": 1794457539,
"203.0.113.3": 1794457992,
"203.0.113.11": 1794458057,
"203.0.113.6": 1794457370,
"203.0.113.1": 1794457822,
"203.0.113.12": 1794457458,
"203.0.113.9": 1794457923,
"203.0.113.5": 1794457993
}
},
"d5": {
"raw": [
0,
0
],
"ifindex": 17,
"total": [
766080,
13086788
],
"months": {
"2026-11": [
766080,
13086788
]
},
"days": {
"2026-11-12": [
766080,
13086788
]
},
"handshake": 1794458348,
"endpoint": "203.0.113.8",
"ips": {
"203.0.113.8": 1794458348,
"203.0.113.6": 1794458060,
"203.0.113.9": 1794458179,
"203.0.113.3": 1794457958,
"203.0.113.1": 1794458039,
"203.0.113.12": 1794458094,
"203.0.113.2": 1794458127,
"203.0.113.10": 1794458264
}
}
},
"people": {
"anna": {
"total": [
2107267,
35008836
],
"months": {
"2026-11": [
2103267,
35004836
]
},
"days": {
"2026-11-12": [
2102267,
35002836
]
},
"period": {
"start": 1793491200,
"rx": 2103267,
"tx": 35004836
}
},
"boris": {
"period": {
"start": 1794460039,
"rx": 247234,
"tx": 4021581
},
"total": [
247245,
4021603
],
"months": {
"2026-11": [
247234,
4021581
]
},
"days": {
"2026-11-12": [
247234,
4021581
]
}
},
"vera": {
"period": {
"start": 1794028039,
"rx": 1071430,
"tx": 19502437
},
"total": [
1071430,
19502437
],
"months": {
"2026-11": [
1071330,
19502237
]
},
"days": {
"2026-11-12": [
1071330,
19502237
]
}
}
},
"updated": 1794460039,
"flushed": 0,
"hours": {
"2026-11-12 05": [
3420831,
58526654
]
}
},
"hash": 174869661
}
+51
View File
@@ -0,0 +1,51 @@
# The settings the usage trace runs on (testdata/trace.uc): two servers, three people, five devices, a
# short history kept so the trace's old days and months are pruned.
config settings 'settings'
option usage_keep_days '30'
option usage_keep_months '12'
config server 'home'
option ifname 'dawn0'
option subnet '10.77.0.0/24'
option private_key 'yAnz5TF+lXXJte14tji3zlMNq+hd2rYUIgJBgB3fBmk='
config server 'cabin'
option ifname 'dawn1'
option subnet '10.78.0.0/24'
option private_key 'yAnz5TF+lXXJte14tji3zlMNq+hd2rYUIgJBgB3fBmk='
option keepalive '300'
config person 'anna'
option server 'home'
config person 'boris'
option server 'home'
config person 'vera'
option server 'cabin'
config device 'd1'
option person 'anna'
option kind 'phone'
option public_key 'HIgo9xNzJMWLKASShiTqIybxZ0U3wGLiUeJ1PKf8ykw='
config device 'd2'
option person 'anna'
option kind 'computer'
option public_key 'TrMvSoP4jYQlY6RIzBgbssQqY3vxI2Pi+y71lOWWXX0='
config device 'd3'
option person 'boris'
option kind 'computer'
option public_key 'u4pL0000000000000000000000000000000000DyRrM='
config device 'd4'
option person 'vera'
option kind 'router'
option public_key 'Yc0000000000000000000000000000000000ovbrNy4='
config device 'd5'
option person 'vera'
option kind 'tablet'
option public_key 'xTIBA5rboUvnH4htodjb6e697QjLERt1NAB4mZqp8Dg='
+410
View File
@@ -0,0 +1,410 @@
// Package usage is the traffic accounting: the tunnels' counters folded into what each device and person
// used, by the day, the month and the person's limit month, and the router's traffic by the hour. It is
// dawnline.live's collect() and its helpers in Go, rule for rule, on the same file: usage.json is read and
// written by both backends while both exist, so its shape here is the file's.
//
// Pure but for Load and Save: the settings, a look at the tunnels and the clock go in, the counters change.
// The clock is the router's local one (days and months are the router's calendar).
package usage
import (
"os"
"regexp"
"sort"
"strconv"
"strings"
"time"
"github.com/tickcount/luci-app-dawnline/backend/awg"
"github.com/tickcount/luci-app-dawnline/backend/files"
"github.com/tickcount/luci-app-dawnline/backend/model"
"github.com/tickcount/luci-app-dawnline/backend/presence"
"github.com/tickcount/luci-app-dawnline/backend/zone"
)
// hoursKept is how many hours of router-wide traffic are kept (today and yesterday): Overview's bars.
const hoursKept = 48
// addressesKept is how many recent client addresses a device keeps (for the map).
const addressesKept = 8
// Pair is bytes received from a device and sent to it. Numbers are float64 as the file's are read: whole
// to 2^53, far more than a router moves.
type Pair [2]float64
// Period is a person's limit month: when it began and what it counted.
type Period struct {
Start float64 `json:"start"`
Rx float64 `json:"rx"`
Tx float64 `json:"tx"`
}
// Record is what is kept of a device or a person. A device has Raw (its peer's counters at the last
// read), Ifindex (the link they were read on), its last handshake, address and recent addresses; a person
// has Period. Both have the totals and the history.
type Record struct {
Raw *Pair `json:"raw,omitempty"`
Ifindex *int `json:"ifindex,omitempty"`
Total *Pair `json:"total,omitempty"`
Months map[string]Pair `json:"months,omitempty"`
Days map[string]Pair `json:"days,omitempty"`
Period *Period `json:"period,omitempty"`
Handshake float64 `json:"handshake,omitempty"`
Endpoint *string `json:"endpoint,omitempty"`
IPs map[string]float64 `json:"ips,omitempty"`
}
// Usage is the whole file.
type Usage struct {
Devices map[string]*Record `json:"devices"`
People map[string]*Record `json:"people"`
Hours map[string]Pair `json:"hours,omitempty"`
Updated float64 `json:"updated"`
Flushed float64 `json:"flushed"`
}
// Empty is a file with nothing counted yet.
func Empty() *Usage {
return &Usage{Devices: map[string]*Record{}, People: map[string]*Record{}}
}
// Load reads the counters: the live copy in the run directory, else what flash holds. With fromFlash only
// flash counts (a restore put new counters there, and the live ones are the old).
func Load(run, data string, fromFlash bool) *Usage {
u := Empty()
if !fromFlash && files.ReadJSON(run+"/usage.json", u) {
return fill(u)
}
u = Empty()
files.ReadJSON(data+"/usage.json", u)
return fill(u)
}
func fill(u *Usage) *Usage {
if u.Devices == nil {
u.Devices = map[string]*Record{}
}
if u.People == nil {
u.People = map[string]*Record{}
}
return u
}
// Save writes the counters to the run directory (tmpfs), and to flash too when persist is set: flash is
// not worn by a write a minute.
func Save(u *Usage, run, data string, persist bool, now int64) error {
os.MkdirAll(run, 0o755)
if err := files.WriteJSON(run+"/usage.json", u, 0o600); err != nil {
return err
}
if !persist {
return nil
}
u.Flushed = float64(now)
os.MkdirAll(data, 0o700)
return files.WriteJSON(data+"/usage.json", u, 0o600)
}
// Policy is how much history is kept.
type Policy struct {
Days, Months int
}
var whole = regexp.MustCompile(`^-?[0-9]{1,9}$`)
// PolicyOf reads the retention settings as dawnline.live policy() does: a whole number within its range,
// or the default.
func PolicyOf(r model.Retention) Policy {
get := func(v any, dflt, lo, hi int) int {
s, _ := v.(string)
s = strings.TrimSpace(s)
if !whole.MatchString(s) {
return dflt
}
n, _ := strconv.Atoi(s)
if n < lo || n > hi {
return dflt
}
return n
}
return Policy{Days: get(r.UsageKeepDays, 62, 7, 1830), Months: get(r.UsageKeepMonths, 24, 1, 120)}
}
// Live is a device as the last look found it.
type Live struct {
Online bool `json:"online"`
Handshake float64 `json:"handshake"`
Endpoint string `json:"endpoint"`
Present bool `json:"present"`
}
func monthKey(t time.Time) string { return t.Format("2006-01") }
func dayKey(t time.Time) string { return t.Format("2006-01-02") }
func hourKey(t time.Time) string { return t.Format("2006-01-02 15") }
// monthAfter is the same moment a month later on the router's calendar (a day the next month does not
// have is its last one: 31 Jan → 28 or 29 Feb): when a limit month that began at ts ends.
func MonthAfter(ts int64) int64 { return monthAfter(ts) }
func monthAfter(ts int64) int64 {
t := time.Unix(ts, 0).In(zone.Local())
y, mo := t.Year(), t.Month()+1
if mo > 12 {
y, mo = y+1, 1
}
// the last day of that month: the day before the 1st of the one after it
last := time.Date(y, mo+1, 1, 12, 0, 0, 0, zone.Local()).AddDate(0, 0, -1).Day()
return time.Date(y, mo, min(t.Day(), last), t.Hour(), t.Minute(), t.Second(), 0, zone.Local()).Unix()
}
// LimitMonth is the limit month of a person's record that runs at ts, or nil when none does. A limit month
// is not the calendar's: it begins with the person's first traffic and ends a month later; the next one
// begins only with their next traffic, so time they are away uses up none of it. A record from before
// limit months has none: the calendar month that ran then is its first one, with what it had counted, and
// it ends on the 1st as it would have.
func LimitMonth(u *Record, ts int64) *Period {
if u == nil {
return nil
}
p := u.Period
if p == nil {
t := time.Unix(ts, 0).In(zone.Local())
mm, counted := u.Months[monthKey(t)]
if !counted {
return nil
}
first := time.Date(t.Year(), t.Month(), 1, 0, 0, 0, 0, zone.Local()).Unix()
p = &Period{Start: float64(first), Rx: mm[0], Tx: mm[1]}
}
if ts < monthAfter(int64(p.Start)) {
return p
}
return nil
}
func add(bucket map[string]Pair, key string, rx, tx float64) {
v := bucket[key]
bucket[key] = Pair{v[0] + rx, v[1] + tx}
}
// prune keeps the newest `keep` keys of a history (they sort by date).
func prune(bucket map[string]Pair, keep int) {
if len(bucket) <= keep {
return
}
keys := make([]string, 0, len(bucket))
for k := range bucket {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys[:len(keys)-keep] {
delete(bucket, k)
}
}
// account adds traffic to a record's total and its day and month.
func account(rec *Record, rx, tx float64, t time.Time) {
total := Pair{}
if rec.Total != nil {
total = *rec.Total
}
rec.Total = &Pair{total[0] + rx, total[1] + tx}
if rec.Months == nil {
rec.Months = map[string]Pair{}
}
if rec.Days == nil {
rec.Days = map[string]Pair{}
}
add(rec.Months, monthKey(t), rx, tx)
add(rec.Days, dayKey(t), rx, tx)
}
// accountMonth adds a person's traffic to their limit month: the one that runs, or a new one that begins
// with this traffic. Called before account, which a record from before limit months is read by.
func accountMonth(rec *Record, rx, tx float64, ts int64) {
p := LimitMonth(rec, ts)
if p == nil {
p = &Period{Start: float64(ts)}
}
rec.Period = &Period{Start: p.Start, Rx: p.Rx + rx, Tx: p.Tx + tx}
}
// Collect folds a look at the tunnels into the counters and returns each device as that look found it.
// Counters restart from zero when a peer is added again or its tunnel is set up again: a drop is taken as
// a reset, and so is a new link (each record keeps the index of the link its counters were read on: new
// counters that already passed the old ones would be counted short).
//
// A device missing from the look had its counters reset only when its tunnel was read or is down: one the
// tool could not read says nothing, and forgetting the last counters there would count them again on the
// next good read.
func Collect(m *model.Model, u *Usage, d awg.Dump, ts int64) map[string]Live {
now := time.Unix(ts, 0).In(zone.Local())
live := map[string]Live{}
var all Pair
for _, id := range m.DeviceIDs() {
dev := m.Devices[id]
rec := u.Devices[id]
if rec == nil {
rec = &Record{Raw: &Pair{}}
u.Devices[id] = rec
}
var srv *model.Server
if p := m.People[dev.Person]; p != nil {
srv = m.Servers[p.Server]
}
peer, there := d.Peers[dev.PublicKey]
if !there {
ifname := ""
if srv != nil {
ifname = srv.Ifname
}
if d.Reads[ifname] != awg.Failed {
rec.Raw = &Pair{}
}
endpoint := ""
if rec.Endpoint != nil {
endpoint = *rec.Endpoint
}
live[id] = Live{Handshake: rec.Handshake, Endpoint: endpoint}
continue
}
raw := Pair{}
if rec.Raw != nil {
raw = *rec.Raw
}
if ifc, read := d.Ifaces[peer.Ifname]; read {
if rec.Ifindex != nil && *rec.Ifindex != ifc.Ifindex {
raw = Pair{}
}
index := ifc.Ifindex
rec.Ifindex = &index
}
rx, tx := float64(peer.Rx), float64(peer.Tx)
drx, dtx := rx, tx
if rx >= raw[0] {
drx = rx - raw[0]
}
if tx >= raw[1] {
dtx = tx - raw[1]
}
rec.Raw = &Pair{rx, tx}
if drx > 0 || dtx > 0 {
account(rec, drx, dtx, now)
all[0] += drx
all[1] += dtx
p := u.People[dev.Person]
if p == nil {
p = &Record{}
u.People[dev.Person] = p
}
accountMonth(p, drx, dtx, ts)
account(p, drx, dtx, now)
}
ip := presence.EndpointIP(peer.Endpoint)
if peer.Handshake > 0 {
rec.Handshake = float64(peer.Handshake)
rec.Endpoint = &ip
if rec.IPs == nil {
rec.IPs = map[string]float64{}
}
if ip != "" {
rec.IPs[ip] = float64(peer.Handshake)
}
keepNewest(rec.IPs, addressesKept)
}
l := Live{Present: true, Handshake: rec.Handshake, Endpoint: ip,
Online: peer.Handshake > 0 && ts-peer.Handshake < int64(model.OnlineWindowOf(srv, dev))}
if peer.Handshake > 0 {
l.Handshake = float64(peer.Handshake)
}
if l.Endpoint == "" && rec.Endpoint != nil {
l.Endpoint = *rec.Endpoint
}
live[id] = l
}
// Counters of devices that no longer exist are forgotten (their bytes stay in the person's totals).
for id := range u.Devices {
if m.Devices[id] == nil {
delete(u.Devices, id)
}
}
if all[0] > 0 || all[1] > 0 {
if u.Hours == nil {
u.Hours = map[string]Pair{}
}
add(u.Hours, hourKey(now), all[0], all[1])
}
if u.Hours != nil {
prune(u.Hours, hoursKept)
}
Prune(u, PolicyOf(m.Settings.Retention))
u.Updated = float64(ts)
return live
}
// keepNewest keeps the n addresses seen last.
func keepNewest(ips map[string]float64, n int) {
if len(ips) <= n {
return
}
keys := make([]string, 0, len(ips))
for k := range ips {
keys = append(keys, k)
}
sort.SliceStable(keys, func(i, j int) bool { return ips[keys[i]] > ips[keys[j]] })
for _, k := range keys[n:] {
delete(ips, k)
}
}
// Prune applies the history's retention to every record.
func Prune(u *Usage, pol Policy) {
for _, group := range []map[string]*Record{u.Devices, u.People} {
for _, rec := range group {
if rec.Months != nil {
prune(rec.Months, pol.Months)
}
if rec.Days != nil {
prune(rec.Days, pol.Days)
}
}
}
}
// wipe forgets a record's traffic history; the kernel's counters and the last handshake stay.
func wipe(rec *Record) {
if rec == nil {
return
}
rec.Total, rec.Months, rec.Days, rec.Period = nil, nil, nil, nil
}
// Reset forgets traffic history. scope: all | person | device | limit. A person's reset also clears their
// devices and restarts their data limit; a device's leaves the person's totals (and so their limit) alone;
// `limit` restarts a person's limit month and touches nothing else (an ended month is left in its place,
// not none: a record from before limit months would be read by its calendar month again).
func Reset(u *Usage, m *model.Model, scope, id string) {
switch scope {
case "all":
u.Hours = nil
for _, rec := range u.Devices {
wipe(rec)
}
for _, rec := range u.People {
wipe(rec)
}
case "person":
wipe(u.People[id])
if p := m.People[id]; p != nil {
for _, did := range p.Devices {
wipe(u.Devices[did])
}
}
case "device":
wipe(u.Devices[id])
case "limit":
if rec := u.People[id]; rec != nil {
rec.Period = &Period{}
}
}
}
+282
View File
@@ -0,0 +1,282 @@
package usage
import (
"encoding/json"
"fmt"
"os"
"reflect"
"strings"
"testing"
"time"
"github.com/tickcount/luci-app-dawnline/backend/awg"
"github.com/tickcount/luci-app-dawnline/backend/model"
"github.com/tickcount/luci-app-dawnline/backend/uci"
"github.com/tickcount/luci-app-dawnline/backend/zone"
)
// The generator of testdata/trace.uc, number for number.
type gen struct{ x int64 }
func (g *gen) rnd(n int64) int64 {
g.x = (g.x*1103515245 + 12345) % 2147483648
return (g.x >> 8) % n
}
// plain is a document as JSON reads it, without what is empty: an empty history is the same as none.
func plain(v any) any {
raw, _ := json.Marshal(v)
var out any
json.Unmarshal(raw, &out)
return tidy(out)
}
func tidy(v any) any {
if m, ok := v.(map[string]any); ok {
for k, e := range m {
e = tidy(e)
if sub, isMap := e.(map[string]any); isMap && len(sub) == 0 {
delete(m, k)
continue
}
m[k] = e
}
}
return v
}
func differ(path string, got, want any) []string {
g, gok := got.(map[string]any)
w, wok := want.(map[string]any)
if gok && wok {
var out []string
for k := range w {
if _, has := g[k]; !has {
out = append(out, fmt.Sprintf("%s.%s: missing, ucode has %v", path, k, w[k]))
}
}
for k, v := range g {
if _, has := w[k]; !has {
out = append(out, fmt.Sprintf("%s.%s: %v, ucode has none", path, k, v))
continue
}
out = append(out, differ(path+"."+k, v, w[k])...)
}
return out
}
if reflect.DeepEqual(got, want) {
return nil
}
return []string{fmt.Sprintf("%s: %v, ucode has %v", path, got, want)}
}
// The same made-up looks through Collect as trace.uc put through dawnline.live collect(), from the same
// usage file, at the same moment: the same counters at the end, and the same view of every device at
// every look.
func TestHeldToUcode(t *testing.T) {
zone.Set(time.UTC)
text, err := os.ReadFile("testdata/usage.uci")
if err != nil {
t.Fatal(err)
}
c, err := uci.Parse(string(text))
if err != nil {
t.Fatal(err)
}
m := model.Load(c, nil)
raw, err := os.ReadFile("testdata/trace.json")
if err != nil {
t.Fatal(err)
}
var golden struct {
TS int64 `json:"ts"`
Steps int `json:"steps"`
Seed json.RawMessage `json:"seed"`
Final any `json:"final"`
Hash int64 `json:"hash"`
}
if err := json.Unmarshal(raw, &golden); err != nil {
t.Fatal(err)
}
u := Empty()
if err := json.Unmarshal(golden.Seed, u); err != nil {
t.Fatalf("the seeded usage does not read: %v", err)
}
ts0 := golden.TS
g := &gen{x: 4711}
ifindex := []int{5, 9}
type peer struct {
present bool
rx, tx, hs int64
ep string
}
st := map[string]*peer{}
for _, id := range m.DeviceIDs() {
st[id] = &peer{present: true, hs: ts0 - 3000}
}
hash := int64(7)
for i := 0; i < golden.Steps; i++ {
d := awg.Dump{OK: true, Reads: map[string]string{}, Ifaces: map[string]awg.Iface{}, Peers: map[string]awg.Peer{}}
state := map[string]string{}
for n, sid := range m.ServerIDs() {
srv := m.Servers[sid]
switch r := g.rnd(100); {
case r < 4:
state[sid] = awg.Failed
case r < 7:
state[sid] = awg.Down
default:
state[sid] = awg.Read
}
if g.rnd(100) < 2 {
ifindex[n]++
}
d.Reads[srv.Ifname] = state[sid]
if state[sid] == awg.Read {
d.Ifaces[srv.Ifname] = awg.Iface{Ifindex: ifindex[n]}
}
}
for _, id := range m.DeviceIDs() {
dev, s := m.Devices[id], st[id]
sid := m.People[dev.Person].Server
if g.rnd(100) < 3 {
s.present = !s.present
if s.present {
s.rx, s.tx = 0, 0
}
}
if !s.present {
continue
}
if g.rnd(100) < 60 {
s.rx += g.rnd(5000)
}
if g.rnd(100) < 60 {
s.tx += g.rnd(90000)
}
if g.rnd(100) < 3 {
s.rx = g.rnd(100)
s.tx = g.rnd(100)
}
if g.rnd(100) < 20 {
s.hs += 1 + g.rnd(50)
}
if g.rnd(100) < 10 {
s.ep = fmt.Sprintf("203.0.113.%d:4000", 1+g.rnd(12))
}
if state[sid] == awg.Read {
d.Peers[dev.PublicKey] = awg.Peer{Ifname: m.Servers[sid].Ifname, Endpoint: s.ep, Handshake: s.hs, Rx: s.rx, Tx: s.tx}
}
}
seen := Collect(m, u, d, ts0)
for _, id := range m.DeviceIDs() {
v := seen[id]
on, present := 0, 0
if v.Online {
on = 1
}
if v.Present {
present = 1
}
for _, ch := range []byte(fmt.Sprintf("%s:%d:%d:%s:%d;", id, on, int64(v.Handshake), v.Endpoint, present)) {
hash = (hash*131 + int64(ch)) % 1000000007
}
}
}
if d := differ("", plain(u), tidy(golden.Final)); len(d) > 0 {
t.Errorf("the counters ended differently than ucode's:\n%s", strings.Join(d[:min(len(d), 25)], "\n"))
}
if hash != golden.Hash {
t.Errorf("what the looks showed of the devices differs somewhere: hash %d, ucode has %d", hash, golden.Hash)
}
}
func TestLimitMonth(t *testing.T) {
zone.Set(time.UTC)
at := func(s string) int64 {
v, err := time.Parse("2006-01-02 15:04:05", s)
if err != nil {
t.Fatal(err)
}
return v.Unix()
}
// a month later on the calendar; a day the next month lacks is its last one
for from, to := range map[string]string{
"2026-01-31 10:00:00": "2026-02-28 10:00:00",
"2028-01-31 10:00:00": "2028-02-29 10:00:00",
"2026-12-15 23:59:59": "2027-01-15 23:59:59",
"2026-03-31 00:00:00": "2026-04-30 00:00:00",
} {
if got := monthAfter(at(from)); got != at(to) {
t.Errorf("a month after %s: %s, want %s", from, time.Unix(got, 0).UTC().Format("2006-01-02 15:04:05"), to)
}
}
running := &Record{Period: &Period{Start: float64(at("2026-11-01 00:00:00")), Rx: 5}}
if p := LimitMonth(running, at("2026-11-30 23:59:59")); p == nil || p.Rx != 5 {
t.Error("a running limit month is not found")
}
if LimitMonth(running, at("2026-12-01 00:00:00")) != nil {
t.Error("an ended limit month still runs")
}
old := &Record{Months: map[string]Pair{"2026-11": {7, 9}}}
if p := LimitMonth(old, at("2026-11-11 12:00:00")); p == nil || p.Start != float64(at("2026-11-01 00:00:00")) || p.Tx != 9 {
t.Errorf("a record from before limit months: %+v", p)
}
if LimitMonth(&Record{}, at("2026-11-11 12:00:00")) != nil || LimitMonth(nil, 0) != nil {
t.Error("a limit month out of nothing")
}
}
func TestReset(t *testing.T) {
c, _ := uci.Parse("config person 'anna'\nconfig device 'd1'\n\toption person 'anna'\nconfig device 'd2'\n\toption person 'x'\n")
m := model.Load(c, nil)
fresh := func() *Usage {
return &Usage{
Devices: map[string]*Record{"d1": {Raw: &Pair{1, 1}, Total: &Pair{5, 5}, Handshake: 9}, "d2": {Total: &Pair{7, 7}}},
People: map[string]*Record{"anna": {Total: &Pair{5, 5}, Period: &Period{Start: 3, Rx: 5}}},
Hours: map[string]Pair{"2026-11-11 10": {1, 1}},
}
}
u := fresh()
Reset(u, m, "person", "anna")
if u.People["anna"].Total != nil || u.Devices["d1"].Total != nil || u.Devices["d1"].Raw == nil || u.Devices["d1"].Handshake != 9 || u.Devices["d2"].Total == nil {
t.Errorf("person: %+v %+v", u.People["anna"], u.Devices["d1"])
}
u = fresh()
Reset(u, m, "device", "d1")
if u.Devices["d1"].Total != nil || u.People["anna"].Total == nil {
t.Error("device")
}
u = fresh()
Reset(u, m, "limit", "anna")
if p := u.People["anna"].Period; p == nil || p.Start != 0 || p.Rx != 0 || u.People["anna"].Total == nil {
t.Errorf("limit: %+v", u.People["anna"])
}
u = fresh()
Reset(u, m, "all", "")
if u.Hours != nil || u.Devices["d2"].Total != nil || u.People["anna"].Period != nil {
t.Error("all")
}
}
// Days, hours and months are the router's own, not UTC's: a Go program on OpenWrt knows no zone unless it
// is told (package zone). The night this was found: 00:06 in Moscow, 21:06 the day before in UTC.
func TestDaysAreTheRoutersOwn(t *testing.T) {
msk, err := zone.FromPOSIX("MSK-3")
if err != nil {
t.Fatal(err)
}
zone.Set(msk)
defer zone.Set(time.UTC)
ts := time.Date(2026, 11, 11, 21, 6, 32, 0, time.UTC).Unix()
now := time.Unix(ts, 0).In(zone.Local())
if dayKey(now) != "2026-11-12" || hourKey(now) != "2026-11-12 00" || monthKey(now) != "2026-11" {
t.Fatalf("the day %s, the hour %s, the month %s", dayKey(now), hourKey(now), monthKey(now))
}
// a limit month that began on the 31st at 23:30 Moscow time ends on the last of the next month there
began := time.Date(2026, 1, 31, 20, 30, 0, 0, time.UTC).Unix()
if got := time.Unix(monthAfter(began), 0).In(msk).Format("2006-01-02 15:04"); got != "2026-02-28 23:30" {
t.Fatalf("a month after: %s", got)
}
}