Release v117 with persistent logs and WAN tunnel recovery
This commit is contained in:
1 parent
5f65a3d811
commit
b62db40daf
61 files changed
+3080
-438
No files matched your search
@@ -175,16 +175,53 @@ wget -O - "https://raw.githubusercontent.com/kzolotarev95/luci-app-owrt-remote/m
|
||||
и ошибки Xray. Успешные Heartbeat без изменения состояния и повторяющиеся
|
||||
события не заполняют журнал; подробности каждого соединения Xray отключены.
|
||||
В просмотре успех отмечается зелёным, предупреждение — жёлтым, ошибка — красным.
|
||||
Журнал хранится в `/tmp/owrt-remote-logs/`, занимает около 2 МБ максимум,
|
||||
очищается каждые 24 часа и исчезает после перезагрузки. Сбор работает отдельно
|
||||
от туннеля; остановка Xray не останавливает журнал. Ключи и пароли скрываются.
|
||||
Перед записью проверяется tmpfs: журнал и его служебное состояние не размещаются
|
||||
на Flash. После очистки одинаковые факты могут быть записаны заново.
|
||||
Постоянный журнал и курсоры доставки хранятся на проверенном доступном для записи
|
||||
Flash-разделе: обычно `/overlay/owrt-remote/logs/`. `/overlay` используется только
|
||||
если это настоящая точка монтирования JFFS2/UBIFS/ext4/f2fs; tmpfs и корневой
|
||||
OverlayFS не считаются постоянным хранилищем. На системах с непосредственно
|
||||
записываемым постоянным корнем используется `/etc/owrt-remote/logs/`.
|
||||
Предел — 2 МиБ данных и 256 неизменяемых фрагментов до 16 КиБ каждый.
|
||||
Доставленные фрагменты удаляются через 24 часа или раньше при нехватке места.
|
||||
Неподтверждённые записи не удаляются автоматически: при заполненной очереди
|
||||
новые события сверх лимита пропускаются, после восстановления пишется их число.
|
||||
|
||||
В `/tmp/owrt-remote-log-runtime/` остаются только буфер записи до 16 КиБ,
|
||||
небольшой кеш повторов, блокировки, FIFO и ограниченные пакеты HTTP.
|
||||
Буфер сбрасывается раз в 60 секунд; обычная остановка сборщика сбрасывает его
|
||||
перед выходом. После внезапного отключения питания возможна потеря последней
|
||||
минуты в RAM; завершённые фрагменты имеют контрольную сумму и сохраняются
|
||||
после перезагрузки. Состояние меняется атомарно на том же разделе, а sync
|
||||
выполняется только при фиксации пакета, не после каждой строки.
|
||||
Если Flash недоступна, используется ограниченный журнал в RAM до 64 КиБ
|
||||
плюс буфер 16 КиБ с предупреждением. При ошибке записи Flash сохранённая
|
||||
очередь остаётся на месте; возвращение к ней — после восстановления хранилища
|
||||
и перезагрузки роутера. Менее 2 МиБ свободного Flash останавливает новые записи
|
||||
на неё. Системный logd, WAN, DNS, Podkop, Xray и конфигурация сети не меняются.
|
||||
Сбор работает отдельно от туннеля. Ключи и пароли скрываются.
|
||||
|
||||
Контрольная сумма фрагментов вычисляется доступным `sha256sum`, `md5sum`
|
||||
или `cksum`; пустая/ошибочная сумма не позволяет зафиксировать новый фрагмент.
|
||||
Фрагменты без суммы, оставленные ранним локальным исправлением, восстанавливаются
|
||||
из сохранённых файлов в отдельный журнал и повторно отправляются на VPS.
|
||||
Пустой пакет состояния больше не подтверждает неполученные байты журнала.
|
||||
Старые предупреждения о ротации остаются историей; уже потерянные файлы
|
||||
восстановить нельзя. Новая контрольная сумма проверяет дальнейшую целостность,
|
||||
но не доказывает целостность фрагментов, у которых раньше не было суммы.
|
||||
|
||||
Подробности устройства хранения и проверки: [tests/PERSISTENT_LOGS.md](tests/PERSISTENT_LOGS.md).
|
||||
|
||||
В карточке роутера на VPS есть **Глобал логи**, открывающиеся в новой вкладке.
|
||||
Архив хранится в постоянной SQLite-базе `/var/lib/owrt-remote/hub.db`
|
||||
(или по явно настроенному пути БД), до 2 МиБ на роутер. Тип файловой системы
|
||||
и путь показаны в окне логов; для tmpfs/ramfs выводится предупреждение.
|
||||
Локальный установщик VPS проверяет постоянный записываемый диск перед заменой
|
||||
Hub и не меняет путь БД или конфигурацию сервера.
|
||||
|
||||
Там доступны те же скачивание, просмотр с обновлением каждые 3 секунды и очистка,
|
||||
а также зелёные, жёлтые и красные записи. Роутер доставляет новые записи на VPS
|
||||
отдельно от Heartbeat: повторная отправка после обрыва не создаёт дубликатов.
|
||||
Новые события обычно поступают после пакетной записи на Flash — до 60 секунд;
|
||||
проверка очереди и обновление просмотра выполняются каждые 3 секунды.
|
||||
Остановка Remote кнопкой «Остановить» не прекращает доставку журнала: пока
|
||||
работает интернет, записи об остановке и дальнейшие действия поступают на VPS.
|
||||
Над журналом Hub показывает полученное состояние: Remote выключен, Xray
|
||||
@@ -195,9 +232,9 @@ Hub сообщает об этом прямо; последнее предупр
|
||||
|
||||
Кнопка доступна и при оффлайн: Hub хранит последние доставленные записи в своей
|
||||
базе, до 2 МБ на роутер. Архив сохраняется после перезагрузки роутера и Hub;
|
||||
очистка ОЗУ роутера раз в сутки не удаляет копию на VPS. Hub записывает факт
|
||||
ротация локального журнала не удаляет копию на VPS. Hub записывает факт
|
||||
истечения ожидания Heartbeat. Записи, созданные после полной потери связи,
|
||||
появятся только при её восстановлении, если ещё сохранены в ОЗУ роутера.
|
||||
появятся только при её восстановлении, если сохранены в ограниченной очереди роутера.
|
||||
Очистка из Hub сразу удаляет архив VPS, а очистка журнала оффлайн-роутера
|
||||
применится при следующей доставке. Доступ к журналу требует полного доступа
|
||||
к этому роутеру; удаление роутера удаляет его архив.
|
||||
|
||||
@@ -5,6 +5,7 @@ START=98
|
||||
STOP=11
|
||||
|
||||
start_service() {
|
||||
/bin/sh /usr/lib/owrt-remote/logs.sh migrate || logger -t owrt-remote 'WARN: старый RAM-журнал ещё не перенесён; см. свободное место Flash.'
|
||||
procd_open_instance journal
|
||||
procd_set_param command /bin/sh /usr/lib/owrt-remote/logs.sh collect
|
||||
procd_set_param respawn 3600 5 5
|
||||
@@ -18,3 +19,9 @@ start_service() {
|
||||
procd_set_param respawn 3600 5 5
|
||||
procd_close_instance
|
||||
}
|
||||
|
||||
stop_service() {
|
||||
# Flush a last bounded batch on ordinary stop/shutdown. Sudden power loss can
|
||||
# lose only the current RAM write buffer; committed immutable chunks survive.
|
||||
/bin/sh /usr/lib/owrt-remote/logs.sh flush force >/dev/null 2>&1 || true
|
||||
}
|
||||
+623
-176
@@ -1,89 +1,425 @@
|
||||
#!/bin/sh
|
||||
# The private Remote journal lives in RAM. It never changes the system logd.
|
||||
LOG_DIR="${OWRT_REMOTE_LOG_DIR:-/tmp/owrt-remote-logs}"
|
||||
LOG_LIMIT=1048576
|
||||
# Persistent, bounded spool; only the small write buffer and process state use RAM.
|
||||
RUN_DIR="${OWRT_REMOTE_LOG_RUNTIME:-/tmp/owrt-remote-log-runtime}"
|
||||
LOG_LIMIT=2097152
|
||||
LOG_KEEP_SECONDS=86400
|
||||
LOG_PART_LIMIT=256
|
||||
LOG_BUFFER_LIMIT=16384
|
||||
LOG_BATCH_LIMIT=16384
|
||||
LOG_FLUSH_SECONDS=60
|
||||
LOG_FREE_KB=2048
|
||||
|
||||
log_clock() { cut -d. -f1 /proc/uptime; }
|
||||
log_ram_storage() {
|
||||
local resolved parent
|
||||
if [ -e "$LOG_DIR" ] || [ -L "$LOG_DIR" ]; then
|
||||
resolved="$(readlink -f "$LOG_DIR")" || return 1
|
||||
else
|
||||
# BusyBox realpath may require the last component to exist.
|
||||
# Resolve the existing parent before creating the journal directory.
|
||||
parent="$(readlink -f "$(dirname "$LOG_DIR")")" || return 1
|
||||
resolved="$parent/$(basename "$LOG_DIR")"
|
||||
fi
|
||||
awk -v path="$resolved" '
|
||||
log_epoch() { date +%s; }
|
||||
log_self_pid() {
|
||||
# $$ is the parent shell's PID inside ash subshells/pipelines. Reading proc
|
||||
# with the shell builtin yields the actual lock holder, including the writer.
|
||||
local self rest
|
||||
if read -r self rest </proc/self/stat 2>/dev/null; then printf '%s\n' "$self"; else printf '%s\n' "$$"; fi
|
||||
}
|
||||
log_mount() {
|
||||
# Resolve symlinks, including the existing ancestor of a new directory.
|
||||
local path suffix
|
||||
path="$1"; suffix=''
|
||||
case "$path" in /*) ;; *) return 1 ;; esac
|
||||
while [ ! -e "$path" ]; do
|
||||
[ "$path" != / ] || return 1
|
||||
suffix="/$(basename "$path")$suffix"; path="$(dirname "$path")"
|
||||
done
|
||||
path="$(readlink -f "$path")$suffix" || return 1
|
||||
awk -v path="$path" '
|
||||
(path == $2 || index(path, $2 "/") == 1 || $2 == "/") && length($2) > best {
|
||||
best = length($2); filesystem = $3
|
||||
best=length($2); mount=$2; fs=$3; flags=$4
|
||||
}
|
||||
END { exit(filesystem == "tmpfs" ? 0 : 1) }
|
||||
END { if (best) print mount, fs, flags; else exit 1 }
|
||||
' /proc/mounts
|
||||
}
|
||||
log_ram_storage() {
|
||||
local mount fs flags
|
||||
read -r mount fs flags <<EOF
|
||||
$(log_mount "$RUN_DIR")
|
||||
EOF
|
||||
[ "$fs" = tmpfs ]
|
||||
}
|
||||
log_flash_storage() {
|
||||
local mount fs flags
|
||||
read -r mount fs flags <<EOF
|
||||
$(log_mount "$1")
|
||||
EOF
|
||||
case "$fs" in jffs2|ubifs|ext4|f2fs|btrfs|ext2|ext3) ;; *) return 1 ;; esac
|
||||
case ",$flags," in *,rw,*) ;; *) return 1 ;; esac
|
||||
# A tmpfs/root overlay and an unmounted /overlay are never persistent storage.
|
||||
[ "$mount" != /overlay ] || [ -d /overlay ]
|
||||
}
|
||||
log_init() {
|
||||
local candidate selected mount fs flags free
|
||||
umask 077
|
||||
log_ram_storage || { printf 'Remote journal requires RAM (tmpfs); refusing persistent storage.\n' >&2; return 1; }
|
||||
mkdir -p "$LOG_DIR" || return 1
|
||||
chmod 700 "$LOG_DIR"
|
||||
log_ram_storage || { printf 'Remote: runtime directory must be tmpfs.\n' >&2; return 1; }
|
||||
mkdir -p "$RUN_DIR" || return 1
|
||||
chmod 700 "$RUN_DIR" || return 1
|
||||
# Pin one storage choice for this boot; never interleave two journal IDs.
|
||||
selected="$(cat "$RUN_DIR/storage" 2>/dev/null)"
|
||||
if [ -n "$selected" ]; then
|
||||
LOG_DIR="$selected"
|
||||
else
|
||||
candidate="${OWRT_REMOTE_LOG_DIR:-}"
|
||||
if [ -z "$candidate" ]; then
|
||||
# Select /overlay only when it really is a writable persistent mount.
|
||||
read -r mount fs flags <<EOF
|
||||
$(log_mount /overlay)
|
||||
EOF
|
||||
if [ "$mount" = /overlay ] && log_flash_storage /overlay; then
|
||||
candidate=/overlay/owrt-remote/logs
|
||||
elif log_flash_storage /; then
|
||||
# OpenWrt installations with a directly writable ext4/UBIFS root.
|
||||
candidate=/etc/owrt-remote/logs
|
||||
fi
|
||||
fi
|
||||
if [ -n "$candidate" ] && log_flash_storage "$candidate" &&
|
||||
mkdir -p "$candidate" && [ -w "$candidate" ]; then
|
||||
LOG_DIR="$candidate"
|
||||
chmod 700 "$LOG_DIR" || return 1
|
||||
free="$(df -Pk "$LOG_DIR" | awk 'NR==2 {print $4}')"
|
||||
case "$free" in ''|*[!0-9]*) free=0 ;; esac
|
||||
# Reading an existing queue is still useful on a full disk; starting
|
||||
# a new one must not consume the last bytes needed for router settings.
|
||||
if [ ! -s "$LOG_DIR/state" ] && [ "$free" -lt "$LOG_FREE_KB" ]; then LOG_DIR="$RUN_DIR/fallback"; fi
|
||||
else
|
||||
LOG_DIR="$RUN_DIR/fallback"
|
||||
fi
|
||||
fi
|
||||
PERSISTENT=1
|
||||
if [ "$LOG_DIR" = "$RUN_DIR/fallback" ]; then
|
||||
PERSISTENT=0; LOG_LIMIT=65536; LOG_PART_LIMIT=8
|
||||
mkdir -p "$LOG_DIR" || return 1
|
||||
fi
|
||||
}
|
||||
log_lock() {
|
||||
local attempt owner
|
||||
log_init || return 1
|
||||
attempt=0
|
||||
while ! mkdir "$LOG_DIR/lock" 2>/dev/null; do
|
||||
owner="$(cat "$LOG_DIR/lock/pid" 2>/dev/null)"
|
||||
while ! mkdir "$RUN_DIR/lock" 2>/dev/null; do
|
||||
owner="$(cat "$RUN_DIR/lock/pid" 2>/dev/null)"
|
||||
case "$owner" in
|
||||
''|*[!0-9]*)
|
||||
# Recover a writer killed between mkdir and writing its PID.
|
||||
if [ "$attempt" -ge 20 ]; then
|
||||
rm -f "$LOG_DIR/lock/pid"
|
||||
rmdir "$LOG_DIR/lock" 2>/dev/null || true
|
||||
fi ;;
|
||||
*) if ! kill -0 "$owner" 2>/dev/null; then
|
||||
rm -f "$LOG_DIR/lock/pid"
|
||||
rmdir "$LOG_DIR/lock" 2>/dev/null || true
|
||||
fi ;;
|
||||
[ "$attempt" -lt 20 ] || log_reap_lock "$RUN_DIR/lock" "$attempt" ;;
|
||||
*) kill -0 "$owner" 2>/dev/null || log_reap_lock "$RUN_DIR/lock" "$attempt" ;;
|
||||
esac
|
||||
attempt=$((attempt + 1))
|
||||
[ "$attempt" -lt 100 ] || return 1
|
||||
attempt=$((attempt + 1)); [ "$attempt" -lt 100 ] || return 1
|
||||
sleep 0.1
|
||||
done
|
||||
printf '%s\n' "$$" >"$LOG_DIR/lock/pid"
|
||||
}
|
||||
log_unlock() {
|
||||
rm -f "$LOG_DIR/lock/pid"
|
||||
rmdir "$LOG_DIR/lock" 2>/dev/null || true
|
||||
}
|
||||
log_reset_locked() {
|
||||
: >"$LOG_DIR/current.log"
|
||||
rm -f "$LOG_DIR/previous.log"
|
||||
rm -f "$LOG_DIR/facts" "$LOG_DIR/facts.new" "$LOG_DIR/heartbeat-ok"
|
||||
rm -f "$LOG_DIR/journal-id" "$LOG_DIR/total" "$LOG_DIR/sync-ack"
|
||||
log_clock >"$LOG_DIR/age"
|
||||
}
|
||||
log_cursor_locked() {
|
||||
local total
|
||||
if [ ! -s "$LOG_DIR/journal-id" ]; then
|
||||
cat /proc/sys/kernel/random/uuid >"$LOG_DIR/journal-id" 2>/dev/null ||
|
||||
printf '%s-%s-%s\n' "$(date +%s)" "$(log_clock)" "$$" >"$LOG_DIR/journal-id"
|
||||
log_self_pid >"$RUN_DIR/lock/pid"
|
||||
# Another first writer may have selected storage while we waited.
|
||||
if [ -s "$RUN_DIR/storage" ]; then
|
||||
LOG_DIR="$(cat "$RUN_DIR/storage")"
|
||||
[ "$LOG_DIR" != "$RUN_DIR/fallback" ] || { PERSISTENT=0; LOG_LIMIT=65536; LOG_PART_LIMIT=8; }
|
||||
else
|
||||
printf '%s\n' "$LOG_DIR" >"$RUN_DIR/storage"
|
||||
fi
|
||||
if [ ! -s "$LOG_DIR/total" ]; then
|
||||
total="$(wc -c <"$LOG_DIR/current.log")"
|
||||
[ ! -f "$LOG_DIR/previous.log" ] || total=$((total + $(wc -c <"$LOG_DIR/previous.log")))
|
||||
printf '%s\n' "$total" >"$LOG_DIR/total"
|
||||
# Change storage only under the same append/export lock. On write/mount
|
||||
# failure preserve the Flash queue and switch this boot to a bounded fallback.
|
||||
if [ "$LOG_DIR" != "$RUN_DIR/fallback" ] && { ! log_flash_storage "$LOG_DIR" || [ ! -w "$LOG_DIR" ] || [ -f "$RUN_DIR/write-warning" ]; }; then
|
||||
log_fallback_locked || { log_unlock; return 1; }
|
||||
fi
|
||||
}
|
||||
log_maintain_locked() {
|
||||
local now age
|
||||
now="$(log_clock)"
|
||||
age="$(cat "$LOG_DIR/age" 2>/dev/null)"
|
||||
case "$age" in ''|*[!0-9]*) log_reset_locked; return ;; esac
|
||||
if [ "$now" -lt "$age" ] || [ "$((now - age))" -ge "$LOG_KEEP_SECONDS" ]; then
|
||||
log_reset_locked
|
||||
log_reap_lock() {
|
||||
local owner reap_owner
|
||||
# Serialize stale-lock removal itself, so two contenders cannot remove a
|
||||
# freshly acquired lock after observing the same previous dead owner.
|
||||
if ! mkdir "$1/reap" 2>/dev/null; then
|
||||
reap_owner="$(cat "$1/reap/pid" 2>/dev/null)"
|
||||
case "$reap_owner" in
|
||||
''|*[!0-9]*) [ "${2:-0}" -ge 20 ] || return 0 ;;
|
||||
*) kill -0 "$reap_owner" 2>/dev/null && return 0 ;;
|
||||
esac
|
||||
rm -f "$1/reap/pid"; rmdir "$1/reap" 2>/dev/null || return 0
|
||||
return 0
|
||||
fi
|
||||
log_self_pid >"$1/reap/pid"
|
||||
owner="$(cat "$1/pid" 2>/dev/null)"
|
||||
case "$owner" in
|
||||
''|*[!0-9]*) rm -f "$1/pid" ;;
|
||||
*) if kill -0 "$owner" 2>/dev/null; then rm -f "$1/reap/pid"; rmdir "$1/reap"; return 0; else rm -f "$1/pid"; fi ;;
|
||||
esac
|
||||
rm -f "$1/reap/pid"; rmdir "$1/reap"; rmdir "$1" 2>/dev/null || true
|
||||
}
|
||||
log_fallback_locked() {
|
||||
LOG_DIR="$RUN_DIR/fallback"; PERSISTENT=0; LOG_LIMIT=65536; LOG_PART_LIMIT=8
|
||||
mkdir -p "$LOG_DIR" || return 1
|
||||
printf '%s\n' "$LOG_DIR" >"$RUN_DIR/storage"
|
||||
rm -f "$RUN_DIR/recovered" "$RUN_DIR/recovered-v2" "$RUN_DIR/ack"
|
||||
}
|
||||
log_unlock() { rm -f "$RUN_DIR/lock/pid"; rmdir "$RUN_DIR/lock" 2>/dev/null || true; }
|
||||
log_warn_locked() {
|
||||
# One warning until successful recovery; never feed our own collector a loop.
|
||||
if [ ! -f "$RUN_DIR/$1" ]; then
|
||||
local warning size
|
||||
warning="$(printf '[WARN] %s owrt-remote: %s' "$(date '+%Y-%m-%d %H:%M:%S')" "$2")"
|
||||
size=0; [ ! -f "$RUN_DIR/pending" ] || size="$(wc -c <"$RUN_DIR/pending")"
|
||||
if [ "$((size + $(printf '%s\n' "$warning" | wc -c)))" -le "$LOG_BUFFER_LIMIT" ]; then printf '%s\n' "$warning" >>"$RUN_DIR/pending"; fi
|
||||
: >"$RUN_DIR/$1"
|
||||
printf 'Remote: %s\n' "$2" >&2
|
||||
fi
|
||||
}
|
||||
log_new_id() {
|
||||
cat /proc/sys/kernel/random/uuid 2>/dev/null || printf '%s-%s-%s\n' "$(log_epoch)" "$(log_clock)" "$$"
|
||||
}
|
||||
log_checksum() {
|
||||
# OpenWrt does not necessarily ship the optional BusyBox cksum applet.
|
||||
# Validate command output before publishing a chunk or advancing its cursor.
|
||||
local method value
|
||||
case "${2:-}" in
|
||||
s256-*) method=s256 ;;
|
||||
md5-*) method=md5 ;;
|
||||
*[!0-9]*|'')
|
||||
if command -v sha256sum >/dev/null 2>&1; then method=s256
|
||||
elif command -v md5sum >/dev/null 2>&1; then method=md5
|
||||
else method=crc; fi ;;
|
||||
*) method=crc ;;
|
||||
esac
|
||||
case "$method" in
|
||||
s256) value="$(sha256sum "$1" 2>/dev/null)" || return 1 ;;
|
||||
md5) value="$(md5sum "$1" 2>/dev/null)" || return 1 ;;
|
||||
crc) value="$(cksum <"$1" 2>/dev/null)" || return 1 ;;
|
||||
esac
|
||||
# Both BusyBox and GNU utilities separate the digest from the file by space.
|
||||
value="${value%% *}"
|
||||
case "$method" in
|
||||
s256|md5)
|
||||
case "$value" in ''|*[!0-9a-f]*) return 1 ;; esac
|
||||
if [ "$method" = s256 ]; then [ "${#value}" = 64 ] || return 1; else [ "${#value}" = 32 ] || return 1; fi
|
||||
printf '%s-%s\n' "$method" "$value" ;;
|
||||
crc) case "$value" in ''|*[!0-9]*) return 1 ;; esac; printf '%s\n' "$value" ;;
|
||||
esac
|
||||
}
|
||||
log_checkpoint_locked() {
|
||||
STATE_CHANGED=0
|
||||
# Same filesystem + rename: no partially rewritten cursor is ever observed.
|
||||
# Unchanged state causes no Flash write, including idle sync polls.
|
||||
if [ "$(cat "$LOG_DIR/state" 2>/dev/null)" = "$JOURNAL $ACK $TOTAL ${CLEAR:--}" ]; then return 0; fi
|
||||
printf '%s %s %s %s\n' "$JOURNAL" "$ACK" "$TOTAL" "${CLEAR:--}" >"$LOG_DIR/state.new" &&
|
||||
mv -f "$LOG_DIR/state.new" "$LOG_DIR/state" || return 1
|
||||
STATE_CHANGED=1
|
||||
}
|
||||
log_commit_locked() {
|
||||
# Durability is a batch operation, never per log line. BusyBox builds without
|
||||
# sync -f use ordinary sync; no extra package or fsync-per-line is required.
|
||||
[ "$PERSISTENT" = 0 ] || [ "$STATE_CHANGED" = 0 ] || { sync -f "$LOG_DIR" 2>/dev/null || sync; }
|
||||
}
|
||||
log_load_locked() {
|
||||
local extra file start end crc actual
|
||||
log_finish_migration_locked || return 1
|
||||
JOURNAL=''; ACK=0; TOTAL=0; CLEAR='-'
|
||||
if [ -s "$LOG_DIR/state" ]; then
|
||||
read -r JOURNAL ACK TOTAL CLEAR extra <"$LOG_DIR/state"
|
||||
case "$JOURNAL" in ''|*[!a-zA-Z0-9._:-]*) JOURNAL='' ;; esac
|
||||
case "$ACK:$TOTAL" in *[!0-9:]*|:*|*:) JOURNAL='' ;; esac
|
||||
case "$CLEAR" in ''|*[!a-zA-Z0-9._:-]*) JOURNAL='' ;; esac
|
||||
[ "${#JOURNAL}" -le 80 ] && [ "${#CLEAR}" -le 80 ] && [ "${#ACK}" -le 16 ] && [ "${#TOTAL}" -le 16 ] || JOURNAL=''
|
||||
[ -z "$extra" ] || JOURNAL=''
|
||||
fi
|
||||
if [ -z "$JOURNAL" ]; then
|
||||
# If a state sector was lost, recover the newest committed generation.
|
||||
# Reset only the ack: Hub offsets make re-delivery harmless.
|
||||
file="$(ls -t "$LOG_DIR"/part.* 2>/dev/null | head -n 1)"
|
||||
if [ -n "$file" ]; then
|
||||
JOURNAL="${file##*/part.}"; JOURNAL="${JOURNAL%.*}"; JOURNAL="${JOURNAL%.*}"; JOURNAL="${JOURNAL%.*}"
|
||||
case "$JOURNAL" in ''|*[!a-zA-Z0-9._:-]*) JOURNAL='' ;; esac
|
||||
fi
|
||||
[ -n "$JOURNAL" ] || JOURNAL="$(log_new_id)"
|
||||
ACK=0; TOTAL=0; CLEAR='-'
|
||||
log_checkpoint_locked && log_commit_locked || return 1
|
||||
fi
|
||||
# Validate immutable chunks once per collector boot, never load their contents.
|
||||
if [ ! -f "$RUN_DIR/recovered-v2" ]; then
|
||||
log_repair_legacy_locked || return 1
|
||||
# A completed explicit clear may have left old chunks after a power cut.
|
||||
for file in "$LOG_DIR"/part.*; do
|
||||
[ -f "$file" ] || continue
|
||||
case "$file" in "$LOG_DIR/part.$JOURNAL."*) ;; *) rm -f "$file" ;; esac
|
||||
done
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
log_part_fields "$file" || continue
|
||||
actual="$(log_checksum "$file" "$crc")" || return 1
|
||||
if [ "$actual" != "$crc" ] || [ "$(wc -c <"$file")" -ne "$((end - start))" ]; then
|
||||
# Keep one damaged chunk on Flash for inspection; never export it.
|
||||
mv -f "$file" "$LOG_DIR/damaged.log" || return 1
|
||||
log_warn_locked damaged-warning 'Повреждённый фрагмент журнала изолирован; возможна потеря последнего пакета после сбоя питания.'
|
||||
fi
|
||||
done
|
||||
rm -f "$LOG_DIR/chunk.new" "$LOG_DIR/state.new"
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
log_part_fields "$file" || continue
|
||||
[ "$end" -le "$TOTAL" ] || TOTAL="$end"
|
||||
done
|
||||
log_checkpoint_locked && log_commit_locked || return 1
|
||||
: >"$RUN_DIR/recovered-v2"
|
||||
fi
|
||||
# A process can die after durable rename/checkpoint but before clearing its
|
||||
# RAM buffer. Reconcile that one in-flight batch before accepting another line.
|
||||
if [ -s "$RUN_DIR/inflight" ]; then
|
||||
read -r extra start end crc <"$RUN_DIR/inflight"
|
||||
case "$start:$end" in *[!0-9:]*|:*|*:) return 1 ;; esac
|
||||
case "$crc" in ''|*[!0-9a-fsmd-]*) return 1 ;; esac
|
||||
file="$LOG_DIR/$(printf 'part.%s.%018d.%018d.%s' "$extra" "$start" "$end" "$crc")"
|
||||
if [ "$extra" = "$JOURNAL" ] && [ -f "$file" ]; then
|
||||
actual="$(log_checksum "$file" "$crc")" || return 1
|
||||
if [ "$actual" = "$crc" ] && [ "$(wc -c <"$file")" -eq "$((end - start))" ]; then
|
||||
[ "$TOTAL" -ge "$end" ] || TOTAL="$end"
|
||||
log_checkpoint_locked && log_commit_locked || return 1
|
||||
: >"$RUN_DIR/pending"
|
||||
fi
|
||||
fi
|
||||
rm -f "$RUN_DIR/inflight"
|
||||
fi
|
||||
[ "$ACK" -le "$TOTAL" ] || ACK=0
|
||||
if [ -s "$RUN_DIR/ack" ]; then
|
||||
read -r extra actual <"$RUN_DIR/ack"
|
||||
if [ "$extra" = "$JOURNAL" ]; then
|
||||
case "$actual" in ''|*[!0-9]*) ;; *) if [ "$actual" -le "$TOTAL" ] && [ "$actual" -gt "$ACK" ]; then ACK="$actual"; fi ;; esac
|
||||
fi
|
||||
fi
|
||||
if [ ! -f "$RUN_DIR/last-flush" ]; then log_clock >"$RUN_DIR/last-flush"; fi
|
||||
[ -f "$RUN_DIR/pending" ] || : >"$RUN_DIR/pending"
|
||||
if [ "$PERSISTENT" = 0 ]; then log_warn_locked fallback-warning 'Flash недоступна: используется ограниченный журнал в ОЗУ (64 КиБ + буфер 16 КиБ); после перезагрузки он исчезнет.'; fi
|
||||
}
|
||||
log_part_fields() {
|
||||
local fields
|
||||
fields="${1##*/part.$JOURNAL.}"
|
||||
start="${fields%%.*}"; fields="${fields#*.}"; end="${fields%%.*}"; crc="${fields#*.}"
|
||||
# Windows fixtures strip trailing dots; also recover a legacy missing suffix.
|
||||
if [ "$fields" = "$end" ] && [ "${2:-}" = repair ]; then crc=''; fi
|
||||
case "$start:$end" in *[!0-9:]*|:*|*:) return 1 ;; esac
|
||||
[ "${#start}" -le 18 ] && [ "${#end}" -le 18 ] || return 1
|
||||
case "$crc" in
|
||||
'') [ "${2:-}" = repair ] || return 1 ;;
|
||||
s256-*) fields="${crc#s256-}"; case "$fields" in *[!0-9a-f]*) return 1 ;; esac; [ "${#fields}" = 64 ] || return 1 ;;
|
||||
md5-*) fields="${crc#md5-}"; case "$fields" in *[!0-9a-f]*) return 1 ;; esac; [ "${#fields}" = 32 ] || return 1 ;;
|
||||
*[!0-9]*) return 1 ;;
|
||||
*) [ "${#crc}" -le 10 ] || return 1 ;;
|
||||
esac
|
||||
# Avoid octal arithmetic for zero-padded names.
|
||||
while [ "${start#0}" != "$start" ]; do start="${start#0}"; done; start="${start:-0}"
|
||||
while [ "${end#0}" != "$end" ]; do end="${end#0}"; done; end="${end:-0}"
|
||||
[ "$end" -gt "$start" ] && [ "$((end - start))" -le "$LOG_BUFFER_LIMIT" ]
|
||||
}
|
||||
log_repair_legacy_locked() {
|
||||
local file start end crc found base maximum bytes free new_id dest actual
|
||||
found=0; base="$TOTAL"; maximum="$TOTAL"; bytes=0
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
log_part_fields "$file" repair || continue
|
||||
[ -n "$crc" ] || found=1
|
||||
[ "$start" -ge "$base" ] || base="$start"
|
||||
[ "$end" -le "$maximum" ] || maximum="$end"
|
||||
bytes=$((bytes + end - start))
|
||||
done
|
||||
[ "$found" = 1 ] || return 0
|
||||
# The previous bug confirmed empty exports without delivering these chunks.
|
||||
# Publish a new generation so that the Hub accepts their replay. Only missing
|
||||
# digest files of the expected length can be recovered; no invented checksum.
|
||||
if [ "$PERSISTENT" = 1 ]; then
|
||||
free="$(df -Pk "$LOG_DIR" | awk 'NR==2 {print $4}')"
|
||||
case "$free" in ''|*[!0-9]*) return 1 ;; esac
|
||||
[ "$free" -ge "$((LOG_FREE_KB + (bytes + 1023) / 1024))" ] || return 1
|
||||
fi
|
||||
new_id="recovered-$(log_new_id)"
|
||||
mkdir -p "$LOG_DIR/migrating" || return 1
|
||||
# An interrupted, uncommitted attempt is safe to rebuild from the originals.
|
||||
rm -f "$LOG_DIR"/migrating/part.* "$LOG_DIR/migrating/chunk.new"
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
log_part_fields "$file" repair || return 1
|
||||
[ "$(wc -c <"$file")" -eq "$((end - start))" ] || return 1
|
||||
if [ -n "$crc" ]; then actual="$(log_checksum "$file" "$crc")" && [ "$actual" = "$crc" ] || return 1; fi
|
||||
cp "$file" "$LOG_DIR/migrating/chunk.new" || return 1
|
||||
crc="$(log_checksum "$LOG_DIR/migrating/chunk.new")" || return 1
|
||||
dest="$(printf 'part.%s.%018d.%018d.%s' "$new_id" "$((start - base))" "$((end - base))" "$crc")"
|
||||
mv -f "$LOG_DIR/migrating/chunk.new" "$LOG_DIR/migrating/$dest" || return 1
|
||||
done
|
||||
printf '%s 0 %s %s\n' "$new_id" "$((maximum - base))" "$CLEAR" >"$LOG_DIR/migrating/state.new" &&
|
||||
mv -f "$LOG_DIR/migrating/state.new" "$LOG_DIR/migrating/state" || return 1
|
||||
sync -f "$LOG_DIR" 2>/dev/null || sync
|
||||
log_finish_migration_locked || return 1
|
||||
JOURNAL="$new_id"; ACK=0; TOTAL=$((maximum - base))
|
||||
rm -f "$RUN_DIR/ack" "$RUN_DIR/inflight"
|
||||
log_warn_locked repaired-warning 'Восстановлены фрагменты без контрольной суммы. Сохранённая история повторно отправляется на VPS; ранее целостность этих фрагментов не проверялась.'
|
||||
}
|
||||
log_prune_locked() {
|
||||
local file start end crc age now bytes count
|
||||
now="$(log_epoch)"; bytes=0; count=0
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
bytes=$((bytes + $(wc -c <"$file"))); count=$((count + 1))
|
||||
done
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
log_part_fields "$file" || continue
|
||||
[ "$end" -le "$ACK" ] || continue
|
||||
age="$(stat -c %Y "$file" 2>/dev/null)"; age="${age:-$now}"
|
||||
# Only confirmed chunks may expire or make room. Backwards wall-clock is safe.
|
||||
if [ "$((now - age))" -ge "$LOG_KEEP_SECONDS" ] || [ "$bytes" -gt "$((LOG_LIMIT - LOG_BUFFER_LIMIT))" ] || [ "$count" -ge "$LOG_PART_LIMIT" ]; then
|
||||
rm -f "$file" || return 1
|
||||
bytes=$((bytes - (end - start))); count=$((count - 1))
|
||||
fi
|
||||
done
|
||||
STORED_BYTES="$bytes"; STORED_PARTS="$count"
|
||||
}
|
||||
log_flush_locked() {
|
||||
local force now last bytes free crc dest dropped
|
||||
force="${1:-}"; now="$(log_clock)"; last="$(cat "$RUN_DIR/last-flush")"
|
||||
[ "$force" = force ] || [ "$now" -lt "$last" ] || [ "$((now - last))" -ge "$LOG_FLUSH_SECONDS" ] || return 0
|
||||
# A successful Hub ack is checkpointed before confirmed chunks can be removed.
|
||||
log_checkpoint_locked || { log_warn_locked write-warning 'Flash: ошибка записи состояния; временный буфер ограничен, рабочие настройки не изменены.'; return 1; }
|
||||
log_commit_locked || return 1
|
||||
log_prune_locked || return 1
|
||||
bytes="$(wc -c <"$RUN_DIR/pending")"
|
||||
if [ "$bytes" -gt 0 ]; then
|
||||
if [ "$STORED_BYTES" -gt "$((LOG_LIMIT - bytes))" ] || [ "$STORED_PARTS" -ge "$LOG_PART_LIMIT" ]; then
|
||||
log_warn_locked queue-warning 'Очередь логов заполнена: неподтверждённые записи сохранены; новые события сверх лимита пропускаются до освобождения места.'
|
||||
log_clock >"$RUN_DIR/last-flush"
|
||||
return 0
|
||||
fi
|
||||
if [ "$PERSISTENT" = 1 ]; then
|
||||
free="$(df -Pk "$LOG_DIR" | awk 'NR==2 {print $4}')"
|
||||
case "$free" in ''|*[!0-9]*) free=0 ;; esac
|
||||
if [ "$free" -lt "$LOG_FREE_KB" ]; then log_warn_locked write-warning 'На Flash менее 2 МиБ свободного места; запись логов приостановлена, буфер в ОЗУ ограничен.'; return 1; fi
|
||||
fi
|
||||
# An immutable chunk is either fully committed by rename, or ignored on recovery.
|
||||
# A fixed-size RAM buffer is the only source; never copy the entire journal.
|
||||
cp "$RUN_DIR/pending" "$LOG_DIR/chunk.new" || { log_warn_locked write-warning 'Flash: ошибка записи журнала; данные остаются в ограниченном буфере ОЗУ.'; return 1; }
|
||||
crc="$(log_checksum "$LOG_DIR/chunk.new")" || { log_warn_locked checksum-warning 'Не удалось вычислить контрольную сумму; журнал не подтверждён, записи остаются в буфере.'; return 1; }
|
||||
printf '%s %s %s %s\n' "$JOURNAL" "$TOTAL" "$((TOTAL + bytes))" "$crc" >"$RUN_DIR/inflight"
|
||||
dest="$(printf 'part.%s.%018d.%018d.%s' "$JOURNAL" "$TOTAL" "$((TOTAL + bytes))" "$crc")"
|
||||
mv -f "$LOG_DIR/chunk.new" "$LOG_DIR/$dest" || { log_warn_locked write-warning 'Flash: не удалось зафиксировать фрагмент журнала.'; return 1; }
|
||||
TOTAL=$((TOTAL + bytes))
|
||||
log_checkpoint_locked && log_commit_locked || { log_warn_locked write-warning 'Flash: ошибка фиксации состояния журнала.'; return 1; }
|
||||
: >"$RUN_DIR/pending"
|
||||
rm -f "$RUN_DIR/inflight"
|
||||
rm -f "$RUN_DIR/write-warning" "$RUN_DIR/queue-warning" "$RUN_DIR/buffer-warning"
|
||||
dropped="$(cat "$RUN_DIR/dropped" 2>/dev/null)"; dropped="${dropped:-0}"
|
||||
if [ "$dropped" -gt 0 ]; then
|
||||
printf '[WARN] %s owrt-remote: Восстановлена запись журнала; пропущено новых событий сверх лимита: %s.\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$dropped" >"$RUN_DIR/pending"
|
||||
rm -f "$RUN_DIR/dropped"
|
||||
fi
|
||||
fi
|
||||
log_clock >"$RUN_DIR/last-flush"
|
||||
}
|
||||
log_open() {
|
||||
log_lock || return 1
|
||||
trap 'log_unlock' EXIT
|
||||
trap 'exit 1' INT TERM
|
||||
if ! log_load_locked; then
|
||||
if [ "$PERSISTENT" = 1 ]; then
|
||||
log_fallback_locked && log_load_locked || { log_close; return 1; }
|
||||
else
|
||||
log_close; return 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
log_close() { log_unlock; trap - EXIT INT TERM; }
|
||||
log_redact() {
|
||||
# Never retain command blocks containing credentials or authenticated URLs.
|
||||
LC_ALL=C awk -v fallback="${1:-OK}" '
|
||||
@@ -110,16 +446,23 @@ log_new_fact_locked() {
|
||||
case "$key" in
|
||||
'Heartbeat: OK;'*)
|
||||
heartbeat=1
|
||||
if [ "$(cat "$LOG_DIR/heartbeat-ok" 2>/dev/null)" = "$key" ]; then return 1; fi
|
||||
printf '%s\n' "$key" >"$LOG_DIR/heartbeat-ok"
|
||||
# A recovery lets a later failure with the same cause appear again.
|
||||
if [ -f "$LOG_DIR/facts" ]; then
|
||||
awk -F '\t' '$2 !~ /Heartbeat:|heartbeat failed/ { print }' "$LOG_DIR/facts" >"$LOG_DIR/facts.new"
|
||||
mv -f "$LOG_DIR/facts.new" "$LOG_DIR/facts"
|
||||
fi ;;
|
||||
if [ "$(cat "$RUN_DIR/heartbeat-ok" 2>/dev/null)" = "$key" ]; then return 1; fi
|
||||
printf '%s\n' "$key" >"$RUN_DIR/heartbeat-ok"
|
||||
# A recovery lets the same failure appear again; keys are fixed-size hashes.
|
||||
: >"$RUN_DIR/facts" ;;
|
||||
esac
|
||||
if [ "$heartbeat" = 1 ]; then return 0; fi
|
||||
key="${1%%]*}] $key"
|
||||
# Update heartbeat recovery state before replacing the full key with its hash.
|
||||
case "$key" in
|
||||
*Heartbeat:*|*'heartbeat failed'*)
|
||||
case "$1" in '[ERROR]'*|'[WARN]'*) rm -f "$RUN_DIR/heartbeat-ok" ;; esac ;;
|
||||
*'Сервис: запуск') rm -f "$RUN_DIR/heartbeat-ok" ;;
|
||||
esac
|
||||
printf '%s\n' "${1%%]*}] $key" >"$RUN_DIR/fact-key"
|
||||
key="$(log_checksum "$RUN_DIR/fact-key")" || { rm -f "$RUN_DIR/fact-key"; return 0; }
|
||||
rm -f "$RUN_DIR/fact-key"
|
||||
# Dedup needs only a fingerprint, not the full SHA-256. Keep 64 entries <4 KiB.
|
||||
key="$(printf '%.40s' "$key")"
|
||||
now="$(log_clock)"
|
||||
# A bounded cache suppresses repeated identical facts while they continue.
|
||||
# After a quiet period a new occurrence is recorded, rather than lost forever.
|
||||
@@ -130,116 +473,213 @@ log_new_fact_locked() {
|
||||
for (i = (n > 63 ? n - 62 : 1); i <= n; i++) print saved[i]
|
||||
print now "\t" key
|
||||
exit(repeated ? 1 : 0)
|
||||
}' "$LOG_DIR/facts" >"$LOG_DIR/facts.new" 2>/dev/null
|
||||
}' "$RUN_DIR/facts" >"$RUN_DIR/facts.new" 2>/dev/null
|
||||
new_fact=$?
|
||||
mv -f "$LOG_DIR/facts.new" "$LOG_DIR/facts"
|
||||
mv -f "$RUN_DIR/facts.new" "$RUN_DIR/facts"
|
||||
[ "$new_fact" = 0 ] || return 1
|
||||
case "$key" in
|
||||
*Heartbeat:*|*'heartbeat failed'*)
|
||||
case "$1" in '[ERROR]'*|'[WARN]'*) rm -f "$LOG_DIR/heartbeat-ok" ;; esac ;;
|
||||
*'Сервис: запуск') rm -f "$LOG_DIR/heartbeat-ok" ;;
|
||||
esac
|
||||
return 0
|
||||
}
|
||||
log_append() {
|
||||
local line size
|
||||
# Process one bounded line at a time and release the lock between writes.
|
||||
local line size bytes dropped
|
||||
log_redact "${1:-OK}" | while IFS= read -r line; do
|
||||
log_lock || exit 1
|
||||
trap 'log_unlock' EXIT
|
||||
trap 'exit 0' INT TERM
|
||||
log_maintain_locked
|
||||
log_cursor_locked
|
||||
[ -f "$LOG_DIR/facts" ] || : >"$LOG_DIR/facts"
|
||||
if ! log_new_fact_locked "$line"; then
|
||||
log_unlock
|
||||
trap - EXIT INT TERM
|
||||
continue
|
||||
log_open || exit 1
|
||||
log_flush_locked || true
|
||||
[ -f "$RUN_DIR/facts" ] || : >"$RUN_DIR/facts"
|
||||
if log_new_fact_locked "$line"; then
|
||||
size="$(wc -c <"$RUN_DIR/pending")"; bytes="$(printf '%s\n' "$line" | wc -c)"
|
||||
# Reserve room for a single overflow warning, too.
|
||||
if [ "$((size + bytes))" -le "$((LOG_BUFFER_LIMIT - 1024))" ]; then
|
||||
printf '%s\n' "$line" >>"$RUN_DIR/pending"
|
||||
else
|
||||
dropped="$(cat "$RUN_DIR/dropped" 2>/dev/null)"; dropped="${dropped:-0}"
|
||||
[ "$dropped" -ge 2147483647 ] || dropped=$((dropped + 1))
|
||||
printf '%s\n' "$dropped" >"$RUN_DIR/dropped"
|
||||
log_warn_locked buffer-warning 'Буфер логов заполнен: новые события сверх лимита пропускаются, неподтверждённый журнал сохранён.'
|
||||
fi
|
||||
fi
|
||||
size="$(wc -c <"$LOG_DIR/current.log")"
|
||||
if [ "$size" -ge "$LOG_LIMIT" ]; then
|
||||
mv -f "$LOG_DIR/current.log" "$LOG_DIR/previous.log"
|
||||
: >"$LOG_DIR/current.log"
|
||||
fi
|
||||
printf '%s\n' "$line" >>"$LOG_DIR/current.log"
|
||||
printf '%s\n' "$(( $(cat "$LOG_DIR/total") + $(printf '%s\n' "$line" | wc -c) ))" >"$LOG_DIR/total"
|
||||
log_unlock
|
||||
trap - EXIT INT TERM
|
||||
log_close
|
||||
done
|
||||
}
|
||||
log_read() {
|
||||
local previous_size current_size
|
||||
log_lock || return 1
|
||||
log_maintain_locked
|
||||
# Open the retained files under the lock, then release it before streaming
|
||||
# to a slow phone. Downloads must not prevent the collector from writing.
|
||||
previous_size=0
|
||||
if [ -f "$LOG_DIR/previous.log" ]; then
|
||||
previous_size="$(wc -c <"$LOG_DIR/previous.log")"
|
||||
exec 3<"$LOG_DIR/previous.log"
|
||||
else
|
||||
exec 3</dev/null
|
||||
local file start end crc snapshot mode owner
|
||||
log_open || return 1
|
||||
# Capture only file names and the small pending buffer. Slow HTTP clients must
|
||||
# never hold the append lock; concurrent expiry may omit an already-acked chunk.
|
||||
# One viewer snapshot at a time bounds temporary files even with slow clients.
|
||||
if ! mkdir "$RUN_DIR/view-lock" 2>/dev/null; then
|
||||
owner="$(cat "$RUN_DIR/view-lock/pid" 2>/dev/null)"
|
||||
case "$owner" in ''|*[!0-9]*) ;; *) kill -0 "$owner" 2>/dev/null && { log_close; return 1; } ;; esac
|
||||
rm -f "$RUN_DIR/view-lock/pid"; rmdir "$RUN_DIR/view-lock" 2>/dev/null || { log_close; return 1; }
|
||||
mkdir "$RUN_DIR/view-lock" || { log_close; return 1; }
|
||||
fi
|
||||
current_size="$(wc -c <"$LOG_DIR/current.log")"
|
||||
exec 4<"$LOG_DIR/current.log"
|
||||
log_unlock
|
||||
if [ "${1:-}" = live ]; then
|
||||
{ head -c "$previous_size" <&3; head -c "$current_size" <&4; } | tail -c 65536
|
||||
printf '%s\n' "$$" >"$RUN_DIR/view-lock/pid"
|
||||
rm -f "$RUN_DIR"/view.*
|
||||
snapshot="$RUN_DIR/view.$$"; mode="${1:-}"
|
||||
: >"$snapshot.list"
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ ! -f "$file" ] || printf '%s\n' "$file" >>"$snapshot.list"
|
||||
done
|
||||
cp "$RUN_DIR/pending" "$snapshot.pending" || { rm -f "$RUN_DIR/view-lock/pid"; rmdir "$RUN_DIR/view-lock"; log_close; return 1; }
|
||||
log_close
|
||||
trap 'rm -f "$snapshot.list" "$snapshot.pending" "$RUN_DIR/view-lock/pid"; rmdir "$RUN_DIR/view-lock" 2>/dev/null || true' EXIT
|
||||
trap 'exit 1' INT TERM
|
||||
if [ "$mode" = live ]; then
|
||||
{ while IFS= read -r file; do [ ! -f "$file" ] || cat "$file"; done <"$snapshot.list"; cat "$snapshot.pending"; } | tail -c 65536
|
||||
else
|
||||
head -c "$previous_size" <&3
|
||||
head -c "$current_size" <&4
|
||||
while IFS= read -r file; do [ ! -f "$file" ] || cat "$file"; done <"$snapshot.list"
|
||||
cat "$snapshot.pending"
|
||||
fi
|
||||
exec 3<&- 4<&-
|
||||
rm -f "$snapshot.list" "$snapshot.pending" "$RUN_DIR/view-lock/pid"; rmdir "$RUN_DIR/view-lock"; trap - EXIT INT TERM
|
||||
}
|
||||
log_clear() {
|
||||
log_lock || return 1
|
||||
log_reset_locked
|
||||
log_unlock
|
||||
log_reset_locked() {
|
||||
local clear
|
||||
clear="${1:--}"
|
||||
# Commit a new generation before deleting its predecessor; a crash cannot
|
||||
# attach old offsets to a new journal. Explicit clear also discards unacked data.
|
||||
JOURNAL="$(log_new_id)"; ACK=0; TOTAL=0; CLEAR="$clear"
|
||||
# Chunks include their generation ID: atomically switching state makes all
|
||||
# previous chunks inactive, even if power fails during their subsequent removal.
|
||||
log_checkpoint_locked && log_commit_locked || return 1
|
||||
rm -f "$LOG_DIR"/part.* "$LOG_DIR/damaged.log"
|
||||
: >"$RUN_DIR/pending"
|
||||
rm -f "$RUN_DIR/ack" "$RUN_DIR/facts" "$RUN_DIR/heartbeat-ok" "$RUN_DIR/dropped" "$RUN_DIR/buffer-warning" "$RUN_DIR/inflight"
|
||||
log_clock >"$RUN_DIR/last-flush"
|
||||
}
|
||||
log_clear() { log_open || return 1; log_reset_locked; result=$?; log_close; return "$result"; }
|
||||
log_export() {
|
||||
local journal total previous current ack_id ack start available count clear_ack chunk
|
||||
log_lock || return 1
|
||||
log_maintain_locked
|
||||
log_cursor_locked
|
||||
journal="$(cat "$LOG_DIR/journal-id")"; total="$(cat "$LOG_DIR/total")"
|
||||
ack_id=''; ack=0
|
||||
if [ -s "$LOG_DIR/sync-ack" ]; then read -r ack_id ack <"$LOG_DIR/sync-ack"; fi
|
||||
[ "$ack_id" = "$journal" ] || ack=0
|
||||
case "$ack" in ''|*[!0-9]*) ack=0 ;; esac
|
||||
[ "$ack" -le "$total" ] || ack=0
|
||||
previous=0
|
||||
if [ -f "$LOG_DIR/previous.log" ]; then previous="$(wc -c <"$LOG_DIR/previous.log")"; exec 3<"$LOG_DIR/previous.log"; else exec 3</dev/null; fi
|
||||
current="$(wc -c <"$LOG_DIR/current.log")"; exec 4<"$LOG_DIR/current.log"
|
||||
start=$((total - previous - current)); [ "$ack" -le "$start" ] || start="$ack"
|
||||
available=$((total - start)); count="$available"; [ "$count" -le 65536 ] || count=65536
|
||||
clear_ack="$(cat "$LOG_DIR/sync-clear-ack" 2>/dev/null)"
|
||||
log_unlock
|
||||
chunk="$LOG_DIR/export.$$"
|
||||
trap 'rm -f "$chunk"; exec 3<&- 4<&-' EXIT
|
||||
: >"$chunk"
|
||||
if [ "$count" -gt 0 ]; then
|
||||
{ head -c "$previous" <&3; head -c "$current" <&4; } | tail -c "$available" | head -c "$count" >"$chunk"
|
||||
fi
|
||||
count="$(wc -c <"$chunk")"
|
||||
printf '{"journal_id":"%s","from":%s,"to":%s,"clear_ack":"%s","data":"' "$journal" "$start" "$((start + count))" "$clear_ack"
|
||||
base64 <"$chunk" | tr -d '\r\n'
|
||||
printf '"}\n'
|
||||
rm -f "$chunk"; exec 3<&- 4<&-; trap - EXIT
|
||||
local file start end crc from to skip count chunk clear
|
||||
log_open || return 1
|
||||
log_flush_locked || true
|
||||
# Reap a temporary export left by SIGKILL. The journal lock excludes another
|
||||
# exporter; there is never more than one raw packet in RAM.
|
||||
rm -f "$RUN_DIR"/export.*
|
||||
# An empty/status packet proves no bytes beyond the last actual ack.
|
||||
from="$ACK"; to="$ACK"
|
||||
chunk="$RUN_DIR/export.$$"; : >"$chunk"
|
||||
trap 'rm -f "$chunk"; log_unlock' EXIT
|
||||
# One immutable chunk per packet, never more than 16 KiB before base64.
|
||||
for file in "$LOG_DIR"/part."$JOURNAL".*; do
|
||||
[ -f "$file" ] || continue
|
||||
log_part_fields "$file" || continue
|
||||
[ "$end" -gt "$ACK" ] || continue
|
||||
from="$start"; [ "$ACK" -le "$start" ] || from="$ACK"
|
||||
skip=$((from - start)); count=$((end - from)); [ "$count" -le "$LOG_BATCH_LIMIT" ] || count="$LOG_BATCH_LIMIT"
|
||||
# dd streams a fixed-size segment; no large tail ring or shell string.
|
||||
dd if="$file" bs=1 skip="$skip" count="$count" 2>/dev/null >"$chunk" || { rm -f "$chunk"; log_close; return 1; }
|
||||
to=$((from + $(wc -c <"$chunk"))); break
|
||||
done
|
||||
clear="$CLEAR"; [ "$clear" != - ] || clear=''
|
||||
printf '{"journal_id":"%s","from":%s,"to":%s,"clear_ack":"%s","data":"' "$JOURNAL" "$from" "$to" "$clear"
|
||||
base64 <"$chunk" | tr -d '\r\n'; printf '"}\n'
|
||||
rm -f "$chunk"
|
||||
log_close
|
||||
}
|
||||
log_ack() {
|
||||
local journal offset clear_token
|
||||
journal="$1"; offset="$2"; clear_token="${3:-}"
|
||||
case "$journal$clear_token" in *[!a-zA-Z0-9._:-]*) return 1 ;; esac
|
||||
local journal offset clear
|
||||
journal="$1"; offset="$2"; clear="${3:-}"
|
||||
case "$journal$clear" in ''|*[!a-zA-Z0-9._:-]*) return 1 ;; esac
|
||||
case "$offset" in ''|*[!0-9]*) return 1 ;; esac
|
||||
log_lock || return 1
|
||||
log_maintain_locked
|
||||
if [ -n "$clear_token" ] && [ "$(cat "$LOG_DIR/sync-clear-ack" 2>/dev/null)" != "$clear_token" ]; then
|
||||
log_reset_locked
|
||||
printf '%s\n' "$clear_token" >"$LOG_DIR/sync-clear-ack"
|
||||
elif [ "$(cat "$LOG_DIR/journal-id" 2>/dev/null)" = "$journal" ]; then
|
||||
printf '%s %s\n' "$journal" "$offset" >"$LOG_DIR/sync-ack"
|
||||
[ "${#journal}" -le 80 ] && [ "${#clear}" -le 80 ] && [ "${#offset}" -le 16 ] || return 1
|
||||
log_open || return 1
|
||||
if [ -n "$clear" ] && [ "$CLEAR" != "$clear" ]; then
|
||||
log_reset_locked "$clear" || { log_close; return 1; }
|
||||
elif [ "$JOURNAL" = "$journal" ]; then
|
||||
if [ "$offset" -gt "$TOTAL" ] || [ "$offset" -lt "$ACK" ]; then log_close; return 1; fi
|
||||
ACK="$offset"
|
||||
# Checkpoint with the next periodic flush. A crash only replays confirmed
|
||||
# bytes; the existing Hub journal-ID/offset protocol suppresses duplicates.
|
||||
printf '%s %s\n' "$JOURNAL" "$ACK" >"$RUN_DIR/ack.new" && mv -f "$RUN_DIR/ack.new" "$RUN_DIR/ack"
|
||||
fi
|
||||
log_unlock
|
||||
log_close
|
||||
}
|
||||
log_flush() { log_open || return 1; log_flush_locked "${1:-}"; result=$?; log_close; return "$result"; }
|
||||
log_status() {
|
||||
log_open || return 1
|
||||
printf 'storage=%s\npersistent=%s\nruntime=%s\nlimit_bytes=%s\nmax_parts=%s\nbuffer_limit_bytes=%s\nflush_seconds=%s\njournal_id=%s\ncommitted_bytes=%s\nack_bytes=%s\npending_bytes=%s\n' "$LOG_DIR" "$PERSISTENT" "$RUN_DIR" "$LOG_LIMIT" "$LOG_PART_LIMIT" "$LOG_BUFFER_LIMIT" "$LOG_FLUSH_SECONDS" "$JOURNAL" "$TOTAL" "$ACK" "$(wc -c <"$RUN_DIR/pending")"
|
||||
log_close
|
||||
}
|
||||
log_migrate() {
|
||||
local legacy name free mount fs flags old_id old_total old_ack ack_id previous current base file bytes index size crc target token
|
||||
legacy=/tmp/owrt-remote-logs
|
||||
[ -d "$legacy" ] || return 0
|
||||
[ -f "$legacy/current.log" ] || [ -f "$legacy/previous.log" ] || return 0
|
||||
log_open || return 1
|
||||
read -r mount fs flags <<EOF
|
||||
$(log_mount "$legacy")
|
||||
EOF
|
||||
if [ "$fs" != tmpfs ]; then log_close; return 0; fi
|
||||
if [ "$PERSISTENT" != 1 ]; then
|
||||
# Never leave the old 2 MiB RAM journal alongside a 64 KiB fallback. Keep
|
||||
# one bounded tail, with an explicit warning about the discarded history.
|
||||
# Legacy bytes were already redacted. Copy a bounded tail as a stream,
|
||||
# avoiding a slow per-line replay or a second long-lived RAM archive.
|
||||
free=$((LOG_BUFFER_LIMIT - 1024 - $(wc -c <"$RUN_DIR/pending")))
|
||||
[ "$free" -le 8192 ] || free=8192
|
||||
if [ "$free" -gt 0 ]; then
|
||||
{ cat "$legacy/previous.log" "$legacy/current.log" 2>/dev/null; } | tail -c "$free" >>"$RUN_DIR/pending"
|
||||
fi
|
||||
log_warn_locked legacy-warning 'Flash недоступна: от старого RAM-журнала сохранён только ограниченный хвост; остальная история отброшена.'
|
||||
else
|
||||
# Called with the old collector stopped. Retain its journal identity and
|
||||
# byte cursor, so undelivered history resumes without duplicating Hub data.
|
||||
free="$(df -Pk "$LOG_DIR" | awk 'NR==2 {print $4}')"
|
||||
case "$free" in ''|*[!0-9]*) free=0 ;; esac
|
||||
if [ "$free" -lt 6144 ]; then log_close; printf 'Remote: legacy archive retained in RAM; migration needs 6 MiB free Flash.\n' >&2; return 1; fi
|
||||
old_id="$(cat "$legacy/journal-id" 2>/dev/null)"
|
||||
case "$old_id" in ''|*[!a-zA-Z0-9._:-]*) old_id="$JOURNAL" ;; esac
|
||||
old_total="$(cat "$legacy/total" 2>/dev/null)"
|
||||
previous=0; current=0
|
||||
[ ! -f "$legacy/previous.log" ] || previous="$(wc -c <"$legacy/previous.log")"
|
||||
[ ! -f "$legacy/current.log" ] || current="$(wc -c <"$legacy/current.log")"
|
||||
bytes=$((previous + current))
|
||||
if [ "$bytes" -gt "$LOG_LIMIT" ]; then log_close; printf 'Remote: legacy journal exceeds 2 MiB; export it before updating.\n' >&2; return 1; fi
|
||||
case "$old_total" in ''|*[!0-9]*) old_total="$bytes" ;; esac
|
||||
[ "$old_total" -ge "$bytes" ] || old_total="$bytes"
|
||||
base=$((old_total - bytes)); old_ack=0; ack_id=''
|
||||
if [ -f "$legacy/sync-ack" ]; then read -r ack_id old_ack <"$legacy/sync-ack"; fi
|
||||
case "$old_ack" in ''|*[!0-9]*) old_ack=0 ;; esac
|
||||
[ "$ack_id" = "$old_id" ] && [ "$old_ack" -le "$old_total" ] || old_ack=0
|
||||
# Do not replace a nonempty new journal (e.g. after a failed retry).
|
||||
if [ "$TOTAL" -gt 0 ] && [ "$JOURNAL" != "$old_id" ]; then log_close; printf 'Remote: legacy journal conflict; no history was removed.\n' >&2; return 1; fi
|
||||
JOURNAL="$old_id"; TOTAL="$base"; ACK="$old_ack"
|
||||
token="$(cat "$legacy/sync-clear-ack" 2>/dev/null)"
|
||||
case "$token" in ''|*[!a-zA-Z0-9._:-]*) CLEAR='-' ;; *) CLEAR="$token" ;; esac
|
||||
# A staging directory on the same Flash filesystem keeps restart recovery
|
||||
# from importing only half a legacy journal after power loss.
|
||||
mkdir -p "$LOG_DIR/migrating" || { log_close; return 1; }
|
||||
for file in "$legacy/previous.log" "$legacy/current.log"; do
|
||||
[ -f "$file" ] || continue
|
||||
size="$(wc -c <"$file")"; index=0
|
||||
while [ "$size" -gt 0 ]; do
|
||||
dd if="$file" bs=16384 skip="$index" count=1 2>/dev/null >"$LOG_DIR/migrating/chunk.new" || { log_close; return 1; }
|
||||
bytes="$(wc -c <"$LOG_DIR/migrating/chunk.new")"
|
||||
[ "$bytes" -gt 0 ] || { log_close; return 1; }
|
||||
crc="$(log_checksum "$LOG_DIR/migrating/chunk.new")" || { log_close; return 1; }
|
||||
target="$(printf 'part.%s.%018d.%018d.%s' "$JOURNAL" "$TOTAL" "$((TOTAL + bytes))" "$crc")"
|
||||
mv -f "$LOG_DIR/migrating/chunk.new" "$LOG_DIR/migrating/$target" || { log_close; return 1; }
|
||||
TOTAL=$((TOTAL + bytes)); size=$((size - bytes)); index=$((index + 1))
|
||||
done
|
||||
done
|
||||
# Commit a recovery manifest before publishing any fragments. On reboot,
|
||||
# even when the old /tmp is gone, the entire staged queue can be finished.
|
||||
printf '%s %s %s %s\n' "$JOURNAL" "$ACK" "$TOTAL" "$CLEAR" >"$LOG_DIR/migrating/state.new" &&
|
||||
mv -f "$LOG_DIR/migrating/state.new" "$LOG_DIR/migrating/state" || { log_close; return 1; }
|
||||
sync -f "$LOG_DIR" 2>/dev/null || sync
|
||||
log_finish_migration_locked || { log_close; return 1; }
|
||||
rm -f "$RUN_DIR/ack"
|
||||
sync -f "$LOG_DIR" 2>/dev/null || sync
|
||||
fi
|
||||
rm -f "$legacy/current.log" "$legacy/previous.log" "$legacy/age" "$legacy/journal-id" "$legacy/total" "$legacy/sync-ack" "$legacy/sync-clear-ack" "$legacy/facts" "$legacy/facts.new" "$legacy/heartbeat-ok"
|
||||
log_close
|
||||
}
|
||||
log_finish_migration_locked() {
|
||||
local file
|
||||
[ -s "$LOG_DIR/migrating/state" ] || return 0
|
||||
for file in "$LOG_DIR"/migrating/part.*; do [ ! -f "$file" ] || mv -f "$file" "$LOG_DIR/" || return 1; done
|
||||
mv -f "$LOG_DIR/migrating/state" "$LOG_DIR/state" || return 1
|
||||
sync -f "$LOG_DIR" 2>/dev/null || sync
|
||||
rmdir "$LOG_DIR/migrating" 2>/dev/null || true
|
||||
rm -f "$RUN_DIR/recovered" "$RUN_DIR/recovered-v2" "$RUN_DIR/ack"
|
||||
}
|
||||
log_sync_loop() {
|
||||
while :; do
|
||||
@@ -248,30 +688,37 @@ log_sync_loop() {
|
||||
done
|
||||
}
|
||||
log_collect() {
|
||||
# These variables must remain visible to EXIT even after this function returns.
|
||||
log_init || return 1
|
||||
fifo="$LOG_DIR/stream.$$"
|
||||
mkfifo "$fifo" || return 1
|
||||
reader=''; writer=''; timer=''
|
||||
trap 'trap - EXIT; for child in "$reader" "$writer" "$timer"; do [ -z "$child" ] || kill "$child" 2>/dev/null || true; done; wait; rm -f "$fifo"' EXIT
|
||||
local owner
|
||||
log_open || return 1
|
||||
log_close
|
||||
if ! mkdir "$RUN_DIR/collector-lock" 2>/dev/null; then
|
||||
owner="$(cat "$RUN_DIR/collector-lock/pid" 2>/dev/null)"
|
||||
case "$owner" in ''|*[!0-9]*) ;; *) kill -0 "$owner" 2>/dev/null && return 0 ;; esac
|
||||
rm -f "$RUN_DIR/collector-lock/pid"; rmdir "$RUN_DIR/collector-lock" 2>/dev/null || return 1
|
||||
mkdir "$RUN_DIR/collector-lock" || return 1
|
||||
fi
|
||||
log_self_pid >"$RUN_DIR/collector-lock/pid"
|
||||
rm -f "$RUN_DIR"/stream.*
|
||||
fifo="$RUN_DIR/stream.$$"; reader=''; writer=''; timer=''
|
||||
trap 'trap - EXIT; for child in "$reader" "$writer" "$timer"; do [ -z "$child" ] || kill "$child" 2>/dev/null || true; done; wait; rm -f "$fifo" "$RUN_DIR/collector-lock/pid"; rmdir "$RUN_DIR/collector-lock" 2>/dev/null || true; (log_flush force) || true' EXIT
|
||||
trap 'exit 0' INT TERM
|
||||
# Follow only new Remote/Xray service records: a clear must not reload history.
|
||||
logread -f -e 'owrt-remote\(\[[0-9][0-9]*\]\)\{0,1\}:' >"$fifo" 2>/dev/null &
|
||||
reader=$!
|
||||
while IFS= read -r line; do printf '%s\n' "$line" | log_append; done <"$fifo" &
|
||||
writer=$!
|
||||
mkfifo "$fifo" || return 1
|
||||
logread -f -e 'owrt-remote\(\[[0-9][0-9]*\]\)\{0,1\}:' >"$fifo" 2>/dev/null & reader=$!
|
||||
while IFS= read -r line; do printf '%s\n' "$line" | log_append; done <"$fifo" & writer=$!
|
||||
while :; do
|
||||
log_lock || return 1
|
||||
log_maintain_locked
|
||||
log_unlock
|
||||
sleep 60 & timer=$!
|
||||
# Lock-operation traps must not replace the collector's child/FIFO cleanup.
|
||||
(log_flush) || true
|
||||
sleep 5 & timer=$!
|
||||
wait "$timer" || return 0
|
||||
kill -0 "$reader" 2>/dev/null && kill -0 "$writer" 2>/dev/null || return 1
|
||||
done
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
check) log_init ;;
|
||||
check) log_open && log_close ;;
|
||||
status) log_status ;;
|
||||
migrate) log_migrate ;;
|
||||
flush) log_flush "${2:-}" ;;
|
||||
collect) log_collect ;;
|
||||
read) log_read "${2:-}" ;;
|
||||
clear) log_clear ;;
|
||||
@@ -282,5 +729,5 @@ case "${1:-}" in
|
||||
event)
|
||||
case "${3:-ok}" in bad|error) level=ERROR ;; warn) level=WARN ;; *) level=OK ;; esac
|
||||
{ printf '%s owrt-remote: %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$2"; cat; } | log_append "$level" ;;
|
||||
*) printf 'Usage: logs.sh collect|read [live]|clear|append|event MESSAGE\n' >&2; exit 2 ;;
|
||||
*) printf 'Usage: logs.sh collect|read [live]|clear|append|event MESSAGE|status|flush [force]\n' >&2; exit 2 ;;
|
||||
esac
|
||||
+157
-13
@@ -951,6 +951,13 @@ post_json() (
|
||||
die "для heartbeat нужен uclient-fetch, wget или curl"
|
||||
)
|
||||
|
||||
log_post_json_bounded() (
|
||||
# Also bounds remote_curl_post's private response body. The subshell limit
|
||||
# cannot affect heartbeat/Xray. POSIX -f uses 512-byte blocks (32 KiB here).
|
||||
ulimit -f 64 || exit 1
|
||||
post_json "$1" "$2" "$3"
|
||||
)
|
||||
|
||||
global_logs_state() {
|
||||
local enabled xray
|
||||
enabled=false; xray=stopped
|
||||
@@ -960,13 +967,17 @@ global_logs_state() {
|
||||
}
|
||||
|
||||
sync_global_logs() (
|
||||
local helper router_id batch file response endpoint base token journal offset clear_token rc sync_lock owner attempt
|
||||
local helper router_id file response endpoint base token journal offset clear_token rc sync_lock owner attempt runtime TMPDIR stale
|
||||
helper="${OWRT_REMOTE_LOG_HELPER:-/usr/lib/owrt-remote/logs.sh}"
|
||||
[ -r "$helper" ] || return 0
|
||||
token="$(uci_get hub_token '')"; [ -n "$token" ] || return 0
|
||||
# The journal is independent of Remote's enabled flag and Xray lifecycle.
|
||||
sh "$helper" check || return 1
|
||||
sync_lock="${OWRT_REMOTE_LOG_DIR:-/tmp/owrt-remote-logs}/sync-lock"
|
||||
runtime="${OWRT_REMOTE_LOG_RUNTIME:-/tmp/owrt-remote-log-runtime}"
|
||||
# Older helpers create only their journal directory. An independent agent
|
||||
# update still needs a private runtime directory for delivery and its lock.
|
||||
(umask 077; mkdir -p "$runtime") || return 1
|
||||
sync_lock="$runtime/sync-lock"
|
||||
if ! mkdir "$sync_lock" 2>/dev/null; then
|
||||
# Give a sender time to write its PID, then recover an abandoned lock.
|
||||
for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
@@ -975,23 +986,56 @@ sync_global_logs() (
|
||||
done
|
||||
owner="$(cat "$sync_lock/pid" 2>/dev/null)"
|
||||
case "$owner" in ''|*[!0-9]*) ;; *) kill -0 "$owner" 2>/dev/null && return 0 ;; esac
|
||||
rm -f "$sync_lock/pid"; rmdir "$sync_lock" 2>/dev/null || return 0
|
||||
if ! mkdir "$sync_lock/reap" 2>/dev/null; then
|
||||
owner="$(cat "$sync_lock/reap/pid" 2>/dev/null)"
|
||||
case "$owner" in ''|*[!0-9]*) sleep 1 ;; *) kill -0 "$owner" 2>/dev/null && return 0 ;; esac
|
||||
rm -f "$sync_lock/reap/pid"; rmdir "$sync_lock/reap" 2>/dev/null || return 0
|
||||
return 0
|
||||
fi
|
||||
printf '%s\n' "$$" >"$sync_lock/reap/pid"
|
||||
owner="$(cat "$sync_lock/pid" 2>/dev/null)"
|
||||
case "$owner" in ''|*[!0-9]*) ;; *) if kill -0 "$owner" 2>/dev/null; then rm -f "$sync_lock/reap/pid"; rmdir "$sync_lock/reap"; return 0; fi ;; esac
|
||||
rm -f "$sync_lock/pid"
|
||||
rm -f "$sync_lock/reap/pid"
|
||||
rmdir "$sync_lock/reap" || return 0
|
||||
rmdir "$sync_lock" 2>/dev/null || return 0
|
||||
mkdir "$sync_lock" 2>/dev/null || return 0
|
||||
fi
|
||||
printf '%s\n' "$$" >"$sync_lock/pid"
|
||||
# ash keeps $$ unchanged inside a subshell. Store the actual sender PID so
|
||||
# a killed sender is recoverable while its parent service is still alive.
|
||||
if read -r owner attempt </proc/self/stat 2>/dev/null; then
|
||||
printf '%s\n' "$owner" >"$sync_lock/pid"
|
||||
else
|
||||
printf '%s\n' "$$" >"$sync_lock/pid"
|
||||
fi
|
||||
file=''; response=''
|
||||
trap 'rm -f "$file" "$response" "$sync_lock/pid"; rmdir "$sync_lock" 2>/dev/null || true' EXIT
|
||||
trap 'exit 1' INT TERM
|
||||
# A killed sender can leave one bounded payload; reclaim it under the singleton.
|
||||
rm -f "$runtime"/sync.*
|
||||
# Give this sender its own temporary HTTP directory; never sweep heartbeat's
|
||||
# files. Clean a curl subprocess killed before its EXIT trap could run.
|
||||
TMPDIR="$runtime/http"; export TMPDIR
|
||||
mkdir -p "$TMPDIR" || return 1
|
||||
for stale in "$TMPDIR"/owrt-remote-http.*; do
|
||||
[ -d "$stale" ] || continue
|
||||
rm -f "$stale/body" "$stale/stats" "$stale/error" "$stale/dns"
|
||||
rmdir "$stale" 2>/dev/null || true
|
||||
done
|
||||
if truthy "$(uci_get wan_direct 0)" && [ ! -s "$WAN_STATE/active-interface" ]; then wan_prepare || return 1; fi
|
||||
batch="$(sh "$helper" export)" || return 1
|
||||
router_id="$(safe_id "$(uci_get router_id main)")"
|
||||
# export verifies that this directory is on tmpfs; payloads also stay in RAM.
|
||||
# Export at most 16 KiB of raw records directly into a bounded request file.
|
||||
# Neither the Flash journal nor a base64 packet is copied into a shell variable.
|
||||
umask 077
|
||||
file="${OWRT_REMOTE_LOG_DIR:-/tmp/owrt-remote-logs}/sync.$$"; response="$file.response"
|
||||
printf '{"id":%s,"batch":%s,"state":%s}\n' "$(json_string "$router_id")" "$batch" "$(global_logs_state)" >"$file"
|
||||
file="$runtime/sync.$$"; response="$file.response"
|
||||
{
|
||||
printf '{"id":%s,"batch":' "$(json_string "$router_id")"
|
||||
sh "$helper" export || exit 1
|
||||
printf ',"state":%s}\n' "$(global_logs_state)"
|
||||
} >"$file" || return 1
|
||||
for endpoint in $(heartbeat_candidate_endpoints); do
|
||||
base="$(heartbeat_base_from_endpoint "$endpoint")"
|
||||
if post_json "$base/api/router-logs" "$token" "$file" >"$response" 2>/dev/null &&
|
||||
if log_post_json_bounded "$base/api/router-logs" "$token" "$file" >"$response" 2>/dev/null &&
|
||||
grep -Eq '"ok"[[:space:]]*:[[:space:]]*true' "$response"; then
|
||||
journal="$(jsonfilter -i "$response" -e '@.journal_id' 2>/dev/null)"
|
||||
offset="$(jsonfilter -i "$response" -e '@.offset' 2>/dev/null)"
|
||||
@@ -1136,8 +1180,9 @@ heartbeat_diagnostics() {
|
||||
if [ -r "$state" ]; then cat "$state"; else printf 'Замеров пока нет; выполни owrt-remote heartbeat.\n'; fi
|
||||
}
|
||||
|
||||
send_heartbeat() (
|
||||
send_heartbeat() {
|
||||
local hub_url token endpoint tmp response event used fallback_base errors rc dir metrics http attempt max_attempts reason previous previous_reason safe_endpoint elapsed started diagnostic
|
||||
(
|
||||
event="${1:-}"
|
||||
hub_url="$(uci_get hub_url '')"
|
||||
token="$(uci_get hub_token '')"
|
||||
@@ -1210,19 +1255,96 @@ send_heartbeat() (
|
||||
[ "$used" = "1" ] || die "hub_url/public_url/vps_host пустые"
|
||||
case "$hub_url" in https://*) log 'Heartbeat: fallback остановлен; reason=https_required; автоматический переход на HTTP отключён' ;; esac
|
||||
return 1
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
wan_reconcile_xray() {
|
||||
local config bound device
|
||||
truthy "$(uci_get enabled 0)" || return 0
|
||||
device="$(wan_device)" || return 1
|
||||
if ! xray_running; then
|
||||
wan_recover_xray
|
||||
return $?
|
||||
fi
|
||||
config="$(uci_get xray_config /etc/xray/owrt-remote-client.json)"
|
||||
[ -r "$config" ] || return 0
|
||||
bound="$(jsonfilter -i "$config" -e '@.outbounds[@.tag="vps-interconn"].streamSettings.sockopt.interface' 2>/dev/null)"
|
||||
device="$(wan_device)" || return 1
|
||||
[ "$bound" != "$device" ] || return 0
|
||||
xray_running || return 0
|
||||
wan_reconnect_xray "$bound" "$device"
|
||||
}
|
||||
|
||||
wan_recover_xray() {
|
||||
local lock owner rest state running heartbeat now last attempt
|
||||
(
|
||||
umask 077
|
||||
lock="${TMPDIR:-/tmp}/owrt-remote-xray-recovery"
|
||||
# A stopped procd instance cannot be revived by sending it a signal.
|
||||
# Serialize re-registration; do not interrupt the healthy heartbeat instance.
|
||||
attempt=0
|
||||
while ! mkdir "$lock" 2>/dev/null; do
|
||||
owner="$(cat "$lock/pid" 2>/dev/null)"
|
||||
case "$owner" in
|
||||
''|*[!0-9]*)
|
||||
# Allow a new worker to record its PID before reaping an empty lock.
|
||||
if [ "$attempt" -lt 10 ]; then attempt=$((attempt+1)); sleep 0.1; continue; fi ;;
|
||||
*) kill -0 "$owner" 2>/dev/null && return 0 ;;
|
||||
esac
|
||||
wan_recovery_reap "$lock" "$attempt"
|
||||
attempt=$((attempt+1)); [ "$attempt" -lt 20 ] || return 0
|
||||
done
|
||||
trap 'rm -f "$lock/pid"; rmdir "$lock" 2>/dev/null || true' EXIT
|
||||
trap 'exit 1' INT TERM
|
||||
owner=$$
|
||||
# ash keeps $$ unchanged in a subshell; /proc/self identifies this worker.
|
||||
if [ -r /proc/self/stat ]; then read -r owner rest </proc/self/stat || owner=$$; fi
|
||||
printf '%s\n' "$owner" >"$lock/pid"
|
||||
truthy "$(uci_get enabled 0)" || return 0
|
||||
wan_select_interface >/dev/null || return 0
|
||||
xray_running && return 0
|
||||
state="$(ubus call service list '{"name":"owrt-remote"}' 2>/dev/null)" || return 1
|
||||
running="$(printf '%s' "$state" | jsonfilter -e '@["owrt-remote"].instances.xray.running' 2>/dev/null)"
|
||||
[ "$running" != true ] || return 0
|
||||
heartbeat="$(printf '%s' "$state" | jsonfilter -e '@["owrt-remote"].instances.heartbeat.running' 2>/dev/null)"
|
||||
# Respect an explicit service stop, including stops made outside the panel.
|
||||
[ "$heartbeat" = true ] || return 0
|
||||
now="$(uptime_seconds)"
|
||||
now="$(safe_num "$now" 0)"
|
||||
[ "$now" -gt 0 ] || now="$(safe_num "$(date +%s 2>/dev/null)" 0)"
|
||||
last="$(safe_num "$(cat "${TMPDIR:-/tmp}/owrt-remote-xray-recovery.last" 2>/dev/null)" 0)"
|
||||
if [ "$last" -gt 0 ] && [ "$now" -ge "$last" ] && [ $((now-last)) -lt 300 ]; then return 0; fi
|
||||
printf '%s\n' "$now" >"${TMPDIR:-/tmp}/owrt-remote-xray-recovery.last"
|
||||
log 'Xray: WAN доступен, туннель остановлен; запрошен повторный запуск отдельного instance'
|
||||
ubus call service add '{"name":"owrt-remote","instances":{"xray":{"command":["/usr/sbin/owrt-remote","run"],"env":{"OWRT_REMOTE_SERVICE_LOG":"1"},"respawn":["3600","5","5"],"stdout":true,"stderr":true}}}' >/dev/null || {
|
||||
log 'ОШИБКА: procd не принял запрос восстановления Xray; Heartbeat продолжает работать'
|
||||
return 1
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
wan_recovery_reap() {
|
||||
local owner reaper rest
|
||||
# The reaper itself must be serialized and recoverable after SIGKILL.
|
||||
if ! mkdir "$1/reap" 2>/dev/null; then
|
||||
reaper="$(cat "$1/reap/pid" 2>/dev/null)"
|
||||
case "$reaper" in
|
||||
''|*[!0-9]*) [ "$2" -ge 15 ] || return 0 ;;
|
||||
*) kill -0 "$reaper" 2>/dev/null && return 0 ;;
|
||||
esac
|
||||
rm -f "$1/reap/pid"; rmdir "$1/reap" 2>/dev/null || return 0
|
||||
return 0
|
||||
fi
|
||||
reaper=$$
|
||||
if [ -r /proc/self/stat ]; then read -r reaper rest </proc/self/stat || reaper=$$; fi
|
||||
printf '%s\n' "$reaper" >"$1/reap/pid"
|
||||
owner="$(cat "$1/pid" 2>/dev/null)"
|
||||
case "$owner" in
|
||||
''|*[!0-9]*) ;;
|
||||
*) if kill -0 "$owner" 2>/dev/null; then rm -f "$1/reap/pid"; rmdir "$1/reap"; return 0; fi ;;
|
||||
esac
|
||||
rm -f "$1/pid" "$1/reap/pid"
|
||||
rmdir "$1/reap" 2>/dev/null && rmdir "$1" 2>/dev/null || true
|
||||
}
|
||||
|
||||
wan_reconnect_xray() {
|
||||
local restart_lock owner
|
||||
restart_lock="${TMPDIR:-/tmp}/owrt-remote-wan-restart"
|
||||
@@ -1265,6 +1387,10 @@ heartbeat_self_heal() {
|
||||
printf '%s\n' "$fails" >"$fail_file"
|
||||
|
||||
[ "$fails" -lt "$threshold" ] && return 0
|
||||
if truthy "$(uci_get wan_direct 0)" && ! wan_select_interface >/dev/null; then
|
||||
log 'Heartbeat: перезапуск отложен; ни один выбранный WAN не имеет физического IPv4 default route'
|
||||
return 0
|
||||
fi
|
||||
if active_ssh_session; then
|
||||
log "heartbeat failed ${threshold} times, restart postponed because SSH session is active"
|
||||
return 0
|
||||
@@ -1535,11 +1661,29 @@ ensure_xray_available_with_retry() {
|
||||
return 1
|
||||
}
|
||||
|
||||
wait_for_selected_wan() {
|
||||
local waiting
|
||||
waiting=0
|
||||
while truthy "$(uci_get wan_direct 0)"; do
|
||||
truthy "$(uci_get enabled 0)" || return 1
|
||||
if wan_select_interface >/dev/null; then
|
||||
[ "$waiting" = 0 ] || log 'Xray: выбранный WAN снова доступен; продолжается запуск туннеля'
|
||||
return 0
|
||||
fi
|
||||
[ "$waiting" = 1 ] || log 'Xray: ожидание выбранного WAN с физическим IPv4 default route; конфигурация сохранена'
|
||||
waiting=1
|
||||
sleep 5
|
||||
done
|
||||
}
|
||||
|
||||
run_xray() {
|
||||
local enabled xray_bin xray_config
|
||||
enabled="$(uci_get enabled 0)"
|
||||
truthy "$enabled" || die "service disabled: set option enabled '1'"
|
||||
|
||||
# Missing WAN is a temporary condition, not a failed Xray configuration.
|
||||
# Stay supervised instead of exhausting procd's six startup attempts.
|
||||
wait_for_selected_wan || return 1
|
||||
ensure_xray_available_with_retry || die "cannot prepare Xray for tunnel"
|
||||
xray_bin="$(xray_bin_path)"
|
||||
xray_config="$(render_client_config)" || die "cannot create Xray configuration"
|
||||
|
||||
@@ -147,7 +147,7 @@ logs_response() {
|
||||
fi
|
||||
if ! sh "$LOG_HELPER" check >/dev/null 2>&1; then
|
||||
printf 'Status: 503 Service Unavailable\r\nContent-Type: text/plain; charset=utf-8\r\nCache-Control: no-store\r\n\r\n'
|
||||
printf 'Журнал недоступен: для логов требуется ОЗУ (tmpfs). Запись на Flash запрещена.\n'
|
||||
printf 'Журнал временно недоступен: не удалось подготовить хранилище или получить блокировку. Настройки роутера не изменены.\n'
|
||||
return
|
||||
fi
|
||||
if [ "$mode" = clear ]; then
|
||||
@@ -505,7 +505,7 @@ render_page() {
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="owrt-remote-ui-version" content="$OWRT_REMOTE_UI_VERSION">
|
||||
<title>OpenWrt Remote Hub v116</title>
|
||||
<title>OpenWrt Remote Hub v117</title>
|
||||
<style>
|
||||
:root{color-scheme:dark;--bg:#07040f;--panel:rgba(19,14,32,.88);--text:#f7f2ff;--muted:#b9adc9;--line:rgba(169,126,255,.24);--blue:#7c3aed;--green:#22c55e;--red:#fb7185;--amber:#f59e0b;--teal:#a855f7;--grid:rgba(168,85,247,.16)}
|
||||
*{box-sizing:border-box}body{position:relative;min-height:100vh;margin:0;overflow-x:hidden;background-color:var(--bg);background-image:radial-gradient(circle at 12% 8%,rgba(168,85,247,.46),transparent 31%),radial-gradient(circle at 82% 12%,rgba(79,70,229,.38),transparent 30%),radial-gradient(circle at 50% 105%,rgba(236,72,153,.26),transparent 35%),linear-gradient(145deg,#07040f,#120a24 48%,#05030a),repeating-linear-gradient(0deg,transparent 0 30px,var(--grid) 31px),repeating-linear-gradient(90deg,transparent 0 30px,var(--grid) 31px);background-size:130% 130%,140% 140%,135% 135%,100% 100%,31px 31px,31px 31px;background-attachment:fixed;color:var(--text);font:14px/1.45 system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;animation:bgFlow 22s ease-in-out infinite alternate}
|
||||
@@ -540,7 +540,7 @@ form{margin:0}.formgrid{display:grid;grid-template-columns:repeat(2,minmax(0,1fr
|
||||
<div class="brand">
|
||||
<div class="brandPanel">
|
||||
<div class="titleRow">
|
||||
<h1 class="appBanner"><span>OpenWrt Remote Hub <span class="appBannerVersion">v116</span></span></h1>
|
||||
<h1 class="appBanner"><span>OpenWrt Remote Hub <span class="appBannerVersion">v117</span></span></h1>
|
||||
</div>
|
||||
<div class="links">
|
||||
<a class="uiPill" href="https://t.me/kzolotarev95" target="_blank" rel="noopener noreferrer">Telegram</a>
|
||||
@@ -564,7 +564,7 @@ form{margin:0}.formgrid{display:grid;grid-template-columns:repeat(2,minmax(0,1fr
|
||||
|
||||
<section class="panel globalLogsPanel" id="globalLogsPanel" hidden>
|
||||
<div class="panelHead"><h2>Глобал логи</h2><button class="btn" id="globalLogsClose" type="button">Закрыть</button></div>
|
||||
<blockquote class="globalLogsNote">Действия панели, изменения связи с Hub и WAN, предупреждения и ошибки Xray. Одинаковые события не дублируются. Журнал хранится только в ОЗУ, очищается после перезагрузки и каждые 24 часа. Максимум около 2 МБ. Ключи и пароли скрываются.</blockquote>
|
||||
<blockquote class="globalLogsNote">Действия панели, Heartbeat, WAN и ошибки Xray. Журнал сохраняется на проверенном Flash-хранилище: до 2 МиБ, до 256 фрагментов; доставленные записи хранятся до 24 часов. Запись пакетами раз в минуту; последние 16 КиБ буфера могут потеряться при внезапном отключении питания. При недоступной Flash — ограниченный журнал в ОЗУ с предупреждением. Переполнение не удаляет неподтверждённые записи: новые события сверх лимита пропускаются. Ключи и пароли скрываются.</blockquote>
|
||||
<div class="globalLogsLegend" aria-label="Цвета журнала"><span class="globalLogOk">● Успешно</span><span class="globalLogWarn">● Предупреждение</span><span class="globalLogError">● Ошибка</span></div>
|
||||
<div class="actions">
|
||||
<a class="btn" href="?key=$(h "$key")&action=logs_download">Скачать логи</a>
|
||||
|
||||
@@ -14,7 +14,7 @@ return view.extend({
|
||||
|
||||
if (!key) {
|
||||
return E('div', { 'class': 'cbi-map' }, [
|
||||
E('h2', {}, _('OpenWrt Remote Hub v116')),
|
||||
E('h2', {}, _('OpenWrt Remote Hub v117')),
|
||||
E('div', { 'class': 'alert-message warning' }, [
|
||||
_('Web key not found. Reinstall the module or create /etc/owrt-remote/web.key.')
|
||||
])
|
||||
@@ -34,7 +34,7 @@ return view.extend({
|
||||
}, 100);
|
||||
|
||||
return E('div', { 'class': 'cbi-map' }, [
|
||||
E('h2', {}, _('OpenWrt Remote Hub v116')),
|
||||
E('h2', {}, _('OpenWrt Remote Hub v117')),
|
||||
E('p', {}, _('Opening remote access panel...')),
|
||||
E('p', {}, [
|
||||
E('a', { 'class': 'btn cbi-button cbi-button-apply', 'href': url }, _('Open panel'))
|
||||
|
||||
@@ -214,6 +214,12 @@ install_xray_runtime() {
|
||||
"$remote_bin" install-xray-tmp || die "failed to install Xray to /tmp"
|
||||
}
|
||||
|
||||
# Stop the old log writer before replacing its storage code. Its bounded legacy
|
||||
# archive is migrated by the new log service, without moving any other /tmp data.
|
||||
if [ "${ROOT%/}" = "" ] && [ -x /etc/init.d/owrt-remote-logs ]; then
|
||||
/etc/init.d/owrt-remote-logs stop || true
|
||||
fi
|
||||
|
||||
install_file "usr/sbin/owrt-remote" 0755
|
||||
install_file "usr/lib/owrt-remote/wan.sh" 0644
|
||||
install_file "usr/lib/owrt-remote/logs.sh" 0644
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Heartbeat: HTTPS, авторизация и диагностика (v116)
|
||||
# Heartbeat: HTTPS, авторизация и диагностика (v117)
|
||||
|
||||
## Разбор сбоя 15.10.2026
|
||||
|
||||
@@ -82,7 +82,7 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\
|
||||
Обновляются агент и WAN helper. Перезапускается только экземпляр Heartbeat
|
||||
под procd; Xray и удалённый туннель скрипт не перезапускает. Остановленный
|
||||
экземпляр остаётся остановленным. Резервная копия первых файлов сохраняется
|
||||
в `/root/owrt-remote-backup-before-heartbeat-transport-v116`.
|
||||
в `/root/owrt-remote-backup-before-heartbeat-transport-v117`.
|
||||
Маркер `HEARTBEAT_TRANSPORT_ROUTER_Install_OK` подтверждает установку кода,
|
||||
а доступность Hub проверяется отдельно:
|
||||
|
||||
|
||||
+12
-12
@@ -1,4 +1,4 @@
|
||||
# Фиксация VPS host и независимое соединение Remote через WAN (v116)
|
||||
# Фиксация VPS host и независимое соединение Remote через WAN (v117)
|
||||
|
||||
`vps_host_mode=manual` сохраняет заданный на роутере адрес. Это значение
|
||||
используется и когда опция отсутствует у старой установки. Только явно
|
||||
@@ -42,8 +42,8 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-hub-url-fix.
|
||||
```
|
||||
|
||||
Откат: те же команды с `-Action Rollback`, без `-HubUrl`. Резервные копии:
|
||||
`/root/owrt-remote-backup-before-hub-url-v116` на роутере и
|
||||
`/opt/owrt-remote/owrt-remote-hub.py.bak-before-hub-url-v116` на VPS.
|
||||
`/root/owrt-remote-backup-before-hub-url-v117` на роутере и
|
||||
`/opt/owrt-remote/owrt-remote-hub.py.bak-before-hub-url-v117` на VPS.
|
||||
`HEARTBEAT_OK` подтверждает ответ Hub. Если файлы установлены, но Hub недоступен,
|
||||
скрипт отдельно сообщает `HEARTBEAT_FAILED`; это не подтверждение связи.
|
||||
|
||||
@@ -125,21 +125,21 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-host-policy.
|
||||
|
||||
Установка загружает локальные файлы, без Git. До остановки службы проверяются
|
||||
доступность WAN и поддержка таблицы реальной командой ip на роутере.
|
||||
На роутер загружается и файл LuCI: версия v116 появляется и в основной
|
||||
На роутер загружается и файл LuCI: версия v117 появляется и в основной
|
||||
CGI-панели, и в странице входа в неё через LuCI.
|
||||
Сначала сохраняются файлы
|
||||
и конфигурация; после замены проверяется heartbeat/health. При ошибке
|
||||
автоматически восстанавливается состояние перед текущей попыткой.
|
||||
Сохранённая резервная копия для ручного отката не перезаписывается.
|
||||
|
||||
* Роутер: `/root/owrt-remote-backup-before-host-policy-v116` — агент,
|
||||
* Роутер: `/root/owrt-remote-backup-before-host-policy-v117` — агент,
|
||||
helper, init, CGI, UCI-конфигурация и Xray-конфиг (в закрытом каталоге).
|
||||
* VPS: `/opt/owrt-remote/owrt-remote-hub.py.bak-before-host-policy-v116`.
|
||||
* VPS: `/opt/owrt-remote/owrt-remote-hub.py.bak-before-host-policy-v117`.
|
||||
База роутеров и настройки авторизации не заменяются.
|
||||
|
||||
Успешная установка заканчивается `HOST_POLICY_ROUTER_V116_OK` на роутере
|
||||
и `HOST_POLICY_VPS_V116_Install_OK` на VPS. Старые резервные копии предыдущей
|
||||
версии сохраняются; новые команды отката используют отдельные копии для v116.
|
||||
Успешная установка заканчивается `HOST_POLICY_ROUTER_V117_OK` на роутере
|
||||
и `HOST_POLICY_VPS_V117_Install_OK` на VPS. Старые резервные копии предыдущей
|
||||
версии сохраняются; новые команды отката используют отдельные копии для v117.
|
||||
|
||||
### Переход с nftables-маркировки на адресную маршрутизацию
|
||||
|
||||
@@ -148,11 +148,11 @@ CGI-панели, и в странице входа в неё через LuCI.
|
||||
Обновление VPS для этого изменения не требуется.
|
||||
|
||||
```powershell
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-host-policy.ps1" -Target Router -Machine "root@192.168.2.1" -PreserveConnectionSettings -BackupName "podkop-compat-v116" -Action Install
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-host-policy.ps1" -Target Router -Machine "root@192.168.2.1" -BackupName "podkop-compat-v116" -Action Rollback
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-host-policy.ps1" -Target Router -Machine "root@192.168.2.1" -PreserveConnectionSettings -BackupName "podkop-compat-v117" -Action Install
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-host-policy.ps1" -Target Router -Machine "root@192.168.2.1" -BackupName "podkop-compat-v117" -Action Rollback
|
||||
```
|
||||
|
||||
Копия: `/root/owrt-remote-backup-before-podkop-compat-v116`.
|
||||
Копия: `/root/owrt-remote-backup-before-podkop-compat-v117`.
|
||||
После установки повторите диагностику Podkop. Предупреждение от прежних правил
|
||||
Remote должно исчезнуть. Настоящие дополнительные правила других приложений
|
||||
по-прежнему обнаруживаются штатной диагностикой.
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
# Persistent client journal (v117 local fix)
|
||||
|
||||
The helper is `/usr/lib/owrt-remote/logs.sh`. It changes only Remote's private
|
||||
journal, not logd, network configuration, policy routing or Xray configuration.
|
||||
|
||||
## Storage and limits
|
||||
|
||||
- Persistent: normally `/overlay/owrt-remote/logs`, only when `/overlay` is an
|
||||
actual writable JFFS2/UBIFS/ext4/f2fs/ext2/ext3/btrfs mount. A directly mounted
|
||||
persistent root uses `/etc/owrt-remote/logs`. Root OverlayFS and tmpfs are
|
||||
rejected. Existing ancestors/symlinks are resolved against `/proc/mounts`;
|
||||
the longest containing mount wins. An explicit `OWRT_REMOTE_LOG_DIR` is
|
||||
validated the same way.
|
||||
- Runtime: `/tmp/owrt-remote-log-runtime`, which must be on tmpfs. Contains a
|
||||
buffer <=16 KiB, <=64 hashed dedup entries, a heartbeat fingerprint, PID locks,
|
||||
one export <=16 KiB, one HTTP body about 22 KiB, one viewer snapshot <=16 KiB
|
||||
plus a bounded filename list. No whole-journal shell variable or RAM archive.
|
||||
HTTP response files are limited by a sender-only RLIMIT_FSIZE (32 KiB in
|
||||
BusyBox ash); this does not alter heartbeat or Xray process limits.
|
||||
- Flash data: <=2 MiB across <=256 immutable chunks, each <=16 KiB. This is an
|
||||
upper bound, not guaranteed capacity: many small chunks reach the file-count
|
||||
bound first. Metadata, filesystem allocation units and the one quarantined
|
||||
damaged chunk add small overhead beyond the raw-data bound.
|
||||
- Rotation runs each minute. Acknowledged chunks expire after 24 hours or
|
||||
earlier to admit new chunks. Unacknowledged chunks do not expire. If the
|
||||
bounded queue/RAM buffer fills, **new** events are skipped; existing
|
||||
unacknowledged chunks remain intact. A warning and dropped-event count make
|
||||
that loss visible. A bounded queue cannot preserve infinitely many events
|
||||
during an infinitely long outage.
|
||||
- Default batch flush: once per 60 seconds, at most 16 KiB of new data; explicit
|
||||
`flush force`, normal service stop and shutdown can flush sooner. There is no
|
||||
fsync per line. Idle polls do not rewrite persistent state. Metadata and
|
||||
filesystem write amplification are additional to the data rate; no universal
|
||||
Flash-lifetime guarantee is possible across devices/filesystems.
|
||||
- Less than 2 MiB free space or write/mount failure stops Flash writes and
|
||||
switches this boot to a warning-marked RAM fallback: <=64 KiB data in <=8
|
||||
chunks +16 KiB buffer. The old Flash queue is left intact. Restore storage
|
||||
and reboot to resume that queue. No automatic mid-boot merging of two journal
|
||||
identities, which could acknowledge the wrong records.
|
||||
|
||||
## Crash and synchronization rules
|
||||
|
||||
Each chunk filename includes journal identity, start/end byte offsets and a validated SHA-256/MD5/POSIX cksum digest. Data is written to `chunk.new` on the persistent filesystem and renamed;
|
||||
state is written to `state.new` on that filesystem and atomically replaced.
|
||||
Batch sync (`sync -f`, or BusyBox `sync` fallback) completes durability. A small
|
||||
RAM in-flight marker reconciles a killed flusher before another append. At
|
||||
boot, checksums reject partial/corrupt chunks, incomplete temporary files are
|
||||
removed, and a completed rename beyond the saved cursor is recovered. One
|
||||
corrupt chunk can be retained as `damaged.log` for inspection. A reboot never
|
||||
clears a valid journal.
|
||||
|
||||
Last-minute events still in RAM can be lost on sudden power removal. This is
|
||||
the explicit buffering tradeoff, not a claim of per-line durable logging.
|
||||
|
||||
Only a successful authenticated Hub reply advances the ack. The recent ack is
|
||||
checkpointed with the next periodic batch before pruning; if power is lost
|
||||
first, records may replay with the same journal/offset. Existing Hub ingestion
|
||||
suppresses these duplicates. Packets are <=16 KiB raw and transport them from
|
||||
files without an entire packet in a shell variable. One `logs-sync` sender is
|
||||
allowed; stale locks and stale payloads recover on the next attempt. Explicit
|
||||
local/Hub clear intentionally discards both confirmed and unconfirmed data.
|
||||
Log-sender curl temporaries use a private runtime subdirectory, reclaimed on
|
||||
the next attempt after SIGKILL; heartbeat's temporary directory is untouched.
|
||||
|
||||
The local installer replaces the helper, log service, agent, CGI and LuCI view. It
|
||||
does not restart the tunnel, heartbeat or network. With the previous collector
|
||||
stopped, it streams the old bounded RAM journal into Flash chunks, preserving
|
||||
its journal ID, total cursor, ack and clear token. Already delivered bytes are
|
||||
not resent; undelivered records remain eligible for delivery. No FIFO, lock or
|
||||
temporary request is migrated. A same-filesystem staging manifest allows a
|
||||
power-interrupted migration to finish even after the old /tmp has disappeared.
|
||||
Migration needs 6 MiB free Flash; if it cannot safely migrate legacy data, the
|
||||
local installer fails and restores code instead of silently deleting history.
|
||||
With no usable Flash, only a bounded legacy tail is retained in RAM with an
|
||||
explicit loss warning, and the former large RAM files are removed. An ordinary
|
||||
upgrade stops the previous collector before replacing files and invokes this
|
||||
migration on log-service startup. Rollback restores code/service state and
|
||||
leaves Flash logs intact for recovery; the old helper starts a fresh RAM log.
|
||||
|
||||
## Automated checks
|
||||
|
||||
```powershell
|
||||
$env:OWRT_LOG_TEST_SHELL = "$PWD\.test-output\busybox\busybox64.exe"
|
||||
python -m unittest discover -s tests -p test_persistent_logs.py -v
|
||||
Remove-Item Env:OWRT_LOG_TEST_SHELL
|
||||
python -m unittest discover -s tests -p test_global_logs.py -v
|
||||
python -m unittest discover -s tests -p test_router_log_transport.py -v
|
||||
```
|
||||
|
||||
Tests use real shell functions and disposable mount/filesystem fixtures. The
|
||||
BusyBox Windows build exercises ash and its applets; it is **not** a claim of
|
||||
testing OpenWrt 24.10.3's exact BusyBox/kernel/UBIFS or physical power loss.
|
||||
|
||||
## Device checks before release
|
||||
|
||||
Install locally, then inspect from Windows PowerShell:
|
||||
|
||||
```powershell
|
||||
ssh root@192.168.2.1 'sh /usr/lib/owrt-remote/logs.sh status; df -h /overlay /tmp; free; ubus call service list; owrt-remote heartbeat-diagnostics'
|
||||
```
|
||||
|
||||
Expect `persistent=1`, a persistent storage path and tmpfs runtime. Confirm the
|
||||
mount explicitly using `mount` and verify `state` and `part.*` appear on Flash.
|
||||
Append a marker and flush before a user-planned reboot:
|
||||
|
||||
```powershell
|
||||
ssh root@192.168.2.1 'printf "persistent-check\n" | sh /usr/lib/owrt-remote/logs.sh append; sh /usr/lib/owrt-remote/logs.sh flush force; sh /usr/lib/owrt-remote/logs.sh status'
|
||||
```
|
||||
|
||||
After reboot, `read` must still show the marker and the same journal ID. Test an
|
||||
outage only on a controlled router with a separate management path; do not
|
||||
change firewall/WAN settings on a remote production router merely for this
|
||||
test. During outage, watch data/chunk/runtime bounds; after reconnect, watch
|
||||
`ack_bytes` advance without duplicate Hub entries. Launch concurrent
|
||||
`owrt-remote logs-sync` calls and inspect service/process counts. Confirm UI,
|
||||
heartbeat, Xray and WAN direct remain functional. Actual sustained RAM,
|
||||
Flash-write and reboot observations must be collected on the device.
|
||||
|
||||
## VPS archive and malformed-fragment recovery
|
||||
|
||||
The Hub retains its existing persistent SQLite archive, normally at
|
||||
`/var/lib/owrt-remote/hub.db`, <=2 MiB of content per router. No new RAM archive
|
||||
or whole-journal queue is introduced on the VPS. The live view still transfers
|
||||
only the last 64 KiB. A storage diagnostic resolves DB symlinks and the longest
|
||||
containing Linux mount, marking tmpfs/ramfs as nonpersistent. The dedicated
|
||||
`deploy-persistent-logs-vps.ps1` verifies the running Hub's actual DB path and
|
||||
writable disk before installing code. It preserves the database and all settings;
|
||||
rollback replaces code only. It refuses installation over an unconfirmed/RAM DB
|
||||
rather than automatically relocating a live database.
|
||||
|
||||
Some OpenWrt builds lack the optional `cksum` applet. The previous local helper
|
||||
failed to check its exit status and could publish names ending in an empty digest.
|
||||
Export skipped these files, but the empty status packet claimed the full cursor;
|
||||
the previous Hub accepted it and falsely acknowledged missing data. Regression
|
||||
checks now simulate missing cksum and failed checksum tools. New chunks use an
|
||||
available SHA-256/MD5/POSIX checksum with validated output. An empty status packet
|
||||
cannot advance the delivered cursor on either side.
|
||||
|
||||
On upgrade, a versioned recovery marker forces inspection even during the same
|
||||
boot. Retained malformed fragments of the expected byte length are copied into a
|
||||
same-filesystem recovery generation with real digests and a committed migration
|
||||
manifest. Old falsely acknowledged offsets cannot suppress that generation.
|
||||
Recovery requires enough free disk for the bounded copy and retains originals
|
||||
until the new manifest is committed. This recovers retained bytes, not already
|
||||
removed files, and cannot retroactively verify their former integrity. A warning
|
||||
records that limitation. A new generation can repeat already delivered history
|
||||
if mixed with affected fragments; normal retries remain deduplicated by offsets.
|
||||
|
||||
Run the real client spool-to-Hub ingestion regression as well as storage/deploy
|
||||
checks:
|
||||
|
||||
```powershell
|
||||
python -m unittest discover -s tests -p test_router_log_archive.py -v
|
||||
python -m unittest discover -s tests -p test_persistent_logs_vps_deploy.py -v
|
||||
```
|
||||
@@ -1,4 +1,4 @@
|
||||
# reCAPTCHA v3 в OpenWrt Remote Hub v116
|
||||
# reCAPTCHA v3 в OpenWrt Remote Hub v117
|
||||
|
||||
В меню владельца, в блоке «Капча на экране входа», доступны цифры,
|
||||
Google reCAPTCHA v2 Checkbox и Google reCAPTCHA v3. Установка обновления
|
||||
@@ -74,8 +74,8 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-recaptcha-v3
|
||||
непосредственно перед попыткой. Резервные копии при повторном запуске
|
||||
не перезаписываются:
|
||||
|
||||
* `/opt/owrt-remote/owrt-remote-hub.py.bak-before-recaptcha-v3-v116`
|
||||
* `/opt/owrt-remote/captcha-before-recaptcha-v3-v116.json` (режим 0600).
|
||||
* `/opt/owrt-remote/owrt-remote-hub.py.bak-before-recaptcha-v3-v117`
|
||||
* `/opt/owrt-remote/captcha-before-recaptcha-v3-v117.json` (режим 0600).
|
||||
|
||||
Скрипт рассчитан на стандартные пути установленного Hub:
|
||||
`/opt/owrt-remote/owrt-remote-hub.py`, `/var/lib/owrt-remote/hub-auth.json`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Аудит ресурсов OpenWrt Remote Hub v116
|
||||
# Аудит ресурсов OpenWrt Remote Hub v117
|
||||
|
||||
Исправления внесены поверх `7f809fd`. Версия панели остаётся v116.
|
||||
Исправления внесены поверх `7f809fd`. Версия панели остаётся v117.
|
||||
|
||||
## SQLite
|
||||
|
||||
@@ -115,7 +115,7 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "vps\deploy-resource-fix
|
||||
|
||||
Установщик загружает локальный файл без Git, запускает изолированные тесты,
|
||||
сохраняет исходный серверный файл:
|
||||
`/opt/owrt-remote/owrt-remote-hub.py.bak-before-resource-fix-v116`,
|
||||
`/opt/owrt-remote/owrt-remote-hub.py.bak-before-resource-fix-v117`,
|
||||
атомарно заменяет Python файл, перезапускает `owrt-remote` и проверяет health.
|
||||
Существующая резервная копия не перезаписывается. При ошибке установки
|
||||
восстанавливается файл, работавший непосредственно перед текущей попыткой.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Виджеты ресурсов VPS (v116)
|
||||
# Виджеты ресурсов VPS (v117)
|
||||
|
||||
Три карточки ЦП, ОЗУ и ДИСК находятся над кнопками «Рестарт Xray VPS»,
|
||||
профилем и выходом. На ПК ширина и высота совпадают с кнопками под ними.
|
||||
@@ -23,7 +23,7 @@ vCPU и объёмы занятой/общей памяти и диска.
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\Documents\openwrt overlay\vps\deploy-vps-widgets.ps1" -Vps "root@193.233.82.38" -Action Install
|
||||
```
|
||||
|
||||
После `VPS_WIDGETS_V116_Install_OK` обновите страницу через Ctrl+F5.
|
||||
После `VPS_WIDGETS_V117_Install_OK` обновите страницу через Ctrl+F5.
|
||||
Скрипт загружает локальный файл, проверяет его до замены, перезапускает
|
||||
службу и проверяет health. При ошибке после замены восстанавливает файл
|
||||
перед текущей попыткой. База данных и настройки не заменяются.
|
||||
@@ -33,8 +33,8 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\
|
||||
```
|
||||
|
||||
Откат использует неизменяемую копию первого состояния до установки:
|
||||
`/opt/owrt-remote/owrt-remote-hub.py.bak-before-vps-widgets-v116`.
|
||||
Успешное завершение: `VPS_WIDGETS_V116_Rollback_OK`.
|
||||
`/opt/owrt-remote/owrt-remote-hub.py.bak-before-vps-widgets-v117`.
|
||||
Успешное завершение: `VPS_WIDGETS_V117_Rollback_OK`.
|
||||
|
||||
## Локальные проверки
|
||||
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
# WAN IPv4 DNS fallback (local v116 fix)
|
||||
# WAN IPv4 DNS fallback (local v117 fix)
|
||||
|
||||
`files/usr/lib/owrt-remote/wan.sh` now explicitly queries
|
||||
`nslookup -type=A -timeout=3 -retry=1 "$host" "$dns"`.
|
||||
@@ -58,7 +58,7 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\
|
||||
```
|
||||
|
||||
The installer saves the first backup at
|
||||
`/root/owrt-remote-backup-before-wan-dns-v116/wan.sh`, checks shell syntax and
|
||||
`/root/owrt-remote-backup-before-wan-dns-v117/wan.sh`, checks shell syntax and
|
||||
reloads only the supervised heartbeat instance so it reads the new helper.
|
||||
It does not restart Xray or regenerate configuration/routing. Installation
|
||||
verification/reload failures restore the file from that attempt.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Несколько WAN интерфейсов и резервирование (v116)
|
||||
# Несколько WAN интерфейсов и резервирование (v117)
|
||||
|
||||
Кнопка **Выбрать интерфейсы** раскрывает список с галочками. Можно отметить
|
||||
один, два, три и больше интерфейсов. Список содержит имена
|
||||
@@ -46,7 +46,7 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\
|
||||
|
||||
Скрипт устанавливает CGI, LuCI, агент, WAN helper и hotplug вместе. До замены
|
||||
сохраняются прежние файлы, UCI-настройки и Xray config в
|
||||
`/root/owrt-remote-backup-before-wan-multi-v116`.
|
||||
`/root/owrt-remote-backup-before-wan-multi-v117`.
|
||||
Успех подтверждает `WAN_MULTI_ROUTER_Install_OK`. После установки обновить
|
||||
панель роутера через Ctrl+F5, отметить интерфейсы и сохранить выбор.
|
||||
VPS обновлять не требуется. Старый `deploy-wan-interface-picker.ps1`
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# Восстановление Xray после пропажи выбранного WAN (v117)
|
||||
|
||||
Сообщения `No selected WAN has a physical IPv4 default route`, ошибки
|
||||
генерации конфигурации и `crash loop 6 crashes` описывают последовательность:
|
||||
временно исчез маршрут на выбранном uplink, каждый запуск агента завершался,
|
||||
а procd прекратил повторные попытки запуска Xray. Затем Heartbeat смог
|
||||
восстановиться самостоятельно. Он делает HTTP-запрос, поэтому его успех
|
||||
не подтверждает работу двух обратных соединений Xray для Админки и SSH.
|
||||
|
||||
В прежнем `wan_reconcile_xray` остановленный Xray игнорировался; при возврате
|
||||
того же интерфейса также ничего не происходило. Это подтверждённый дефект
|
||||
восстановления. Причину исчезновения исходного маршрута и фактическое
|
||||
текущее состояние Xray на удалённом устройстве только эти логи не определяют.
|
||||
|
||||
## Исправление
|
||||
|
||||
- Перед генерацией JSON служебный процесс ожидает выбранный WAN с физическим
|
||||
IPv4 default route. Проверка каждые 5 секунд, одна запись о начале ожидания
|
||||
и одна о возвращении WAN. Конфигурация в ожидании не заменяется.
|
||||
- Ошибки UUID, JSON или проверки Xray при доступном WAN не маскируются
|
||||
ожиданием сети: генерация по-прежнему завершается с настоящей ошибкой.
|
||||
- Heartbeat не перезапускает всю службу, пока все выбранные WAN недоступны.
|
||||
- После подготовки WAN проверяются процесс Xray и состояние procd. Если
|
||||
Xray отсутствует и его instance не работает, но служба включена и Heartbeat
|
||||
работает, повторно регистрируется только instance `xray` через `service add`.
|
||||
Работающий процесс запуска, который ожидает WAN или готовит Xray, не трогается.
|
||||
- Живой Heartbeat и остальные instance не удаляются. При прежнем здоровом
|
||||
туннеле на том же интерфейсе перезапуск не запрашивается.
|
||||
- Блокировка в RAM исключает одновременные запросы восстановления; PID
|
||||
завершившегося worker освобождается. Повторные запросы ограничены интервалом
|
||||
300 секунд, в том числе при ошибке ubus. Успешный ответ ubus означает принятие
|
||||
запроса запуска, а не доказанную доступность Админки.
|
||||
- Явная остановка через панель (`enabled=0`) или остановка всей службы
|
||||
не отменяется автоматикой. Невыбранные интерфейсы не используются.
|
||||
- Источники установки, настройки LAN, DNS, Podkop и логика WAN policy routing
|
||||
не меняются этим исправлением.
|
||||
|
||||
Формат `service add` и строковых параметров respawn сверялся с исходниками:
|
||||
|
||||
- https://github.com/openwrt/procd/blob/master/service/service.c
|
||||
- https://github.com/openwrt/procd/blob/master/service/instance.c
|
||||
- https://github.com/openwrt/openwrt/blob/openwrt-24.10/package/system/procd/files/procd.sh
|
||||
|
||||
## Установка и откат
|
||||
|
||||
Windows PowerShell 5.1, роутер пользователя:
|
||||
|
||||
```powershell
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\Documents\openwrt overlay\vps\deploy-wan-recovery-fix.ps1" -Router "root@192.168.2.1" -Action Install
|
||||
```
|
||||
|
||||
Скрипт обновляет агент и WAN helper парой, сохраняет первую копию в
|
||||
`/root/owrt-remote-backup-before-wan-recovery-v117`, проверяет shell-синтаксис
|
||||
и побайтовое совпадение. Затем перезагружает только работающий Heartbeat,
|
||||
чтобы в RAM не остался прежний shell с уже загруженными функциями.
|
||||
Туннель при исправном Xray не перезапускается. Остановленный Heartbeat
|
||||
не запускается принудительно. Ошибка установки восстанавливает файлы
|
||||
до текущей попытки. UCI и рабочий JSON скрипт установки не заменяет;
|
||||
обычный запуск Xray выполняет штатную проверку и генерацию конфигурации.
|
||||
|
||||
Установка подтверждается `WAN_RECOVERY_ROUTER_Install_OK`, это подтверждение
|
||||
установки кода. Проверка состояния:
|
||||
|
||||
```powershell
|
||||
ssh root@192.168.2.1 'owrt-remote status; owrt-remote heartbeat-diagnostics; logread -e owrt-remote | tail -n 60'
|
||||
```
|
||||
|
||||
Откат к коду до первого применения:
|
||||
|
||||
```powershell
|
||||
powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\Documents\openwrt overlay\vps\deploy-wan-recovery-fix.ps1" -Router "root@192.168.2.1" -Action Rollback
|
||||
```
|
||||
|
||||
Для роутера друга нужен его собственный доступный SSH-адрес, VPN или человек
|
||||
в локальной сети. Через оборванные обратные SSH/HTTP-туннели установить код
|
||||
невозможно. Если WAN уже вернулся, для разового восстановления на самом
|
||||
роутере можно выполнить `/etc/init.d/owrt-remote restart`, затем проверить
|
||||
`owrt-remote status`. Сам по себе этот перезапуск не устанавливает исправление.
|
||||
|
||||
## Локальная проверка
|
||||
|
||||
`test_wan_recovery.py` моделирует длительное отсутствие маршрута, возврат
|
||||
того же интерфейса, остановленный Xray при работающем Heartbeat, начальную
|
||||
подготовку, исправный туннель, ручную остановку, параллельные запросы,
|
||||
оставшуюся блокировку, отказ ubus и настоящую ошибку генерации JSON.
|
||||
Поддерживает `OWRT_RECOVERY_TEST_SHELL` для BusyBox ash.
|
||||
|
||||
`test_wan_recovery_deploy.py` проверяет установку, повторную установку,
|
||||
сохранение первой резервной копии, откат, отсутствие копии, повреждённый
|
||||
payload, отказ проверки и отказ перезапуска Heartbeat без реального SSH.
|
||||
|
||||
Дополнительно запускаются регрессии WAN failover, host policy, HTTP/HTTPS
|
||||
авторизации и перенаправлений. Проверка BusyBox ash выполняется локальной
|
||||
Windows-сборкой BusyBox; OpenWrt 24.10.3 и настоящий procd на устройстве
|
||||
требуют проверки пользователем. Прямой WAN на удалённом устройстве не
|
||||
отключать для искусственного теста без независимого канала доступа.
|
||||
@@ -1,4 +1,4 @@
|
||||
# Исправление запуска прямого WAN в v116
|
||||
# Исправление запуска прямого WAN в v117
|
||||
|
||||
На проблемном роутере обычный `wan` был поднят, имел IPv4-адрес,
|
||||
маршрут и рабочий DNS. Проверка `ip -4 route show table 210` возвращала
|
||||
@@ -16,7 +16,7 @@ instance после шести падений. Из-за остановленн
|
||||
конфигурации останавливают агент до запуска Xray.
|
||||
|
||||
Изменены только `files/usr/lib/owrt-remote/wan.sh` и
|
||||
`files/usr/sbin/owrt-remote`. Версия остаётся v116. Настройки Podkop,
|
||||
`files/usr/sbin/owrt-remote`. Версия остаётся v117. Настройки Podkop,
|
||||
LAN и источники установки GitHub/GitLab не меняются. Обновление VPS
|
||||
для этого исправления не требуется.
|
||||
|
||||
@@ -28,7 +28,7 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File "C:\Users\k.zolotarev95\
|
||||
|
||||
Скрипт перед заменой проверяет синтаксис и подготовку WAN новым helper,
|
||||
сохраняет агент, helper, UCI и Xray-конфигурацию в
|
||||
`/root/owrt-remote-backup-before-wan-stability-v116`. Сохранённая копия
|
||||
`/root/owrt-remote-backup-before-wan-stability-v117`. Сохранённая копия
|
||||
не перезаписывается при повторной установке. После замены выполняются
|
||||
перезапуск, проверка процесса Xray и настоящий heartbeat до Hub.
|
||||
При ошибке восстанавливается состояние до текущей попытки.
|
||||
|
||||
+6
-6
@@ -1,4 +1,4 @@
|
||||
# Применение OpenWrt config через панель роутера (v116)
|
||||
# Применение OpenWrt config через панель роутера (v117)
|
||||
|
||||
При отправке textarea браузер кодирует переносы строк как CRLF. CGI
|
||||
декодировал их без нормализации и записывал CR в исполняемый shell-скрипт.
|
||||
@@ -22,7 +22,7 @@ CRLF и LF, heredoc, пустые строки, кодированные сим
|
||||
|
||||
Скрипт обновляет только CGI на роутере. VPS обновлять для этого фикса
|
||||
не требуется. До замены сохраняется первая резервная копия CGI:
|
||||
`/root/owrt-remote-backup-before-xray-apply-v116/owrt-remote`.
|
||||
`/root/owrt-remote-backup-before-xray-apply-v117/owrt-remote`.
|
||||
|
||||
Из PowerShell Windows 11; замените IP, если адрес роутера отличается:
|
||||
|
||||
@@ -53,7 +53,7 @@ python tests/test_xray_apply.py -v
|
||||
|
||||
Переносы строк в формах описаны в [стандарте HTML](https://html.spec.whatwg.org/multipage/form-control-infrastructure.html).
|
||||
|
||||
## Мобильная вставка команд (локальный фикс v116)
|
||||
## Мобильная вставка команд (локальный фикс v117)
|
||||
|
||||
Воспроизведён отдельный случай `rc=255`: в строке
|
||||
`uci set owrtremote.main.router_name=‘Дом у моря’` типографские кавычки
|
||||
@@ -77,7 +77,7 @@ python tests/test_xray_apply.py -v
|
||||
|
||||
Обновляется только `/www/cgi-bin/owrt-remote` на роутере, без перезапуска
|
||||
служб и изменений настроек при установке. Резервная копия отдельная:
|
||||
`/root/owrt-remote-backup-before-xray-mobile-v116/owrt-remote`.
|
||||
`/root/owrt-remote-backup-before-xray-mobile-v117/owrt-remote`.
|
||||
|
||||
Установка из PowerShell Windows 11:
|
||||
|
||||
@@ -101,7 +101,7 @@ python tests/test_xray_mobile_deploy.py -v
|
||||
```
|
||||
|
||||
|
||||
## Проверка Xray и ошибка Heartbeat после применения (локальный фикс v116)
|
||||
## Проверка Xray и ошибка Heartbeat после применения (локальный фикс v117)
|
||||
|
||||
В предоставленном журнале Xray 26.3.27 запускается успешно, а завершающая
|
||||
команда `owrt-remote heartbeat` возвращает `rc=1` с `Failed to allocate
|
||||
@@ -153,7 +153,7 @@ CGI получает этап выполнения от агента. Ошибк
|
||||
Новый скрипт `vps/deploy-xray-validation-fix.ps1` обновляет CGI и агент,
|
||||
не пересобирает настройки и не перезапускает службы при установке/откате.
|
||||
Первая резервная копия обоих файлов:
|
||||
`/root/owrt-remote-backup-before-xray-validation-v116`. Если установка
|
||||
`/root/owrt-remote-backup-before-xray-validation-v117`. Если установка
|
||||
оборвалась или проверка файлов не прошла, возвращаются файлы до этой попытки.
|
||||
На роутере друга нужно указать его доступный SSH-адрес вместо локального IP.
|
||||
|
||||
|
||||
+12
-90
@@ -1,4 +1,4 @@
|
||||
"""Exercise the real RAM journal and authorized router CGI with temporary files."""
|
||||
"""Exercise the bounded persistent journal and authorized router CGI with temporary files."""
|
||||
import concurrent.futures
|
||||
import base64
|
||||
import json
|
||||
@@ -22,12 +22,14 @@ class GlobalLogTests(unittest.TestCase):
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.log_dir = self.root / 'journal'
|
||||
self.run_dir = self.root / 'runtime'
|
||||
self.helper = self.root / 'logs.sh'
|
||||
source = (ROOT / 'files/usr/lib/owrt-remote/logs.sh').read_text(encoding='utf-8')
|
||||
source = source.replace('log_clock() { cut -d. -f1 /proc/uptime; }',
|
||||
'log_clock() { echo "${TEST_UPTIME:-1000}"; }')
|
||||
self.helper_source = source
|
||||
source = source.replace('log_ram_storage() {', 'log_ram_storage() { return 0;', 1)
|
||||
source = source.replace('log_flash_storage() {', 'log_flash_storage() { return 0;', 1)
|
||||
self.helper.write_text(source, encoding='utf-8', newline='\n')
|
||||
self.cgi = self.root / 'cgi'
|
||||
self.cgi.write_text((ROOT / 'files/www/cgi-bin/owrt-remote').read_text(encoding='utf-8'),
|
||||
@@ -35,12 +37,15 @@ class GlobalLogTests(unittest.TestCase):
|
||||
self.key = self.root / 'key'
|
||||
self.key.write_text('test-key', encoding='utf-8')
|
||||
self.env = {**os.environ, 'OWRT_REMOTE_LOG_DIR': self.log_dir.as_posix(),
|
||||
'OWRT_REMOTE_LOG_RUNTIME': self.run_dir.as_posix(),
|
||||
'OWRT_REMOTE_LOG_HELPER': self.helper.as_posix(), 'TMPDIR': self.root.as_posix(),
|
||||
'KEY_FILE': self.key.as_posix(), 'TEST_UPTIME': '1000'}
|
||||
if os.name == 'nt':
|
||||
self.env['PATH'] = str(GIT_BIN) + os.pathsep + self.env['PATH']
|
||||
|
||||
def helper_call(self, command, text='', *args, env=None):
|
||||
if command == 'export':
|
||||
self.helper_call('flush', '', 'force', env=env)
|
||||
result = subprocess.run([SH, self.helper.as_posix(), command, *args], input=text,
|
||||
encoding='utf-8', capture_output=True, env=env or self.env, timeout=20)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
@@ -61,8 +66,9 @@ class GlobalLogTests(unittest.TestCase):
|
||||
self.assertIn('Xray connected', downloaded)
|
||||
self.assertIn('Cache-Control: no-store', downloaded)
|
||||
# The bounded read is independent of append filtering; supply an existing large journal.
|
||||
with (self.log_dir / 'current.log').open('a', encoding='utf-8') as journal:
|
||||
journal.write(''.join(f'{i}: ' + 'y' * 1024 + '\n' for i in range(70)))
|
||||
state = (self.log_dir / 'state').read_text().split()
|
||||
for i in range(5):
|
||||
(self.log_dir / f'part.{state[0]}.{i:018d}.{i+1:018d}.1').write_text('y' * 15000 + '\n')
|
||||
live = self.cgi_call('logs_live').split('\n\n', 1)[1]
|
||||
# Git for Windows can translate CRLF inside native pipes.
|
||||
self.assertLessEqual(len(live.encode()), 65536)
|
||||
@@ -141,32 +147,6 @@ class GlobalLogTests(unittest.TestCase):
|
||||
lines = self.helper_call('read').splitlines()
|
||||
self.assertEqual([line.split(']', 1)[0] for line in lines], ['[ERROR', '[ERROR', '[ERROR', '[WARN'])
|
||||
|
||||
def test_persistent_storage_is_rejected_before_any_file_is_written(self):
|
||||
mounts = self.root / 'mounts'
|
||||
mounts.write_text('/dev/root / overlay rw 0 0\n', encoding='utf-8')
|
||||
source = self.helper_source.replace('/proc/mounts', mounts.as_posix())
|
||||
self.helper.write_text(source, encoding='utf-8', newline='\n')
|
||||
env = self.env
|
||||
result = subprocess.run([SH, self.helper.as_posix(), 'append'], input='must not be written\n',
|
||||
encoding='utf-8', capture_output=True, env=env, timeout=20)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('refusing persistent storage', result.stderr)
|
||||
self.assertFalse(self.log_dir.exists())
|
||||
self.env = env
|
||||
self.assertIn('503 Service Unavailable', self.cgi_call('logs_live'))
|
||||
self.assertFalse(self.log_dir.exists())
|
||||
# The same location is permitted only when its containing mount is tmpfs.
|
||||
mounts.write_text('/dev/root / overlay rw 0 0\ntmpfs ' + self.root.as_posix() + ' tmpfs rw 0 0\n', encoding='utf-8')
|
||||
self.helper_call('append', 'RAM fact\n')
|
||||
self.assertEqual(self.helper_call('read'), '[OK] RAM fact\n')
|
||||
mounts.write_text('/dev/root / overlay rw 0 0\ntmpfs ' + self.root.as_posix()
|
||||
+ ' tmpfs rw 0 0\n/dev/flash ' + self.log_dir.as_posix()
|
||||
+ ' overlay rw 0 0\n', encoding='utf-8')
|
||||
result = subprocess.run([SH, self.helper.as_posix(), 'append'], input='must not enter nested Flash mount\n',
|
||||
encoding='utf-8', capture_output=True, env=env, timeout=20)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertEqual((self.log_dir / 'current.log').read_text(), '[OK] RAM fact\n')
|
||||
|
||||
def test_clear_requires_post_and_correct_key(self):
|
||||
self.helper_call('append', 'retain this event\n')
|
||||
self.assertIn('Доступ запрещен', self.cgi_call('logs_download', key='wrong'))
|
||||
@@ -178,13 +158,6 @@ class GlobalLogTests(unittest.TestCase):
|
||||
self.helper_call('append', 'new event\n')
|
||||
self.assertEqual(self.helper_call('read'), '[OK] new event\n')
|
||||
|
||||
def test_retention_uses_uptime_and_clears_on_reboot(self):
|
||||
self.helper_call('append', 'old event\n')
|
||||
self.assertIn('old event', self.helper_call('read', env={**self.env, 'TEST_UPTIME': '87399'}))
|
||||
self.assertEqual(self.helper_call('read', env={**self.env, 'TEST_UPTIME': '87400'}), '')
|
||||
self.helper_call('append', 'after daily clear\n', env={**self.env, 'TEST_UPTIME': '87401'})
|
||||
self.assertEqual(self.helper_call('read', env={**self.env, 'TEST_UPTIME': '20'}), '')
|
||||
|
||||
def test_export_acknowledges_only_received_bytes_and_remote_clear_is_idempotent(self):
|
||||
self.helper_call('append', 'First event\n')
|
||||
first = json.loads(self.helper_call('export'))
|
||||
@@ -207,39 +180,6 @@ class GlobalLogTests(unittest.TestCase):
|
||||
self.helper_call('ack', '', second['journal_id'], str(second['to']), 'clear-123')
|
||||
self.assertIn('After clear', self.helper_call('read'), 'retrying clear must not clear new events')
|
||||
|
||||
def test_export_catches_up_in_bounded_batches_after_rotation(self):
|
||||
self.helper_call('clear')
|
||||
raw = b'[OK] ' + b'x' * 100000 + b'\n'
|
||||
(self.log_dir / 'current.log').write_bytes(raw)
|
||||
first = json.loads(self.helper_call('export'))
|
||||
first_raw = base64.b64decode(first['data'])
|
||||
self.assertLessEqual(len(first_raw), 65536)
|
||||
self.helper_call('ack', '', first['journal_id'], str(first['to']))
|
||||
second = json.loads(self.helper_call('export'))
|
||||
self.assertEqual(first_raw + base64.b64decode(second['data']), raw)
|
||||
# Bytes dropped by local rotation are detected by the server from the gap.
|
||||
(self.log_dir / 'current.log').write_bytes(b'[OK] Latest\n')
|
||||
(self.log_dir / 'total').write_text('200000')
|
||||
late = json.loads(self.helper_call('export'))
|
||||
self.assertEqual(late['from'], 200000 - len(b'[OK] Latest\n'))
|
||||
self.assertEqual(base64.b64decode(late['data']), b'[OK] Latest\n')
|
||||
|
||||
def test_rotation_retains_recent_log_and_clear_removes_both(self):
|
||||
self.helper_call('clear')
|
||||
current = self.log_dir / 'current.log'
|
||||
current.write_text('A' * 1048576, encoding='utf-8')
|
||||
self.helper_call('append', 'second file\n')
|
||||
self.assertTrue((self.log_dir / 'previous.log').exists())
|
||||
self.assertEqual(current.read_text(), '[OK] second file\n')
|
||||
current.write_text('B' * 1048576, encoding='utf-8')
|
||||
self.helper_call('append', 'third file\n')
|
||||
output = self.helper_call('read')
|
||||
self.assertNotIn('A', output)
|
||||
self.assertTrue(output.endswith('third file\n'))
|
||||
self.helper_call('clear')
|
||||
self.assertFalse((self.log_dir / 'previous.log').exists())
|
||||
self.assertEqual(current.read_text(), '')
|
||||
|
||||
def test_secrets_are_redacted_before_storage_and_html_is_literal(self):
|
||||
text = ('Heartbeat OK\nuci set owrtremote.main.hub_token=SECRET_TOKEN\n'
|
||||
'Authorization: Bearer SECRET_AUTH\nhttps://hub.test/?key=SECRET_KEY\n'
|
||||
@@ -256,11 +196,11 @@ class GlobalLogTests(unittest.TestCase):
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool:
|
||||
list(pool.map(lambda i: self.helper_call('append', f'event-{i}\n'), range(12)))
|
||||
self.assertEqual(set(self.helper_call('read').splitlines()), {f'[OK] event-{i}' for i in range(12)})
|
||||
self.assertFalse((self.log_dir / 'lock').exists())
|
||||
self.assertFalse((self.run_dir / 'lock').exists())
|
||||
|
||||
def test_stale_lock_recovers(self):
|
||||
self.helper_call('clear')
|
||||
lock = self.log_dir / 'lock'
|
||||
lock = self.run_dir / 'lock'
|
||||
lock.mkdir()
|
||||
(lock / 'pid').write_text('99999999')
|
||||
self.helper_call('append', 'recovered\n')
|
||||
@@ -272,28 +212,10 @@ class GlobalLogTests(unittest.TestCase):
|
||||
|
||||
def test_empty_lock_left_by_killed_writer_recovers(self):
|
||||
self.helper_call('clear')
|
||||
(self.log_dir / 'lock').mkdir()
|
||||
(self.run_dir / 'lock').mkdir()
|
||||
self.helper_call('append', 'after empty lock\n')
|
||||
self.assertEqual(self.helper_call('read'), '[OK] after empty lock\n')
|
||||
|
||||
def test_collector_failure_cleans_both_children_and_fifo(self):
|
||||
source = self.helper.read_text(encoding='utf-8')
|
||||
mocks = '''
|
||||
log_init() { return 0; }
|
||||
mkfifo() { return 0; }
|
||||
logread() { return 0; }
|
||||
log_lock() { return 1; }
|
||||
kill() { printf 'KILL=%s\\n' "$*"; }
|
||||
wait() { return 0; }
|
||||
rm() { printf 'REMOVE=%s\\n' "$*"; }
|
||||
'''
|
||||
self.helper.write_text(source.replace('case "${1:-}" in', mocks + '\ncase "${1:-}" in', 1),
|
||||
encoding='utf-8', newline='\n')
|
||||
result = subprocess.run([SH, self.helper.as_posix(), 'collect'], capture_output=True,
|
||||
encoding='utf-8', env=self.env, timeout=10)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertEqual(result.stdout.count('KILL='), 2)
|
||||
self.assertIn('REMOVE=-f ' + self.log_dir.as_posix() + '/stream.', result.stdout)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
@@ -43,7 +43,7 @@ class GlobalLogsDeploymentTests(unittest.TestCase):
|
||||
mappings['/etc/init.d/rpcd'] = (self.root / 'etc/init.d/rpcd').as_posix()
|
||||
mappings['/etc/init.d/uhttpd'] = (self.root / 'etc/init.d/uhttpd').as_posix()
|
||||
mappings['/tmp/owrt-remote-logs'] = (self.root / 'journal').as_posix()
|
||||
mappings['/root/owrt-remote-backup-before-global-logs-stop-v116'] = self.backup.as_posix()
|
||||
mappings['/root/owrt-remote-backup-before-global-logs-stop-v117'] = self.backup.as_posix()
|
||||
pattern = '|'.join(re.escape(target) for target in sorted(mappings, key=len, reverse=True))
|
||||
self.script = re.sub(pattern, lambda match: mappings[match.group()], self.script)
|
||||
# Existing sandbox service scripts only print markers. Mock readiness.
|
||||
@@ -55,7 +55,7 @@ jsonfilter() { return 0; }
|
||||
logger() { printf '%s\\n' "$3" | sh "$TEST_LOG_HELPER" append; }
|
||||
'''
|
||||
self.script = self.script.replace('for item in cgi agent init;', mocks + '\nfor item in cgi agent init;', 1)
|
||||
self.env = {**os.environ, 'OWRT_REMOTE_LOG_DIR': (self.root / 'journal').as_posix()}
|
||||
self.env = {**os.environ, 'OWRT_REMOTE_LOG_DIR': (self.root / 'journal').as_posix(), 'OWRT_REMOTE_LOG_RUNTIME': (self.root / 'runtime').as_posix()}
|
||||
self.env['TEST_LOG_HELPER'] = (self.root / 'usr/lib/owrt-remote/logs.sh').as_posix()
|
||||
if os.name == 'nt':
|
||||
self.env['PATH'] = str(GIT_BIN) + os.pathsep + self.env['PATH']
|
||||
@@ -75,6 +75,7 @@ logger() { printf '%s\\n' "$3" | sh "$TEST_LOG_HELPER" append; }
|
||||
if item == 'logs.sh':
|
||||
text = text.replace('log_clock() { cut -d. -f1 /proc/uptime; }', 'log_clock() { echo 1000; }')
|
||||
text = text.replace('log_ram_storage() {', 'log_ram_storage() { return 0;', 1)
|
||||
text = text.replace('log_flash_storage() {', 'log_flash_storage() { return 0;', 1)
|
||||
if item == 'logs-init':
|
||||
text = '#!/bin/sh\necho logs-service-$1\n'
|
||||
if bad and item == 'cgi':
|
||||
@@ -103,7 +104,7 @@ logger() { printf '%s\\n' "$3" | sh "$TEST_LOG_HELPER" append; }
|
||||
self.assertIn('GLOBAL_LOGS_ROUTER_Install_OK', result.stdout)
|
||||
for item in ('cgi', 'luci'):
|
||||
installed = self.root / self.targets[item].lstrip('/')
|
||||
self.assertIn('v116', installed.read_text(encoding='utf-8'))
|
||||
self.assertIn('v117', installed.read_text(encoding='utf-8'))
|
||||
for item, data in self.initial.items():
|
||||
self.assertEqual(self.backup.joinpath(item).read_bytes(), data)
|
||||
result = self.deploy('Rollback')
|
||||
@@ -130,7 +131,7 @@ logger() { printf '%s\\n' "$3" | sh "$TEST_LOG_HELPER" append; }
|
||||
self.assert_initial()
|
||||
|
||||
def test_rollback_restores_existing_log_service_and_helper(self):
|
||||
# This fix is normally applied on top of v116's already installed journal.
|
||||
# This fix is normally applied on top of v117's already installed journal.
|
||||
for item in ('logs.sh', 'logs-init'):
|
||||
target = self.root / self.targets[item].lstrip('/')
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -20,7 +20,7 @@ class VpsLogDeployTests(unittest.TestCase):
|
||||
self.target = self.root / 'hub.py'
|
||||
self.previous = b'print("previous Hub")\n'
|
||||
self.target.write_bytes(self.previous)
|
||||
self.backup = self.root / 'hub.py.bak-before-global-logs-ui-v116'
|
||||
self.backup = self.root / 'hub.py.bak-before-global-logs-ui-v117'
|
||||
source = (ROOT / 'vps/deploy-global-logs-vps.ps1').read_text(encoding='utf-8')
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace('/opt/owrt-remote/owrt-remote-hub.py', self.target.as_posix())
|
||||
@@ -55,7 +55,7 @@ sleep() { return 0; }
|
||||
for _ in range(2):
|
||||
result = self.deploy('Install')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn('GLOBAL_LOGS_VPS_V116_Install_OK', result.stdout)
|
||||
self.assertIn('GLOBAL_LOGS_VPS_V117_Install_OK', result.stdout)
|
||||
self.assertIn(b'def receive_router_logs(', self.target.read_bytes())
|
||||
self.assertEqual(self.backup.read_bytes(), self.previous)
|
||||
result = self.deploy('Rollback')
|
||||
|
||||
@@ -25,7 +25,7 @@ class HeartbeatTransportDeploymentTests(unittest.TestCase):
|
||||
self.env["PATH"] = str(GIT_BIN) + os.pathsep + self.env["PATH"]
|
||||
source = (ROOT / "vps/deploy-heartbeat-transport-fix.ps1").read_text(encoding="utf-8")
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-heartbeat-transport-v116",
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-heartbeat-transport-v117",
|
||||
"backup_dir=" + shlex.quote(self.backup.as_posix()))
|
||||
self.script = self.script.replace("printf /usr/lib/owrt-remote/wan.sh", "printf %s " + shlex.quote(self.targets["wan.sh"].as_posix()))
|
||||
self.script = self.script.replace("printf /usr/sbin/owrt-remote", "printf %s " + shlex.quote(self.targets["agent"].as_posix()))
|
||||
|
||||
@@ -0,0 +1,423 @@
|
||||
"""Persistent spool recovery/resource limits using the real shell helper.
|
||||
|
||||
Mount fixtures replace /proc/mounts only in disposable copies. Set
|
||||
OWRT_LOG_TEST_SHELL to a BusyBox executable to exercise its actual ash/applets.
|
||||
"""
|
||||
import base64
|
||||
import concurrent.futures
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
GIT_BIN = Path(os.environ.get('ProgramFiles', 'C:/Program Files')) / 'Git/usr/bin'
|
||||
SH = str(GIT_BIN / 'sh.exe') if os.name == 'nt' else shutil.which('sh')
|
||||
BUSYBOX = os.environ.get('OWRT_LOG_TEST_SHELL', '')
|
||||
|
||||
|
||||
class PersistentLogsTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix='persistent-logs-')
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name).resolve()
|
||||
self.run_dir = self.root / 'runtime'
|
||||
self.flash = self.root / 'flash'
|
||||
self.flash.mkdir()
|
||||
self.helper = self.root / 'logs.sh'
|
||||
self.mounts = self.root / 'mounts'
|
||||
self.root_mount = self.root.as_posix()
|
||||
self.flash_mount = self.flash.as_posix()
|
||||
if os.name == 'nt' and not BUSYBOX:
|
||||
# Git's bundled readlink may emit native C:/ or MSYS /c paths.
|
||||
# The fixture must use what the actual applet returns, not assume it.
|
||||
env = {**os.environ, 'PATH': str(GIT_BIN) + os.pathsep + os.environ['PATH']}
|
||||
def resolved_mount(path):
|
||||
return subprocess.check_output([SH, '-c', 'readlink -f "$1"', '--', path.as_posix()],
|
||||
env=env, text=True).strip()
|
||||
self.root_mount = resolved_mount(self.root)
|
||||
self.flash_mount = resolved_mount(self.flash)
|
||||
self.mounts.write_text(f'tmpfs {self.root_mount} tmpfs rw 0 0\n'
|
||||
f'/dev/ubi0_1 {self.flash_mount} ubifs rw 0 0\n')
|
||||
source = (ROOT / 'files/usr/lib/owrt-remote/logs.sh').read_text(encoding='utf-8')
|
||||
source = source.replace('/proc/mounts', self.mounts.as_posix())
|
||||
if os.name == 'nt':
|
||||
source = source.replace('case "$path" in /*)', 'case "$path" in [a-zA-Z]:/*|/*)')
|
||||
source = source.replace('log_clock() { cut -d. -f1 /proc/uptime; }',
|
||||
'log_clock() { echo "${TEST_UPTIME:-1000}"; }')
|
||||
source = source.replace('log_epoch() { date +%s; }',
|
||||
'log_epoch() { echo "${TEST_EPOCH:-' + str(int(time.time())) + '}"; }')
|
||||
# Real df remains covered in integration tests. Fixtures emulate free space.
|
||||
source = source.replace('free="$(df -Pk "$LOG_DIR" | awk \'NR==2 {print $4}\')"',
|
||||
'free="${TEST_FREE_KB:-8192}"')
|
||||
self.source = source
|
||||
self.source = self.source.replace('legacy=/tmp/owrt-remote-logs', 'legacy=' + (self.root / 'legacy').as_posix())
|
||||
self.helper.write_text(self.source, encoding='utf-8', newline='\n')
|
||||
self.env = {**os.environ, 'OWRT_REMOTE_LOG_RUNTIME': self.run_dir.as_posix(),
|
||||
'OWRT_REMOTE_LOG_DIR': (self.flash / 'journal').as_posix()}
|
||||
self.env['PATH'] = str(GIT_BIN) + os.pathsep + self.env['PATH']
|
||||
self.shell = [BUSYBOX, 'ash'] if BUSYBOX else [SH]
|
||||
if BUSYBOX:
|
||||
self.env['BB_OVERRIDE_APPLETS'] = '1'
|
||||
|
||||
def call(self, command, text='', *args, check=True, **flags):
|
||||
result = subprocess.run([*self.shell, self.helper.as_posix(), command, *args],
|
||||
input=text, encoding='utf-8', capture_output=True,
|
||||
env={**self.env, **flags}, timeout=25)
|
||||
if check:
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
return result
|
||||
|
||||
def status(self):
|
||||
return dict(line.split('=', 1) for line in self.call('status').stdout.splitlines())
|
||||
|
||||
def flush(self, **flags):
|
||||
return self.call('flush', '', 'force', **flags)
|
||||
|
||||
def export(self):
|
||||
return json.loads(self.call('export').stdout)
|
||||
|
||||
def append(self, text, **flags):
|
||||
return self.call('append', text, **flags)
|
||||
|
||||
def acknowledge(self, batch):
|
||||
self.call('ack', '', batch['journal_id'], str(batch['to']))
|
||||
|
||||
@property
|
||||
def journal(self):
|
||||
return self.flash / 'journal'
|
||||
|
||||
def reboot(self):
|
||||
shutil.rmtree(self.run_dir)
|
||||
|
||||
def test_persistent_storage_selected_and_survives_runtime_loss(self):
|
||||
self.append('before reboot\n')
|
||||
self.flush()
|
||||
first = self.export()
|
||||
self.acknowledge(first)
|
||||
self.flush()
|
||||
self.reboot()
|
||||
self.assertEqual(self.status()['persistent'], '1')
|
||||
self.assertIn('before reboot', self.call('read').stdout)
|
||||
after = self.export()
|
||||
self.assertEqual(after['journal_id'], first['journal_id'])
|
||||
self.assertEqual(after['from'], first['to'])
|
||||
self.assertEqual(after['data'], '')
|
||||
self.assertFalse(any(self.journal.glob('stream.*')))
|
||||
self.assertFalse((self.journal / 'lock').exists())
|
||||
|
||||
def test_missing_optional_cksum_does_not_create_empty_digest_or_drop_facts(self):
|
||||
source = self.source.replace('case "${1:-}" in', 'cksum() { return 127; }\ncase "${1:-}" in', 1)
|
||||
self.helper.write_text(source, encoding='utf-8', newline='\n')
|
||||
self.append('first event\nsecond event\n')
|
||||
self.flush()
|
||||
part = next(self.journal.glob('part.*'))
|
||||
self.assertIn('.s256-', part.name)
|
||||
self.assertIn(b'second event', base64.b64decode(self.export()['data']))
|
||||
self.reboot()
|
||||
self.assertIn(b'first event', base64.b64decode(self.export()['data']))
|
||||
|
||||
def test_digest_fingerprint_cache_stays_small_at_64_entries(self):
|
||||
self.call('check')
|
||||
# Seed 63 existing fingerprints; exercise eviction with two real events
|
||||
# instead of starting thousands of Windows utility processes for a fill.
|
||||
(self.run_dir / 'facts').write_text(''.join('1000\ts256-%035d\n' % i for i in range(63)))
|
||||
self.append('distinct event 63\ndistinct event 64\n')
|
||||
self.assertEqual(len((self.run_dir / 'facts').read_text().splitlines()), 64)
|
||||
self.assertLessEqual((self.run_dir / 'facts').stat().st_size, 4096)
|
||||
|
||||
def test_empty_digest_files_falsely_acked_by_old_hub_are_recovered(self):
|
||||
self.append('retained restart event\n'); self.flush()
|
||||
before = self.export()
|
||||
self.acknowledge(before); self.flush()
|
||||
part = next(self.journal.glob('part.*'))
|
||||
part.rename(part.with_name(part.name.rsplit('.', 1)[0] + '.'))
|
||||
# Upgrade during the same boot must override the former recovery marker.
|
||||
(self.run_dir / 'recovered-v2').unlink()
|
||||
(self.run_dir / 'recovered').touch()
|
||||
repaired = self.export()
|
||||
self.assertNotEqual(repaired['journal_id'], before['journal_id'])
|
||||
self.assertTrue(repaired['journal_id'].startswith('recovered-'))
|
||||
self.assertEqual(repaired['from'], 0)
|
||||
self.assertIn(b'retained restart event', base64.b64decode(repaired['data']))
|
||||
self.assertFalse(any(p.name.endswith('.') for p in self.journal.glob('part.*')))
|
||||
self.acknowledge(repaired)
|
||||
self.assertEqual(self.export()['data'], '')
|
||||
self.flush()
|
||||
# Recovery's warning was buffered after the replay chunk; deliver it too.
|
||||
self.acknowledge(self.export())
|
||||
self.flush()
|
||||
self.reboot()
|
||||
self.assertEqual(self.export()['data'], '')
|
||||
|
||||
def test_checksum_failure_keeps_pending_and_does_not_advance_total(self):
|
||||
self.append('keep until checksum works\n')
|
||||
before = self.status()['committed_bytes']
|
||||
source = self.source.replace('log_checksum() {', 'log_checksum() { return 1;', 1)
|
||||
self.helper.write_text(source, encoding='utf-8', newline='\n')
|
||||
self.call('flush', '', 'force', check=False)
|
||||
self.assertFalse(list(self.journal.glob('part.*')))
|
||||
self.assertEqual(self.status()['committed_bytes'], before)
|
||||
self.assertIn('keep until checksum works', (self.run_dir / 'pending').read_text())
|
||||
|
||||
def test_md5_and_legacy_numeric_checksum_formats_remain_readable(self):
|
||||
for digest in ('md5', 'crc'):
|
||||
with self.subTest(digest=digest):
|
||||
self.call('clear')
|
||||
source = self.source.replace('if command -v sha256sum >/dev/null 2>&1; then method=s256',
|
||||
'if false; then method=s256')
|
||||
if digest == 'crc':
|
||||
source = source.replace('elif command -v md5sum >/dev/null 2>&1; then method=md5',
|
||||
'elif false; then method=md5')
|
||||
self.helper.write_text(source, encoding='utf-8', newline='\n')
|
||||
self.append('digest compatibility ' + digest + '\n'); self.flush()
|
||||
value = self.export()
|
||||
self.reboot()
|
||||
self.assertEqual(self.export(), value)
|
||||
self.assertIn(digest.encode(), base64.b64decode(value['data']))
|
||||
name = next(self.journal.glob('part.*')).name.rsplit('.', 1)[1]
|
||||
self.assertEqual(name.startswith('md5-'), digest == 'md5')
|
||||
|
||||
def test_empty_export_never_claims_unreadable_chunks_as_delivered(self):
|
||||
self.append('not yet delivered\n'); self.flush()
|
||||
part = next(self.journal.glob('part.*'))
|
||||
part.rename(part.with_name(part.name.rsplit('.', 1)[0] + '.invalid'))
|
||||
empty = self.export()
|
||||
self.assertEqual((empty['from'], empty['to'], empty['data']), (0, 0, ''))
|
||||
|
||||
def test_failed_or_unmounted_overlay_does_not_receive_journal(self):
|
||||
for kind, flags in [('overlay', 'rw'), ('tmpfs', 'rw'), ('ubifs', 'ro')]:
|
||||
with self.subTest(kind=kind, flags=flags):
|
||||
if self.run_dir.exists(): self.reboot()
|
||||
self.mounts.write_text(f'tmpfs {self.root_mount} tmpfs rw 0 0\n'
|
||||
f'fake {self.flash_mount} {kind} {flags} 0 0\n')
|
||||
self.append('RAM fallback\n')
|
||||
self.assertEqual(self.status()['persistent'], '0')
|
||||
self.assertIn('Flash недоступна', self.call('read').stdout)
|
||||
self.assertFalse(self.journal.exists())
|
||||
|
||||
def test_nested_tmpfs_and_symlink_do_not_pass_flash_check(self):
|
||||
nested = self.flash / 'ram'
|
||||
nested.mkdir()
|
||||
self.mounts.write_text(self.mounts.read_text() + f'tmpfs {self.flash_mount}/ram tmpfs rw 0 0\n')
|
||||
self.env['OWRT_REMOTE_LOG_DIR'] = (nested / 'logs').as_posix()
|
||||
self.append('nested tmpfs\n')
|
||||
self.assertEqual(self.status()['persistent'], '0')
|
||||
self.assertFalse((nested / 'logs').exists())
|
||||
|
||||
def test_periodic_flush_and_idle_polling_do_not_rewrite_flash(self):
|
||||
self.call('check')
|
||||
state = self.journal / 'state'
|
||||
before = state.stat().st_mtime_ns
|
||||
self.append('buffered record\n')
|
||||
self.assertEqual(list(self.journal.glob('part.*')), [])
|
||||
self.assertEqual(self.export()['data'], '')
|
||||
self.call('flush', TEST_UPTIME='1059')
|
||||
self.assertEqual(list(self.journal.glob('part.*')), [])
|
||||
self.call('flush', TEST_UPTIME='1060')
|
||||
self.assertEqual(len(list(self.journal.glob('part.*'))), 1)
|
||||
self.assertNotEqual(state.stat().st_mtime_ns, before)
|
||||
stable = {p.name: p.stat().st_mtime_ns for p in self.journal.iterdir()}
|
||||
for _ in range(3):
|
||||
self.call('flush', TEST_UPTIME='1120')
|
||||
self.export()
|
||||
self.assertEqual({p.name: p.stat().st_mtime_ns for p in self.journal.iterdir()}, stable)
|
||||
|
||||
def test_buffer_and_number_of_archives_are_bounded_without_dropping_unacked(self):
|
||||
self.helper.write_text(self.source.replace('LOG_LIMIT=2097152', 'LOG_LIMIT=32768')
|
||||
.replace('LOG_PART_LIMIT=256', 'LOG_PART_LIMIT=2'),
|
||||
encoding='utf-8', newline='\n')
|
||||
for i in range(4):
|
||||
self.append(f'event {i} ' + 'x' * 4000 + '\n')
|
||||
self.flush()
|
||||
parts = sorted(self.journal.glob('part.*'))
|
||||
self.assertEqual(len(parts), 2)
|
||||
original = {p.name: p.read_bytes() for p in parts}
|
||||
for i in range(5):
|
||||
self.append(f'overflow {i} ' + 'y' * 4000 + '\n')
|
||||
self.flush()
|
||||
self.assertEqual({p.name: p.read_bytes() for p in parts}, original)
|
||||
self.assertLessEqual((self.run_dir / 'pending').stat().st_size, 16384)
|
||||
self.assertLessEqual((self.run_dir / 'facts').stat().st_size, 4096)
|
||||
first = self.export()
|
||||
self.acknowledge(first)
|
||||
self.flush()
|
||||
self.assertFalse(parts[0].exists(), 'only acknowledged chunks make room')
|
||||
self.assertIn('пропущено новых событий', self.call('read').stdout)
|
||||
|
||||
def test_ack_is_monotonic_and_does_not_delete_unconfirmed_after_one_day(self):
|
||||
self.append('not delivered\n')
|
||||
self.flush()
|
||||
batch = self.export()
|
||||
self.call('flush', '', 'force', TEST_EPOCH='2000000000')
|
||||
self.assertEqual(self.export(), batch)
|
||||
invalid = self.call('ack', '', batch['journal_id'], str(batch['to'] + 1), check=False)
|
||||
self.assertNotEqual(invalid.returncode, 0)
|
||||
self.acknowledge(batch)
|
||||
self.call('flush', '', 'force', TEST_EPOCH='2000000000')
|
||||
self.assertEqual(list(self.journal.glob('part.*')), [])
|
||||
invalid = self.call('ack', '', batch['journal_id'], '0', check=False)
|
||||
self.assertNotEqual(invalid.returncode, 0)
|
||||
|
||||
def test_crash_after_chunk_rename_recovers_original_id_and_offsets(self):
|
||||
self.append('committed first\n')
|
||||
self.flush()
|
||||
first = self.export()
|
||||
previous_state = (self.journal / 'state').read_bytes()
|
||||
self.append('second chunk before state checkpoint\n')
|
||||
self.flush()
|
||||
(self.journal / 'state').write_bytes(previous_state)
|
||||
(self.journal / 'chunk.new').write_text('interrupted write')
|
||||
(self.journal / 'state.new').write_text('partial state')
|
||||
self.reboot()
|
||||
self.assertEqual(self.export()['journal_id'], first['journal_id'])
|
||||
self.acknowledge(first)
|
||||
second = self.export()
|
||||
self.assertEqual(second['from'], first['to'])
|
||||
self.assertIn(b'second chunk', base64.b64decode(second['data']))
|
||||
self.assertFalse((self.journal / 'chunk.new').exists())
|
||||
self.assertFalse((self.journal / 'state.new').exists())
|
||||
|
||||
def test_corrupt_chunk_is_quarantined_and_later_chunks_still_deliver(self):
|
||||
self.append('first\n'); self.flush()
|
||||
self.append('second\n'); self.flush()
|
||||
first = sorted(self.journal.glob('part.*'))[0]
|
||||
first.write_bytes(b'partial')
|
||||
self.reboot()
|
||||
batch = self.export()
|
||||
self.assertIn(b'second', base64.b64decode(batch['data']))
|
||||
self.assertTrue((self.journal / 'damaged.log').exists())
|
||||
self.assertIn('Повреждённый фрагмент', self.call('read').stdout)
|
||||
|
||||
def test_disk_full_falls_back_without_touching_router_configuration(self):
|
||||
self.append('retained on flash\n'); self.flush()
|
||||
self.append('buffered before ENOSPC\n')
|
||||
self.call('flush', '', 'force', TEST_FREE_KB='100', check=False)
|
||||
self.append('after disk full\n')
|
||||
self.assertEqual(self.status()['persistent'], '0')
|
||||
self.assertIn('after disk full', self.call('read').stdout)
|
||||
self.assertTrue(list(self.journal.glob('part.*')))
|
||||
|
||||
def test_confirmed_ack_can_replay_after_crash_without_changing_cursor_identity(self):
|
||||
self.append('duplicate safe\n'); self.flush()
|
||||
batch = self.export()
|
||||
self.acknowledge(batch)
|
||||
self.assertEqual(self.export()['data'], '')
|
||||
self.reboot() # ack not yet checkpointed: safe replay, Hub suppresses it.
|
||||
self.assertEqual(self.export(), batch)
|
||||
|
||||
def test_killed_flusher_reconciles_inflight_without_duplicate_data(self):
|
||||
self.append('already renamed\n')
|
||||
self.flush()
|
||||
batch = self.export()
|
||||
part = next(self.journal.glob('part.*'))
|
||||
fields = part.name.rsplit('.', 3)
|
||||
(self.run_dir / 'pending').write_bytes(base64.b64decode(batch['data']))
|
||||
(self.run_dir / 'inflight').write_text(f'{batch["journal_id"]} 0 {batch["to"]} {fields[-1]}\n')
|
||||
self.append('after killed writer\n')
|
||||
text = self.call('read').stdout
|
||||
self.assertEqual(text.count('already renamed'), 1)
|
||||
self.assertIn('after killed writer', text)
|
||||
self.assertFalse((self.run_dir / 'inflight').exists())
|
||||
|
||||
def test_corrupt_state_recovers_committed_generation_and_stale_temporaries(self):
|
||||
self.append('state recovery\n'); self.flush()
|
||||
batch = self.export()
|
||||
(self.journal / 'state').write_text('partial garbage')
|
||||
self.reboot()
|
||||
self.assertEqual(self.export(), batch)
|
||||
(self.run_dir / 'export.12345').write_text('orphan')
|
||||
self.export()
|
||||
self.assertFalse(list(self.run_dir.glob('export.*')))
|
||||
|
||||
def test_parallel_writers_and_stale_lock_recovery(self):
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as pool:
|
||||
list(pool.map(lambda i: self.append(f'concurrent {i}\n'), range(6)))
|
||||
self.assertEqual(len(self.call('read').stdout.splitlines()), 6)
|
||||
(self.run_dir / 'lock').mkdir()
|
||||
(self.run_dir / 'lock/pid').write_text('99999999')
|
||||
self.append('recovered lock\n')
|
||||
self.assertIn('recovered lock', self.call('read').stdout)
|
||||
self.assertFalse((self.run_dir / 'lock').exists())
|
||||
|
||||
def test_stale_lock_reaper_itself_recovers_after_process_crash(self):
|
||||
self.call('check')
|
||||
lock = self.run_dir / 'lock'; lock.mkdir()
|
||||
(lock / 'pid').write_text('99999999')
|
||||
(lock / 'reap').mkdir(); (lock / 'reap/pid').write_text('99999998')
|
||||
self.append('after killed reaper\n')
|
||||
self.assertIn('after killed reaper', self.call('read').stdout)
|
||||
self.assertFalse(lock.exists())
|
||||
|
||||
def test_collector_failure_reaps_children_and_removes_fifo(self):
|
||||
# Actual FIFO integration is tested on POSIX; Windows BusyBox has no
|
||||
# FIFO support. Exercise its traps with mocked process/FIFO operations.
|
||||
mocks = r'''
|
||||
mkfifo() { : >"$1"; }
|
||||
logread() { return 0; }
|
||||
sleep() { return 0; }
|
||||
kill() {
|
||||
printf '%s\n' "$*" >>"$RUN_DIR/kills"
|
||||
[ "$1" != -0 ]
|
||||
}
|
||||
wait() { return 0; }
|
||||
'''
|
||||
source = self.source.replace('case "${1:-}" in', mocks + '\ncase "${1:-}" in', 1)
|
||||
self.helper.write_text(source, encoding='utf-8', newline='\n')
|
||||
result = self.call('collect', check=False)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse(list(self.run_dir.glob('stream.*')))
|
||||
self.assertFalse((self.run_dir / 'collector-lock').exists())
|
||||
lines = (self.run_dir / 'kills').read_text().splitlines()
|
||||
self.assertGreaterEqual(len([line for line in lines if not line.startswith('-0')]), 2)
|
||||
|
||||
def test_legacy_migration_preserves_identity_ack_and_only_undelivered_bytes(self):
|
||||
legacy = self.root / 'legacy'
|
||||
legacy.mkdir()
|
||||
(legacy / 'current.log').write_bytes(b'legacy current\n')
|
||||
(legacy / 'previous.log').write_bytes(b'legacy previous\n')
|
||||
raw = b'legacy previous\nlegacy current\n'
|
||||
(legacy / 'total').write_text(str(len(raw)))
|
||||
(legacy / 'journal-id').write_text('legacy-journal')
|
||||
(legacy / 'sync-ack').write_text('legacy-journal 16')
|
||||
(legacy / 'stream.dead').write_text('must stay transient')
|
||||
self.call('migrate')
|
||||
self.assertEqual(self.call('read').stdout, raw.decode())
|
||||
self.assertFalse((self.journal / 'stream.dead').exists())
|
||||
self.assertFalse((legacy / 'current.log').exists())
|
||||
self.assertFalse((legacy / 'total').exists())
|
||||
self.assertTrue((legacy / 'stream.dead').exists())
|
||||
batch = self.export()
|
||||
self.assertEqual(batch['journal_id'], 'legacy-journal')
|
||||
self.assertEqual(batch['from'], 16)
|
||||
self.assertEqual(base64.b64decode(batch['data']), b'legacy current\n')
|
||||
|
||||
def test_unavailable_flash_legacy_migration_does_not_keep_large_ram_archive(self):
|
||||
self.mounts.write_text(f'tmpfs {self.root_mount} tmpfs rw 0 0\n')
|
||||
legacy = self.root / 'legacy'; legacy.mkdir()
|
||||
(legacy / 'current.log').write_text('old data\n' * 10000)
|
||||
self.call('migrate')
|
||||
self.assertFalse((legacy / 'current.log').exists())
|
||||
self.assertEqual(self.status()['persistent'], '0')
|
||||
self.assertLessEqual((self.run_dir / 'pending').stat().st_size, 16384)
|
||||
self.assertIn('ограниченный хвост', self.call('read').stdout)
|
||||
|
||||
def test_interrupted_legacy_migration_finishes_after_old_ram_is_gone(self):
|
||||
self.append('staged persistent data\n'); self.flush()
|
||||
batch = self.export()
|
||||
staging = self.journal / 'migrating'; staging.mkdir()
|
||||
for part in self.journal.glob('part.*'):
|
||||
part.rename(staging / part.name)
|
||||
shutil.copyfile(self.journal / 'state', staging / 'state')
|
||||
(self.journal / 'state').write_text('interrupted state replacement')
|
||||
self.reboot()
|
||||
self.assertEqual(self.export(), batch)
|
||||
self.assertFalse(staging.exists())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,148 @@
|
||||
"""Install/rollback journal files against a sandbox, never a real router."""
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
GIT_BIN = Path(os.environ.get('ProgramFiles', 'C:/Program Files')) / 'Git/usr/bin'
|
||||
SH = str(GIT_BIN / 'sh.exe') if os.name == 'nt' else shutil.which('sh')
|
||||
|
||||
|
||||
@unittest.skipUnless(SH, 'POSIX shell required')
|
||||
class PersistentLogsDeploymentTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix='persistent-log-deploy-')
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.targets = {
|
||||
'cgi': '/www/cgi-bin/owrt-remote', 'agent': '/usr/sbin/owrt-remote',
|
||||
'init': '/etc/init.d/owrt-remote', 'logs.sh': '/usr/lib/owrt-remote/logs.sh',
|
||||
'logs-init': '/etc/init.d/owrt-remote-logs',
|
||||
'luci': '/www/luci-static/resources/view/owrt_remote.js',
|
||||
}
|
||||
self.initial = {}
|
||||
for item in ('cgi', 'agent', 'logs.sh', 'logs-init', 'luci'):
|
||||
path = self.root / self.targets[item].lstrip('/')
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text('#!/bin/sh\necho previous-' + item + '\n', encoding='utf-8', newline='\n')
|
||||
self.initial[item] = path.read_bytes()
|
||||
self.backup = self.root / 'backup'
|
||||
source = (ROOT / 'vps/deploy-persistent-logs.ps1').read_text(encoding='utf-8')
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
# Absolute paths are replaced only in this disposable test copy.
|
||||
mappings = {target: (self.root / target.lstrip('/')).as_posix() for target in self.targets.values()}
|
||||
mappings['/usr/lib/owrt-remote'] = (self.root / 'usr/lib/owrt-remote').as_posix()
|
||||
mappings['/www/luci-static/resources/view'] = (self.root / 'www/luci-static/resources/view').as_posix()
|
||||
mappings['/tmp/luci-indexcache'] = (self.root / 'luci-indexcache').as_posix()
|
||||
mappings['/tmp/luci-modulecache'] = (self.root / 'luci-modulecache').as_posix()
|
||||
mappings['/etc/init.d/rpcd'] = (self.root / 'etc/init.d/rpcd').as_posix()
|
||||
mappings['/etc/init.d/uhttpd'] = (self.root / 'etc/init.d/uhttpd').as_posix()
|
||||
mappings['/tmp/owrt-remote-logs'] = (self.root / 'journal').as_posix()
|
||||
mappings['/root/owrt-remote-backup-before-persistent-logs-v117'] = self.backup.as_posix()
|
||||
pattern = '|'.join(re.escape(target) for target in sorted(mappings, key=len, reverse=True))
|
||||
self.script = re.sub(pattern, lambda match: mappings[match.group()], self.script)
|
||||
# Existing sandbox service scripts only print markers. Mock readiness.
|
||||
mocks = '''
|
||||
logread() { return 0; }
|
||||
ubus() { printf '{"running": true}\\n'; }
|
||||
mkfifo() { return 0; }
|
||||
jsonfilter() { return 0; }
|
||||
logger() { printf '%s\\n' "$3" | sh "$TEST_LOG_HELPER" append; }
|
||||
'''
|
||||
self.script = self.script.replace('for item in cgi agent logs.sh logs-init;', mocks + '\nfor item in cgi agent logs.sh logs-init;', 1)
|
||||
self.env = {**os.environ, 'OWRT_REMOTE_LOG_DIR': (self.root / 'journal').as_posix(), 'OWRT_REMOTE_LOG_RUNTIME': (self.root / 'runtime').as_posix()}
|
||||
self.env['TEST_LOG_HELPER'] = (self.root / 'usr/lib/owrt-remote/logs.sh').as_posix()
|
||||
if os.name == 'nt':
|
||||
self.env['PATH'] = str(GIT_BIN) + os.pathsep + self.env['PATH']
|
||||
|
||||
def deploy(self, action, bad=False, fail_verify=False):
|
||||
stage = self.root / 'stage'
|
||||
stage.mkdir()
|
||||
candidates = {
|
||||
'cgi': 'files/www/cgi-bin/owrt-remote', 'agent': 'files/usr/sbin/owrt-remote',
|
||||
'init': 'files/etc/init.d/owrt-remote', 'logs.sh': 'files/usr/lib/owrt-remote/logs.sh',
|
||||
'logs-init': 'files/etc/init.d/owrt-remote-logs',
|
||||
'luci': 'files/www/luci-static/resources/view/owrt_remote.js',
|
||||
}
|
||||
if action == 'Install':
|
||||
for item, source in candidates.items():
|
||||
text = (ROOT / source).read_text(encoding='utf-8')
|
||||
if item == 'logs.sh':
|
||||
text = text.replace('legacy=/tmp/owrt-remote-logs', 'legacy=' + shlex.quote((self.root / 'legacy').as_posix()))
|
||||
text = text.replace('log_clock() { cut -d. -f1 /proc/uptime; }', 'log_clock() { echo 1000; }')
|
||||
text = text.replace('log_ram_storage() {', 'log_ram_storage() { return 0;', 1)
|
||||
text = text.replace('log_flash_storage() {', 'log_flash_storage() { return 0;', 1)
|
||||
if item == 'logs-init':
|
||||
text = '#!/bin/sh\necho logs-service-$1\n'
|
||||
if bad and item == 'cgi':
|
||||
text = '#!/bin/sh\nif true; then\n'
|
||||
stage.joinpath(item).write_text(text, encoding='utf-8', newline='\n')
|
||||
script = self.script.replace('stage=/tmp/__UPLOAD_NAME__', 'stage=' + shlex.quote(stage.as_posix()))
|
||||
script = script.replace('__UPLOAD_NAME__', 'sandbox').replace('__ACTION__', action)
|
||||
if fail_verify:
|
||||
script = script.replace('cmp "$stage/$item" "$path"', 'false')
|
||||
result = subprocess.run([SH], input=script, encoding='utf-8', capture_output=True,
|
||||
env=self.env, timeout=30)
|
||||
self.assertFalse(stage.exists())
|
||||
self.assertFalse(list(self.root.rglob('*.new-sandbox')))
|
||||
return result
|
||||
|
||||
def assert_initial(self):
|
||||
for item, data in self.initial.items():
|
||||
self.assertEqual((self.root / self.targets[item].lstrip('/')).read_bytes(), data)
|
||||
|
||||
def test_install_then_rollback_retains_first_backup(self):
|
||||
for _ in range(2):
|
||||
result = self.deploy('Install')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn('PERSISTENT_LOGS_ROUTER_Install_OK', result.stdout)
|
||||
for item in ('cgi', 'luci'):
|
||||
installed = self.root / self.targets[item].lstrip('/')
|
||||
self.assertIn('v117', installed.read_text(encoding='utf-8'))
|
||||
for item, data in self.initial.items():
|
||||
self.assertEqual(self.backup.joinpath(item).read_bytes(), data)
|
||||
result = self.deploy('Rollback')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn('PERSISTENT_LOGS_ROUTER_Rollback_OK', result.stdout)
|
||||
self.assert_initial()
|
||||
|
||||
def test_missing_backup_changes_nothing(self):
|
||||
result = self.deploy('Rollback')
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('No complete backup:', result.stderr)
|
||||
self.assert_initial()
|
||||
|
||||
def test_invalid_candidate_rejected_before_backup(self):
|
||||
result = self.deploy('Install', bad=True)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse(self.backup.exists())
|
||||
self.assert_initial()
|
||||
|
||||
def test_failed_verification_restores_all_previous_files(self):
|
||||
result = self.deploy('Install', fail_verify=True)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('Restoring files from this attempt.', result.stderr)
|
||||
self.assert_initial()
|
||||
|
||||
def test_rollback_restores_existing_log_service_and_helper(self):
|
||||
# This fix is normally applied on top of v117's already installed journal.
|
||||
for item in ('logs.sh', 'logs-init'):
|
||||
target = self.root / self.targets[item].lstrip('/')
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text('#!/bin/sh\necho previous-' + item + '-$1\n', encoding='utf-8', newline='\n')
|
||||
self.initial[item] = target.read_bytes()
|
||||
result = self.deploy('Install')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
result = self.deploy('Rollback')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
for item, data in self.initial.items():
|
||||
self.assertEqual((self.root / self.targets[item].lstrip('/')).read_bytes(), data)
|
||||
self.assertIn('previous-logs-init-start', result.stdout)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Validate VPS install, rollback and automatic recovery in a disposable tree."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
GIT_BIN = Path(os.environ.get('ProgramFiles', 'C:/Program Files')) / 'Git/usr/bin'
|
||||
SH = str(GIT_BIN / 'sh.exe') if os.name == 'nt' else shutil.which('sh')
|
||||
|
||||
|
||||
@unittest.skipUnless(SH, 'POSIX shell required')
|
||||
class VpsLogDeployTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix='vps-log-deploy-')
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.target = self.root / 'hub.py'
|
||||
self.previous = b'print("previous Hub")\n'
|
||||
self.target.write_bytes(self.previous)
|
||||
self.backup = self.root / 'hub.py.bak-before-persistent-logs-sync-v117'
|
||||
source = (ROOT / 'vps/deploy-persistent-logs-vps.ps1').read_text(encoding='utf-8')
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace('/opt/owrt-remote/owrt-remote-hub.py', self.target.as_posix())
|
||||
self.script = self.script.replace('python_bin=/opt/owrt-remote/venv/bin/python', 'python_bin=python')
|
||||
self.script = self.script.replace('[ -x "$python_bin" ] || python_bin=python3', '')
|
||||
mocks = '''
|
||||
systemctl() { [ "${TEST_RESTART_FAIL:-0}" != 1 ] || [ "$1" != restart ]; }
|
||||
journalctl() { echo sandbox-journal; }
|
||||
curl() { [ "${TEST_HEALTH_FAIL:-0}" != 1 ] && printf '{"ok":true}'; }
|
||||
sleep() { return 0; }
|
||||
'''
|
||||
self.script = self.script.replace('test -s "$target"', mocks + '\ntest -s "$target"', 1)
|
||||
# Exercise the actual mount/path validation with an isolated process proc
|
||||
# fixture. Storage classification is covered independently on real paths.
|
||||
original = self.root / 'running.py'
|
||||
original.write_text('print("running")')
|
||||
proc = self.root / 'proc' / '123'
|
||||
proc.mkdir(parents=True)
|
||||
(proc / 'environ').write_bytes(('OWRT_REMOTE_STATE_DIR=' + self.root.as_posix()).encode() + b'\0')
|
||||
(proc / 'cmdline').write_bytes(b'python\0' + original.as_posix().encode() + b'\0')
|
||||
database = self.root / 'hub.db'
|
||||
database.write_bytes(b'existing archive must not change')
|
||||
self.database = database
|
||||
self.script = self.script.replace("Path('/proc/' + pid", "Path('" + (self.root / 'proc').as_posix() + "/' + pid")
|
||||
self.script = self.script.replace("storage = hub.router_log_storage(db_path)",
|
||||
"storage = hub.router_log_storage(db_path)\n" +
|
||||
"storage.update(persistent=os.environ.get('TEST_RAM_DISK') != '1', writable=True)")
|
||||
self.script = self.script.replace('systemctl() { ',
|
||||
'systemctl() { if [ "$1" = show ]; then echo 123; return 0; fi; ')
|
||||
self.env = dict(os.environ)
|
||||
if os.name == 'nt':
|
||||
self.env['PATH'] = str(GIT_BIN) + os.pathsep + self.env['PATH']
|
||||
|
||||
def deploy(self, action, invalid=False, **flags):
|
||||
stage = self.root / 'stage'
|
||||
stage.mkdir()
|
||||
if action == 'Install':
|
||||
data = 'if invalid:\n' if invalid else (ROOT / 'vps/owrt-remote-hub.py').read_text(encoding='utf-8')
|
||||
stage.joinpath('hub.py').write_text(data, encoding='utf-8', newline='\n')
|
||||
script = self.script.replace('stage=/tmp/__UPLOAD_NAME__', 'stage="' + stage.as_posix() + '"')
|
||||
script = script.replace('__UPLOAD_NAME__', 'sandbox').replace('__ACTION__', action)
|
||||
result = subprocess.run([SH], input=script, encoding='utf-8', capture_output=True,
|
||||
env={**self.env, **flags}, timeout=25)
|
||||
self.assertFalse(stage.exists())
|
||||
self.assertFalse(list(self.root.glob('*.new-sandbox')))
|
||||
self.assertEqual(self.database.read_bytes(), b'existing archive must not change')
|
||||
return result
|
||||
|
||||
def test_install_and_rollback_preserve_original_backup(self):
|
||||
for _ in range(2):
|
||||
result = self.deploy('Install')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn('PERSISTENT_LOGS_VPS_V117_Install_OK', result.stdout)
|
||||
self.assertIn(b'def receive_router_logs(', self.target.read_bytes())
|
||||
self.assertEqual(self.backup.read_bytes(), self.previous)
|
||||
result = self.deploy('Rollback')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(self.target.read_bytes(), self.previous)
|
||||
|
||||
def test_failed_health_restores_file_from_this_attempt(self):
|
||||
self.assertEqual(self.deploy('Install').returncode, 0)
|
||||
installed = self.target.read_bytes()
|
||||
result = self.deploy('Rollback', TEST_HEALTH_FAIL='1')
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('Restoring the previous Hub file', result.stderr)
|
||||
self.assertEqual(self.target.read_bytes(), installed)
|
||||
|
||||
def test_ram_database_preflight_does_not_install_or_modify_archive(self):
|
||||
result = self.deploy('Install', TEST_RAM_DISK='1')
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('not confirmed on writable persistent disk', result.stderr)
|
||||
self.assertEqual(self.target.read_bytes(), self.previous)
|
||||
self.assertFalse(self.backup.exists())
|
||||
|
||||
def test_invalid_candidate_and_missing_backup_change_nothing(self):
|
||||
self.assertNotEqual(self.deploy('Install', invalid=True).returncode, 0)
|
||||
self.assertFalse(self.backup.exists())
|
||||
self.assertNotEqual(self.deploy('Rollback').returncode, 0)
|
||||
self.assertEqual(self.target.read_bytes(), self.previous)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -131,7 +131,7 @@ class HubFixture:
|
||||
assert "groups" in json.loads(self.request("/api/router-groups"))
|
||||
assert "notifications" in json.loads(self.request("/api/notifications"))
|
||||
assert self.request("/access/test-router/") == b"router proxy fixture"
|
||||
assert b"v116" in self.request("/")
|
||||
assert b"v117" in self.request("/")
|
||||
self.app.snapshot_router_states()
|
||||
|
||||
def __exit__(self, *args):
|
||||
|
||||
@@ -113,6 +113,81 @@ class ArchiveTests(unittest.TestCase):
|
||||
self.assertIn('новый журнал', self.snapshot()['text'])
|
||||
self.assertIn('first', self.snapshot()['text'])
|
||||
|
||||
def test_empty_status_cannot_confirm_missing_records_or_invent_rotation_gap(self):
|
||||
reply = self.ingest(batch(b'', start=298))
|
||||
self.assertEqual(reply['offset'], 0)
|
||||
self.assertEqual(self.snapshot()['text'], '')
|
||||
raw = b'[OK] retained records arrive after status\n'
|
||||
self.ingest(batch(raw))
|
||||
reply = self.ingest(batch(b'', start=1000))
|
||||
self.assertEqual(reply['offset'], len(raw))
|
||||
self.assertEqual(self.snapshot()['text'].encode(), raw)
|
||||
|
||||
def test_actual_gap_with_data_is_reported_once_but_retries_keep_data(self):
|
||||
value = batch(b'[OK] surviving records\n', start=50)
|
||||
self.ingest(value)
|
||||
self.ingest(value)
|
||||
text = self.snapshot()['text']
|
||||
self.assertEqual(text.count('не доставлена'), 1)
|
||||
self.assertEqual(text.count('surviving records'), 1)
|
||||
|
||||
def test_empty_new_generation_cannot_erase_current_journal_cursor(self):
|
||||
raw = b'[OK] previous generation\n'
|
||||
self.ingest(batch(raw))
|
||||
self.ingest(batch(b'', start=2000, journal='new-journal'))
|
||||
self.ingest(batch(raw))
|
||||
self.assertEqual(self.snapshot()['text'].encode(), raw)
|
||||
|
||||
def test_recovery_generation_replays_previously_falsely_confirmed_chunks(self):
|
||||
# Reproduce the cursor left by the former empty-batch ingestion bug.
|
||||
with self.app.conn() as conn:
|
||||
conn.execute("insert into router_logs(router_id,journal_id,cursor) values ('main','broken',298)")
|
||||
self.ingest(batch(b'[OK] retained restart event\n', journal='recovered-old'))
|
||||
self.assertIn('retained restart event', self.snapshot()['text'])
|
||||
|
||||
def test_real_router_spool_recovers_bad_digest_and_delivers_after_restart(self):
|
||||
from test_persistent_logs import PersistentLogsTests
|
||||
client = PersistentLogsTests()
|
||||
client.setUp()
|
||||
self.addCleanup(client.doCleanups)
|
||||
client.append('Панель: действие=stop; начало\nСервис: остановка туннеля\n')
|
||||
client.flush()
|
||||
original = client.export()
|
||||
client.acknowledge(original); client.flush()
|
||||
with self.app.conn() as conn:
|
||||
conn.execute('insert into router_logs(router_id,journal_id,cursor) values (?,?,?)',
|
||||
('main', original['journal_id'], original['to']))
|
||||
part = next(client.journal.glob('part.*'))
|
||||
part.rename(part.with_name(part.name.rsplit('.', 1)[0] + '.'))
|
||||
(client.run_dir / 'recovered-v2').unlink()
|
||||
repaired = client.export()
|
||||
reply = self.ingest(repaired, {'enabled': False, 'xray': 'stopped'})
|
||||
client.call('ack', '', reply['journal_id'], str(reply['offset']))
|
||||
# A lost acknowledgment may replay, but the archive must stay unchanged.
|
||||
self.ingest(repaired)
|
||||
self.assertEqual(self.snapshot()['text'].count('Сервис: остановка туннеля'), 1)
|
||||
client.append('Сервис: запуск\n'); client.flush()
|
||||
second = client.export()
|
||||
reply = self.ingest(second, {'enabled': True, 'xray': 'running'})
|
||||
client.call('ack', '', reply['journal_id'], str(reply['offset']))
|
||||
text = self.snapshot()['text']
|
||||
self.assertIn('Сервис: запуск', text)
|
||||
self.assertNotIn('не доставлена до ротации', text)
|
||||
self.assertEqual(client.export()['data'], '')
|
||||
|
||||
def test_disk_mount_diagnostic_resolves_deepest_mount_and_escaped_paths(self):
|
||||
path = self.app.db_path.resolve()
|
||||
mount = str(path.parent).replace(' ', r'\040')
|
||||
root = str(Path(path.anchor)).replace(' ', r'\040')
|
||||
mountinfo = f'1 0 8:1 / {root} rw - ext4 /dev/vda1 rw\n'
|
||||
mountinfo += f'2 1 0:1 / {mount} rw - tmpfs tmpfs rw\n'
|
||||
with mock.patch.object(hub.sys, 'platform', 'linux'), mock.patch('builtins.open', mock.mock_open(read_data=mountinfo)):
|
||||
result = hub.router_log_storage(path)
|
||||
self.assertIs(result['persistent'], False)
|
||||
self.assertEqual(result['filesystem'], 'tmpfs')
|
||||
with mock.patch.object(hub.sys, 'platform', 'linux'), mock.patch('builtins.open', mock.mock_open(read_data=mountinfo.splitlines()[0])):
|
||||
self.assertIs(hub.router_log_storage(path)['persistent'], True)
|
||||
|
||||
def test_offline_clear_does_not_restore_old_logs_on_reconnection(self):
|
||||
old = b'[ERROR] old\n'
|
||||
self.ingest(batch(old))
|
||||
|
||||
@@ -35,6 +35,7 @@ const server = http.createServer((req, res) => {
|
||||
reads++;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
return res.end(JSON.stringify({ok:true, online:false, text, bytes:Buffer.byteLength(text), last_sync:1791954000,
|
||||
storage:{path:'/var/lib/owrt-remote/hub.db',filesystem:'ext4',persistent:true},
|
||||
diagnosis:{level:'warn',text:'Remote выключен на роутере. Туннель и Heartbeat остановлены.',at:1791954000}}));
|
||||
}
|
||||
if (url.pathname === '/api/router/main/logs/clear') {
|
||||
@@ -95,6 +96,8 @@ const server = http.createServer((req, res) => {
|
||||
text=initialText();const before=reads;
|
||||
await page.goto(`http://127.0.0.1:${server.address().port}/router/main/logs`);
|
||||
await page.waitForFunction(()=>document.querySelector('#logsText').textContent.includes('Event 119'));
|
||||
assert((await page.locator('#logsStorage').textContent()).includes('Архив VPS: постоянный диск.'));
|
||||
assert((await page.locator('#logsStorage').textContent()).includes('/var/lib/owrt-remote/hub.db'));
|
||||
assert.deepEqual(await page.locator('body').evaluate(el=>{
|
||||
const s=getComputedStyle(el);return {image:s.backgroundImage,color:s.backgroundColor,attachment:s.backgroundAttachment};
|
||||
}),terminalBackground,'log page uses the VPS terminal background');
|
||||
|
||||
@@ -23,6 +23,8 @@ class RouterLogTransportTests(unittest.TestCase):
|
||||
self.env = {**self.journal.env, 'OWRT_REMOTE_WAN_HELPER': (ROOT / 'files/usr/lib/owrt-remote/wan.sh').as_posix(),
|
||||
'TEST_ROOT': self.journal.root.as_posix(), 'TEST_PYTHON': Path(sys.executable).as_posix()}
|
||||
self.mocks = r'''
|
||||
# Windows has no RLIMIT_FSIZE; production ash enforces the actual file limit.
|
||||
ulimit() { return 0; }
|
||||
uci_get() {
|
||||
case "$1" in
|
||||
enabled) printf '%s' "${TEST_ENABLED:-1}" ;; hub_token) printf test-token ;; router_id) printf main ;;
|
||||
@@ -60,6 +62,7 @@ jsonfilter() {
|
||||
'''
|
||||
|
||||
def sync(self, **flags):
|
||||
self.journal.helper_call('flush', '', 'force')
|
||||
script = self.source + '\n' + self.mocks + '\nsync_global_logs\n'
|
||||
return subprocess.run([SH], input=script, encoding='utf-8', capture_output=True,
|
||||
env={**self.env, **flags}, timeout=25)
|
||||
@@ -80,14 +83,34 @@ jsonfilter() {
|
||||
self.assertEqual('direct' in requests, direct == '1')
|
||||
self.assertEqual('prepare' in requests, direct == '1')
|
||||
self.assertEqual(json.loads(self.journal.helper_call('export'))['data'], '')
|
||||
self.assertFalse(list(self.journal.log_dir.glob('sync.*')))
|
||||
self.assertFalse(list(self.journal.run_dir.glob('sync.*')))
|
||||
|
||||
def test_failed_delivery_preserves_unsent_records_and_removes_payload_files(self):
|
||||
before = json.loads(self.journal.helper_call('export'))
|
||||
result = self.sync(TEST_HTTP_FAIL='1')
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertEqual(json.loads(self.journal.helper_call('export')), before)
|
||||
self.assertFalse(list(self.journal.log_dir.glob('sync.*')))
|
||||
self.assertFalse(list(self.journal.run_dir.glob('sync.*')))
|
||||
|
||||
def test_agent_update_delivers_with_older_helper_without_its_own_runtime_directory(self):
|
||||
# The pre-persistent helper creates only /tmp/owrt-remote-logs. A WAN
|
||||
# agent update must not depend on that helper creating the new runtime.
|
||||
helper = self.journal.root / 'old-helper'
|
||||
record = base64.b64encode(b'[OK] older helper record\n').decode()
|
||||
batch = json.dumps({'journal_id': 'older', 'from': 0, 'to': 25, 'data': record})
|
||||
helper.write_text('#!/bin/sh\ncase "$1" in\ncheck) exit 0 ;;\nexport) printf \'%s\\n\' \'' + batch +
|
||||
'\' ;;\nack) printf \'%s\\n\' "$2 $3" >"$TEST_ROOT/older-ack" ;;\nesac\n',
|
||||
encoding='utf-8', newline='\n')
|
||||
runtime = self.journal.root / 'new-runtime'
|
||||
result = subprocess.run([SH], input=self.source + '\n' + self.mocks + '\nsync_global_logs\n',
|
||||
encoding='utf-8', capture_output=True,
|
||||
env={**self.env, 'OWRT_REMOTE_LOG_HELPER': helper.as_posix(),
|
||||
'OWRT_REMOTE_LOG_RUNTIME': runtime.as_posix()}, timeout=25)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertTrue(runtime.is_dir())
|
||||
self.assertTrue((self.journal.root / 'older-ack').is_file())
|
||||
self.assertFalse((runtime / 'sync-lock').exists())
|
||||
self.assertFalse(list(runtime.glob('sync.*')))
|
||||
|
||||
def test_disabled_remote_still_delivers_stop_reason_and_state_without_restart(self):
|
||||
self.journal.helper_call('append', 'Сервис: остановка туннеля\n')
|
||||
@@ -97,14 +120,14 @@ jsonfilter() {
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
payload = json.loads((self.journal.root / 'delivered.json').read_text())
|
||||
self.assertEqual(payload['state'], {'enabled': False, 'xray': 'stopped'})
|
||||
self.assertFalse(list(self.journal.log_dir.glob('sync.*')))
|
||||
self.assertFalse(list(self.journal.run_dir.glob('sync.*')))
|
||||
self.assertEqual(json.loads(self.journal.helper_call('export'))['data'], '')
|
||||
|
||||
def test_overlapping_delivery_does_not_start_second_sender(self):
|
||||
lock = self.journal.log_dir / 'sync-lock'
|
||||
lock = self.journal.run_dir / 'sync-lock'
|
||||
lock.mkdir()
|
||||
# The shell's own PID is alive for the duration of sync_global_logs.
|
||||
script = self.source + '\n' + self.mocks + '\nprintf "%s" "$$" >"$OWRT_REMOTE_LOG_DIR/sync-lock/pid"\nsync_global_logs\n'
|
||||
script = self.source + '\n' + self.mocks + '\nprintf "%s" "$$" >"$OWRT_REMOTE_LOG_RUNTIME/sync-lock/pid"\nsync_global_logs\n'
|
||||
result = subprocess.run([SH], input=script, encoding='utf-8', capture_output=True, env=self.env, timeout=20)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse((self.journal.root / 'requests').exists())
|
||||
@@ -113,15 +136,25 @@ jsonfilter() {
|
||||
def test_wan_failure_releases_delivery_lock_for_next_attempt(self):
|
||||
result = self.sync(TEST_DIRECT='1', TEST_WAN_FAIL='1', TEST_ENABLED='0')
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertFalse((self.journal.log_dir / 'sync-lock').exists())
|
||||
self.assertFalse((self.journal.run_dir / 'sync-lock').exists())
|
||||
self.assertNotEqual(json.loads(self.journal.helper_call('export'))['data'], '')
|
||||
self.assertEqual(self.sync(TEST_DIRECT='1', TEST_ENABLED='0').returncode, 0)
|
||||
|
||||
def test_abandoned_empty_delivery_lock_is_recovered(self):
|
||||
(self.journal.log_dir / 'sync-lock').mkdir()
|
||||
(self.journal.run_dir / 'sync-lock').mkdir()
|
||||
result = self.sync(TEST_ENABLED='0')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse((self.journal.log_dir / 'sync-lock').exists())
|
||||
self.assertFalse((self.journal.run_dir / 'sync-lock').exists())
|
||||
self.assertTrue((self.journal.root / 'delivered.json').exists())
|
||||
self.assertFalse(list(self.journal.run_dir.glob('http/owrt-remote-http.*')))
|
||||
|
||||
def test_abandoned_delivery_lock_reaper_is_cleaned_for_next_attempt(self):
|
||||
lock = self.journal.run_dir / 'sync-lock'
|
||||
lock.mkdir(); (lock / 'pid').write_text('99999999')
|
||||
(lock / 'reap').mkdir(); (lock / 'reap/pid').write_text('99999998')
|
||||
self.assertEqual(self.sync().returncode, 0)
|
||||
self.assertEqual(self.sync().returncode, 0)
|
||||
self.assertFalse(lock.exists())
|
||||
self.assertTrue((self.journal.root / 'delivered.json').exists())
|
||||
|
||||
def test_panel_stop_sends_completed_result_without_waiting_for_restart(self):
|
||||
|
||||
@@ -35,7 +35,7 @@ class WanDnsDeploymentTests(unittest.TestCase):
|
||||
source = (ROOT / "vps/deploy-wan-dns-fix.ps1").read_text(encoding="utf-8")
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace("target=/usr/lib/owrt-remote/wan.sh", "target=" + shlex.quote(self.target.as_posix()))
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-wan-dns-v116",
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-wan-dns-v117",
|
||||
"backup_dir=" + shlex.quote(self.backup.as_posix()))
|
||||
|
||||
def deploy(self, action="Install", bad=False, fail_verify=False, fail_signal=False):
|
||||
|
||||
@@ -129,7 +129,7 @@ class WanFailoverTests(unittest.TestCase):
|
||||
self.prepare()
|
||||
config = self.directory / "xray.json"
|
||||
config.write_text(json.dumps({"outbounds": [{"tag": "vps-interconn", "streamSettings": {"sockopt": {"interface": "eth0"}}}]}))
|
||||
self.config(wan_interface="wan wwan", wan_direct="1", vps_host="192.0.2.10", xray_config=config.as_posix())
|
||||
self.config(enabled="1", wan_interface="wan wwan", wan_direct="1", vps_host="192.0.2.10", xray_config=config.as_posix())
|
||||
result = self.shell('''
|
||||
xray_running() { return 0; }
|
||||
wan_reconnect_xray() { printf '%s -> %s\n' "$1" "$2" >>"$TEST_ROOT/reconnect"; }
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
"""WAN outage/recovery using the real agent shell and a simulated procd."""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
import test_host_policy as base
|
||||
from test_wan_failover import UPLINK_MOCKS, JSONFILTER
|
||||
|
||||
|
||||
@unittest.skipUnless(base.SH, "POSIX shell required")
|
||||
class WanRecoveryTests(unittest.TestCase):
|
||||
setUp = base.RouterShellTests.setUp
|
||||
config = base.RouterShellTests.config
|
||||
|
||||
def shell(self, commands):
|
||||
shell = os.environ.get("OWRT_RECOVERY_TEST_SHELL", base.SH)
|
||||
args = [shell, "ash"] if "busybox" in shell.lower() else [shell]
|
||||
return subprocess.run(args, input=self.agent + "\n" + base.MOCKS + "\n" + UPLINK_MOCKS + "\n" + commands,
|
||||
encoding="utf-8", capture_output=True, cwd=base.ROOT, env=self.env, timeout=40)
|
||||
|
||||
def prepare(self, enabled="1"):
|
||||
self.config(enabled=enabled, wan_direct="1", wan_interface="wan", vps_host="192.0.2.10",
|
||||
hub_url="https://hub.example", hub_token="secret-test-token")
|
||||
self.directory.joinpath("jsonfilter.py").write_text(JSONFILTER, encoding="utf-8")
|
||||
self.directory.joinpath("service.json").write_text(json.dumps({"owrt-remote": {"instances": {
|
||||
"heartbeat": {"running": True, "pid": 123}, "xray": {"running": False}
|
||||
}}}), encoding="utf-8")
|
||||
|
||||
PROCD = r'''
|
||||
xray_running() { return 1; }
|
||||
ubus() {
|
||||
case "$1 $2 $3" in
|
||||
'call service list') cat "$TEST_ROOT/service.json" ;;
|
||||
'call service add')
|
||||
printf '%s\n' "$4" >>"$TEST_ROOT/add-calls"
|
||||
[ ! -e "$TEST_ROOT/add-fail" ] ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
jsonfilter() {
|
||||
"$TEST_PYTHON" -c 'import json,sys
|
||||
state=json.load(sys.stdin)
|
||||
instance="heartbeat" if ".heartbeat." in sys.argv[1] else "xray"
|
||||
value=state.get("owrt-remote",{}).get("instances",{}).get(instance,{}).get("running","")
|
||||
print(str(value).lower())' "$2"
|
||||
}
|
||||
wan_select_interface() { [ ! -e "$TEST_ROOT/wan-down" ] || return 1; printf wan; }
|
||||
wan_device() { [ ! -e "$TEST_ROOT/wan-down" ] || return 1; printf eth0; }
|
||||
'''
|
||||
|
||||
def test_online_heartbeat_after_outage_recovers_dead_tunnel_on_same_uplink_once(self):
|
||||
self.prepare()
|
||||
result = self.shell(self.PROCD + r'''
|
||||
touch "$TEST_ROOT/wan-down"
|
||||
wan_reconcile_xray || true
|
||||
test ! -e "$TEST_ROOT/add-calls" || exit 1
|
||||
rm "$TEST_ROOT/wan-down"
|
||||
wan_reconcile_xray || exit
|
||||
wan_reconcile_xray
|
||||
''')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
requests = self.directory.joinpath("add-calls").read_text().splitlines()
|
||||
self.assertEqual(len(requests), 1)
|
||||
request = json.loads(requests[0])
|
||||
self.assertEqual(set(request["instances"]), {"xray"})
|
||||
self.assertEqual(request["instances"]["xray"]["command"], ["/usr/sbin/owrt-remote", "run"])
|
||||
self.assertEqual(request["instances"]["xray"]["respawn"], ["3600", "5", "5"])
|
||||
self.assertFalse(self.directory.joinpath("owrt-remote-xray-recovery").exists())
|
||||
self.assertNotIn("secret-test-token", result.stderr)
|
||||
|
||||
def test_initial_start_waiting_for_wan_is_left_to_procd(self):
|
||||
self.prepare()
|
||||
state = self.directory / "service.json"
|
||||
data = json.loads(state.read_text())
|
||||
data["owrt-remote"]["instances"]["xray"]["running"] = True
|
||||
state.write_text(json.dumps(data))
|
||||
result = self.shell(self.PROCD + "wan_reconcile_xray")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("add-calls").exists())
|
||||
|
||||
def test_successful_heartbeat_recovers_dead_xray_and_still_returns_hub_response(self):
|
||||
self.prepare()
|
||||
result = self.shell(self.PROCD + r'''
|
||||
wan_prepare() { return 0; }
|
||||
heartbeat_json() { printf '{}'; }
|
||||
heartbeat_candidate_endpoints() { printf 'https://hub.example/api/heartbeat'; }
|
||||
post_json() { printf '{"ok":true}'; }
|
||||
heartbeat_apply_hub_update() { return 0; }
|
||||
send_heartbeat
|
||||
''')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(json.loads(result.stdout), {"ok": True})
|
||||
request = json.loads(self.directory.joinpath("add-calls").read_text())
|
||||
self.assertEqual(set(request["instances"]), {"xray"})
|
||||
self.assertIn("Heartbeat: OK", result.stderr)
|
||||
|
||||
def test_healthy_xray_is_not_restarted_when_wan_returns(self):
|
||||
self.prepare()
|
||||
result = self.shell(self.PROCD + "xray_running() { return 0; }; wan_reconcile_xray")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("add-calls").exists())
|
||||
|
||||
def test_panel_stop_and_external_service_stop_are_respected(self):
|
||||
self.prepare(enabled="0")
|
||||
result = self.shell(self.PROCD + "wan_reconcile_xray")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("add-calls").exists())
|
||||
self.prepare()
|
||||
self.directory.joinpath("service.json").write_text('{}')
|
||||
result = self.shell(self.PROCD + "wan_reconcile_xray")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("add-calls").exists())
|
||||
|
||||
def test_parallel_requests_and_stale_lock_produce_one_recovery(self):
|
||||
self.prepare()
|
||||
lock = self.directory / "owrt-remote-xray-recovery"
|
||||
lock.mkdir()
|
||||
lock.joinpath("pid").write_text("2147483000")
|
||||
result = self.shell(self.PROCD + "wan_reconcile_xray & wan_reconcile_xray & wait")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(len(self.directory.joinpath("add-calls").read_text().splitlines()), 1)
|
||||
self.assertFalse(lock.exists())
|
||||
|
||||
def test_ubus_failure_reports_error_cleans_lock_and_honors_backoff(self):
|
||||
self.prepare()
|
||||
self.directory.joinpath("add-fail").touch()
|
||||
result = self.shell(self.PROCD + "wan_reconcile_xray; rc=$?; wan_reconcile_xray; exit $rc")
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertIn("procd", result.stderr)
|
||||
self.assertEqual(len(self.directory.joinpath("add-calls").read_text().splitlines()), 1)
|
||||
self.assertFalse(self.directory.joinpath("owrt-remote-xray-recovery").exists())
|
||||
|
||||
def test_empty_lock_and_dead_reaper_do_not_block_later_recovery(self):
|
||||
self.prepare()
|
||||
lock = self.directory / "owrt-remote-xray-recovery"
|
||||
lock.mkdir()
|
||||
(lock / "reap").mkdir()
|
||||
(lock / "reap/pid").write_text("2147483000")
|
||||
result = self.shell(self.PROCD + "wan_reconcile_xray")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(len(self.directory.joinpath("add-calls").read_text().splitlines()), 1)
|
||||
self.assertFalse(lock.exists())
|
||||
|
||||
def test_missing_wan_does_not_cause_repeated_service_restarts(self):
|
||||
self.prepare()
|
||||
self.directory.joinpath("wan-down").touch()
|
||||
result = self.shell(self.PROCD + r'''
|
||||
active_ssh_session() { return 1; }
|
||||
heartbeat_self_heal; heartbeat_self_heal; heartbeat_self_heal; heartbeat_self_heal
|
||||
''')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("owrt-remote-heartbeat.restart").exists())
|
||||
self.assertIn("IPv4 default route", result.stderr)
|
||||
|
||||
def test_supervised_start_waits_beyond_six_retries_then_launches_with_valid_config(self):
|
||||
self.prepare()
|
||||
self.directory.joinpath("wan-down").touch()
|
||||
executable = self.directory / "fake-xray"
|
||||
executable.write_text('#!/bin/sh\n[ "$1" != help ] || { echo run; exit; }; printf "%s\\n" "$*" >"$TEST_ROOT/xray-called"\n', newline="\n")
|
||||
executable.chmod(0o755)
|
||||
config = self.directory / "valid-config.json"
|
||||
config.write_text('{"outbounds":[]}')
|
||||
result = self.shell(r'''
|
||||
sleep() {
|
||||
local n
|
||||
n=$(cat "$TEST_ROOT/waits" 2>/dev/null || echo 0)
|
||||
n=$((n+1)); printf '%s' "$n" >"$TEST_ROOT/waits"
|
||||
[ "$n" -lt 8 ] || rm -f "$TEST_ROOT/wan-down"
|
||||
}
|
||||
ensure_xray_available_with_retry() { return 0; }
|
||||
xray_bin_path() { printf '%s' "$TEST_ROOT/fake-xray"; }
|
||||
render_client_config() { printf '%s' "$TEST_ROOT/valid-config.json"; }
|
||||
run_xray
|
||||
''')
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(self.directory.joinpath("waits").read_text(), "8")
|
||||
self.assertEqual(self.directory.joinpath("xray-called").read_text().strip(), 'run -config ' + config.as_posix())
|
||||
self.assertEqual(result.stderr.count("конфигурация сохранена"), 1)
|
||||
self.assertIn("продолжается запуск", result.stderr)
|
||||
|
||||
def test_disabling_service_cancels_wait_without_rewriting_configuration(self):
|
||||
self.prepare()
|
||||
self.directory.joinpath("wan-down").touch()
|
||||
result = self.shell(r'''
|
||||
sleep() { uci set owrtremote.main.enabled=0; }
|
||||
render_client_config() { touch "$TEST_ROOT/rendered"; }
|
||||
run_xray
|
||||
''')
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("rendered").exists())
|
||||
|
||||
def test_configuration_error_with_available_wan_is_not_retried_as_wan_outage(self):
|
||||
self.prepare()
|
||||
result = self.shell(r'''
|
||||
sleep() { touch "$TEST_ROOT/slept"; }
|
||||
ensure_xray_available_with_retry() { return 0; }
|
||||
xray_bin_path() { printf /usr/bin/xray; }
|
||||
render_client_config() { printf 'invalid configuration\n' >&2; return 7; }
|
||||
run_xray
|
||||
''')
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertIn("invalid configuration", result.stderr)
|
||||
self.assertFalse(self.directory.joinpath("slept").exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,122 @@
|
||||
"""Run installation and rollback in a sandbox, without SSH or services."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shlex
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from test_xray_mobile_deploy import SH, GIT_BIN, ROOT
|
||||
|
||||
|
||||
@unittest.skipUnless(SH, "POSIX shell required")
|
||||
class WanRecoveryDeploymentTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix="wan-recovery-deploy-")
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.targets = {"wan.sh": self.root / "wan.sh", "agent": self.root / "agent"}
|
||||
for name, path in self.targets.items():
|
||||
path.write_text("#!/bin/sh\necho previous-" + name + "\n", encoding="utf-8", newline="\n")
|
||||
self.original = {name: path.read_bytes() for name, path in self.targets.items()}
|
||||
self.backup = self.root / "backup"
|
||||
self.env = dict(os.environ)
|
||||
if os.name == "nt":
|
||||
self.env["PATH"] = str(GIT_BIN) + os.pathsep + self.env["PATH"]
|
||||
source = (ROOT / "vps/deploy-wan-recovery-fix.ps1").read_text(encoding="utf-8")
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-wan-recovery-v117",
|
||||
"backup_dir=" + shlex.quote(self.backup.as_posix()))
|
||||
self.script = self.script.replace("printf /usr/lib/owrt-remote/wan.sh", "printf %s " + shlex.quote(self.targets["wan.sh"].as_posix()))
|
||||
self.script = self.script.replace("printf /usr/sbin/owrt-remote", "printf %s " + shlex.quote(self.targets["agent"].as_posix()))
|
||||
|
||||
def deploy(self, action, bad_item=None, fail_verify=False, fail_signal=False, stopped=False):
|
||||
stage = self.root / "stage"
|
||||
stage.mkdir()
|
||||
for item, source in {"wan.sh": "files/usr/lib/owrt-remote/wan.sh", "agent": "files/usr/sbin/owrt-remote"}.items():
|
||||
text = (ROOT / source).read_text(encoding="utf-8")
|
||||
if bad_item == item:
|
||||
text = "#!/bin/sh\nif true; then\n"
|
||||
stage.joinpath(item).write_text(text, encoding="utf-8", newline="\n")
|
||||
script = self.script.replace("stage=/tmp/__UPLOAD_NAME__", "stage=" + shlex.quote(stage.as_posix()))
|
||||
script = script.replace("__ACTION__", action).replace("__UPLOAD_NAME__", "sandbox")
|
||||
if fail_verify:
|
||||
script = script.replace('cmp "$source_dir/$item" "$path"', "false")
|
||||
mocks = r'''
|
||||
|
||||
ubus() {
|
||||
printf '%s\n' "$*" >>"$TEST_ROOT/ubus-calls"
|
||||
if [ "$3" = signal ]; then
|
||||
[ "${TEST_SIGNAL_FAIL:-0}" = 0 ] || return 1
|
||||
printf 202 >"$TEST_ROOT/pid"
|
||||
fi
|
||||
printf '{}'
|
||||
}
|
||||
jsonfilter() { cat >/dev/null; cat "$TEST_ROOT/pid"; }
|
||||
sleep() { :; }
|
||||
'''
|
||||
self.root.joinpath("pid").write_text("" if stopped else "101")
|
||||
result = subprocess.run([SH], input=mocks + script, encoding="utf-8", capture_output=True,
|
||||
env={**self.env, "TEST_ROOT": self.root.as_posix(), "TEST_SIGNAL_FAIL": "1" if fail_signal else "0"}, timeout=30)
|
||||
self.assertFalse(stage.exists())
|
||||
self.assertFalse(list(self.root.glob("*.new-sandbox")))
|
||||
return result
|
||||
|
||||
def test_install_twice_then_rollback_preserves_first_backup(self):
|
||||
for attempt in range(2):
|
||||
result = self.deploy("Install")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("WAN_RECOVERY_ROUTER_Install_OK", result.stdout)
|
||||
for item, path in self.targets.items():
|
||||
self.assertNotEqual(path.read_bytes(), self.original[item])
|
||||
self.assertEqual(self.backup.joinpath(item).read_bytes(), self.original[item])
|
||||
result = self.deploy("Rollback")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
for item, path in self.targets.items():
|
||||
self.assertEqual(path.read_bytes(), self.original[item])
|
||||
|
||||
def test_bad_candidate_and_missing_backup_leave_paired_files_unchanged(self):
|
||||
for item in ("agent", "wan.sh"):
|
||||
result = self.deploy("Install", bad_item=item)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertFalse(self.backup.exists())
|
||||
for name, path in self.targets.items():
|
||||
self.assertEqual(path.read_bytes(), self.original[name])
|
||||
result = self.deploy("Rollback")
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn("No complete backup", result.stderr)
|
||||
for name, path in self.targets.items():
|
||||
self.assertEqual(path.read_bytes(), self.original[name])
|
||||
|
||||
def test_reloads_only_heartbeat_without_touching_xray(self):
|
||||
result = self.deploy("Install")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
calls = self.root.joinpath("ubus-calls").read_text()
|
||||
self.assertIn('service signal {"name":"owrt-remote","instance":"heartbeat","signal":9}', calls)
|
||||
self.assertNotIn('"instance":"xray"', calls)
|
||||
self.assertIn('Heartbeat loop reloaded', result.stdout)
|
||||
|
||||
def test_stopped_loop_stays_stopped(self):
|
||||
result = self.deploy("Install", stopped=True)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertNotIn('service signal', self.root.joinpath("ubus-calls").read_text())
|
||||
self.assertIn('Heartbeat loop is stopped', result.stdout)
|
||||
|
||||
def test_failed_loop_reload_restores_paired_files_and_reports_failure(self):
|
||||
result = self.deploy("Install", fail_signal=True)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('Failed during heartbeat-reload', result.stderr)
|
||||
self.assertNotIn('WAN_RECOVERY_ROUTER_Install_OK', result.stdout)
|
||||
for item, path in self.targets.items():
|
||||
self.assertEqual(path.read_bytes(), self.original[item])
|
||||
|
||||
def test_verify_failure_restores_paired_files(self):
|
||||
result = self.deploy("Install", fail_verify=True)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn("Restoring the agent and WAN helper", result.stderr)
|
||||
for item, path in self.targets.items():
|
||||
self.assertEqual(path.read_bytes(), self.original[item])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -31,7 +31,7 @@ class WanStabilityRollbackTests(unittest.TestCase):
|
||||
source = (ROOT / "vps/deploy-wan-stability-fix.ps1").read_text(encoding="utf-8")
|
||||
script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
script = script.replace('stage=/tmp/__UPLOAD_NAME__', 'stage=' + shlex.quote(self.stage.as_posix()))
|
||||
script = script.replace('backup=/root/owrt-remote-backup-before-wan-stability-v116', 'backup=' + shlex.quote(self.backup.as_posix()))
|
||||
script = script.replace('backup=/root/owrt-remote-backup-before-wan-stability-v117', 'backup=' + shlex.quote(self.backup.as_posix()))
|
||||
script = script.replace('config=/etc/config/owrtremote', 'config=' + shlex.quote(self.files["config"].as_posix()))
|
||||
for name, original in (("agent", "/usr/sbin/owrt-remote"), ("wan.sh", "/usr/lib/owrt-remote/wan.sh")):
|
||||
script = script.replace('printf ' + original + ' ;;', "printf '%s' " + shlex.quote(self.files[name].as_posix()) + ' ;;')
|
||||
|
||||
@@ -29,7 +29,7 @@ class XrayMobileDeploymentTests(unittest.TestCase):
|
||||
source = (ROOT / "vps/deploy-xray-mobile-fix.ps1").read_text(encoding="utf-8")
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace("target=/www/cgi-bin/owrt-remote", "target=" + shlex.quote(self.target.as_posix()))
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-xray-mobile-v116",
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-xray-mobile-v117",
|
||||
"backup_dir=" + shlex.quote(self.backup_dir.as_posix()))
|
||||
|
||||
def deploy(self, action, content=None, fail_verify=False):
|
||||
|
||||
@@ -25,7 +25,7 @@ class XrayValidationDeploymentTests(unittest.TestCase):
|
||||
self.env["PATH"] = str(GIT_BIN) + os.pathsep + self.env["PATH"]
|
||||
source = (ROOT / "vps/deploy-xray-validation-fix.ps1").read_text(encoding="utf-8")
|
||||
self.script = source.split("$RemoteScript = @'\n", 1)[1].split("\n'@", 1)[0]
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-xray-validation-v116",
|
||||
self.script = self.script.replace("backup_dir=/root/owrt-remote-backup-before-xray-validation-v117",
|
||||
"backup_dir=" + shlex.quote(self.backup.as_posix()))
|
||||
self.script = self.script.replace("printf /www/cgi-bin/owrt-remote", "printf %s " + shlex.quote(self.targets["cgi"].as_posix()))
|
||||
self.script = self.script.replace("printf /usr/sbin/owrt-remote", "printf %s " + shlex.quote(self.targets["agent"].as_posix()))
|
||||
|
||||
@@ -15,7 +15,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-global-logs-ui-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-global-logs-ui-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -107,7 +107,7 @@ if [ "$healthy" -ne 1 ]; then
|
||||
exit 1
|
||||
fi
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nGLOBAL_LOGS_VPS_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nGLOBAL_LOGS_VPS_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
|
||||
@@ -13,7 +13,7 @@ $RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup=/root/owrt-remote-backup-before-global-logs-stop-v116
|
||||
backup=/root/owrt-remote-backup-before-global-logs-stop-v117
|
||||
items='cgi agent init logs.sh logs-init luci'
|
||||
changed=0
|
||||
phase=preflight
|
||||
@@ -104,7 +104,7 @@ if [ '__ACTION__' = Install ]; then
|
||||
command -v base64 >/dev/null
|
||||
command -v jsonfilter >/dev/null
|
||||
grep -F -q 'globalLogsToggle' "$stage/cgi"
|
||||
for item in cgi luci; do grep -F -q 'v116' "$stage/$item"; done
|
||||
for item in cgi luci; do grep -F -q 'v117' "$stage/$item"; done
|
||||
grep -F -q 'LOG_KEEP_SECONDS=86400' "$stage/logs.sh"
|
||||
grep -F -q 'log_sync_loop' "$stage/logs.sh"
|
||||
grep -F -q 'sync_global_logs' "$stage/agent"
|
||||
@@ -131,7 +131,8 @@ if [ '__ACTION__' = Install ]; then
|
||||
sleep 1
|
||||
done
|
||||
if [ "$ready" != 1 ]; then echo 'Log collector did not start.' >&2; logread -e owrt-remote | tail -n 30 >&2; exit 1; fi
|
||||
marker='Глобал логи: collector-__UPLOAD_NAME__'
|
||||
# Generate UTF-8 on the router; Windows PowerShell 5.1 reads scripts as ANSI.
|
||||
marker="$(printf '\320\223\320\273\320\276\320\261\320\260\320\273\040\320\273\320\276\320\263\320\270\072') collector-__UPLOAD_NAME__"
|
||||
ready=0
|
||||
for attempt in 1 2 3 4 5; do
|
||||
logger -t owrt-remote "$marker"
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
param(
|
||||
[Parameter(Mandatory = $true)][ValidateNotNullOrEmpty()][string]$Vps,
|
||||
[ValidateSet('Install', 'Rollback')][string]$Action = 'Install',
|
||||
[switch]$PrepareOnly
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
$UploadName = 'owrt-group-colors-v117-' + [guid]::NewGuid().ToString('N')
|
||||
$StageRoot = Join-Path ([System.IO.Path]::GetTempPath()) $UploadName
|
||||
$Utf8 = [System.Text.UTF8Encoding]::new($false)
|
||||
|
||||
$RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-group-colors-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
show_failure() {
|
||||
echo '--- owrt-remote status ---' >&2
|
||||
systemctl --no-pager -l status owrt-remote >&2 || true
|
||||
echo '--- owrt-remote journal ---' >&2
|
||||
journalctl -u owrt-remote -n 80 --no-pager -l >&2 || true
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
code=$?
|
||||
trap - EXIT
|
||||
if [ "$code" -ne 0 ]; then
|
||||
echo "Remote step failed with rc=$code" >&2
|
||||
show_failure
|
||||
fi
|
||||
if [ "$code" -ne 0 ] && [ "$changed" = 1 ] && [ -s "$stage/previous.py" ]; then
|
||||
echo 'Restoring the previous Hub file...' >&2
|
||||
cp -p "$stage/previous.py" "$candidate"
|
||||
mv -f "$candidate" "$target"
|
||||
systemctl restart owrt-remote || true
|
||||
fi
|
||||
rm -f "$candidate"
|
||||
rm -rf "$stage"
|
||||
exit "$code"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
test -s "$target"
|
||||
python_bin=/opt/owrt-remote/venv/bin/python
|
||||
[ -x "$python_bin" ] || python_bin=python3
|
||||
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
test -s "$stage/hub.py"
|
||||
echo 'Validating candidate...'
|
||||
"$python_bin" -m py_compile "$stage/hub.py"
|
||||
"$python_bin" - "$stage/hub.py" <<'PY'
|
||||
from pathlib import Path
|
||||
import sys
|
||||
source = Path(sys.argv[1]).read_text(encoding='utf-8')
|
||||
required = ('v117', '#routerForm select option,#routerGroupFilter option', 'background-color:#221a34', 'id="backupDownload"', 'id="backupRestore"')
|
||||
if 'class="backupCmds"' in source:
|
||||
print('Old backup command instructions remain in candidate', file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
if 'v' + '116' in source:
|
||||
print('Old UI version remains in candidate', file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
missing = [marker for marker in required if marker not in source]
|
||||
if missing:
|
||||
for marker in missing:
|
||||
print('Missing marker:', repr(marker), file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
print('Candidate validation OK')
|
||||
PY
|
||||
if [ ! -e "$backup" ]; then
|
||||
cp -p "$target" "$backup"
|
||||
fi
|
||||
source_file="$stage/hub.py"
|
||||
else
|
||||
test -s "$backup"
|
||||
"$python_bin" -m py_compile "$backup"
|
||||
source_file="$backup"
|
||||
fi
|
||||
|
||||
cp -p "$target" "$stage/previous.py"
|
||||
cp "$source_file" "$candidate"
|
||||
chmod 0755 "$candidate"
|
||||
changed=1
|
||||
mv -f "$candidate" "$target"
|
||||
echo 'Restarting owrt-remote...'
|
||||
if ! systemctl restart owrt-remote; then
|
||||
show_failure
|
||||
exit 1
|
||||
fi
|
||||
|
||||
healthy=0
|
||||
for attempt in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do
|
||||
if systemctl is-active --quiet owrt-remote && curl -fsS --max-time 5 http://127.0.0.1:8088/health; then
|
||||
healthy=1
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
if [ "$healthy" -ne 1 ]; then
|
||||
echo 'Health check did not become ready.' >&2
|
||||
show_failure
|
||||
exit 1
|
||||
fi
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nGROUP_COLORS_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
|
||||
try {
|
||||
[System.IO.Directory]::CreateDirectory($StageRoot) | Out-Null
|
||||
[System.IO.File]::WriteAllText(
|
||||
(Join-Path $StageRoot 'run.sh'),
|
||||
$RemoteScript.Replace("`r`n", "`n") + "`n",
|
||||
$Utf8
|
||||
)
|
||||
|
||||
if ($Action -eq 'Install') {
|
||||
$Content = [System.IO.File]::ReadAllText((Join-Path $RepoRoot 'vps\owrt-remote-hub.py')).Replace("`r`n", "`n")
|
||||
[System.IO.File]::WriteAllText((Join-Path $StageRoot 'hub.py'), $Content, $Utf8)
|
||||
}
|
||||
|
||||
if ($PrepareOnly) {
|
||||
Write-Host "Prepared only: $StageRoot"
|
||||
return
|
||||
}
|
||||
|
||||
& scp -O -r -- $StageRoot "${Vps}:/tmp/"
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Upload failed. VPS was not changed.' }
|
||||
& ssh -T -o ConnectTimeout=10 $Vps "sh /tmp/$UploadName/run.sh"
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Action failed. See the output above; success was not confirmed." }
|
||||
Write-Host "Group colors fix $Action finished successfully. Refresh the Hub panel with Ctrl+F5."
|
||||
}
|
||||
finally {
|
||||
if (-not $PrepareOnly) {
|
||||
$Resolved = [System.IO.Path]::GetFullPath($StageRoot)
|
||||
$Expected = [System.IO.Path]::GetFullPath((Join-Path ([System.IO.Path]::GetTempPath()) $UploadName))
|
||||
if ($Resolved -ne $Expected -or (Split-Path -Leaf $Resolved) -ne $UploadName) {
|
||||
throw 'Unsafe staging cleanup path.'
|
||||
}
|
||||
if (Test-Path -LiteralPath $Resolved) {
|
||||
Remove-Item -LiteralPath $Resolved -Recurse -Force
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,7 @@ $RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup=/root/owrt-remote-backup-before-heartbeat-redirect-v116
|
||||
backup=/root/owrt-remote-backup-before-heartbeat-redirect-v117
|
||||
agent=/usr/sbin/owrt-remote
|
||||
wan=/usr/lib/owrt-remote/wan.sh
|
||||
changed=0
|
||||
|
||||
@@ -17,7 +17,7 @@ $RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup_dir=/root/owrt-remote-backup-before-heartbeat-transport-v116
|
||||
backup_dir=/root/owrt-remote-backup-before-heartbeat-transport-v117
|
||||
changed=0
|
||||
phase=preflight
|
||||
item_path() {
|
||||
|
||||
@@ -5,7 +5,7 @@ param(
|
||||
[string]$PinnedVpsHost = '193.233.82.38',
|
||||
[ValidatePattern('^[a-zA-Z0-9_-]+$')][string]$WanInterface = 'wan',
|
||||
[switch]$PreserveConnectionSettings,
|
||||
[ValidatePattern('^[a-zA-Z0-9_-]+$')][string]$BackupName = 'host-policy-v116',
|
||||
[ValidatePattern('^[a-zA-Z0-9_-]+$')][string]$BackupName = 'host-policy-v117',
|
||||
[switch]$PrepareOnly
|
||||
)
|
||||
|
||||
@@ -90,7 +90,7 @@ test -s "$config"
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
for item in agent wan.sh cgi init; do test -s "$stage/$item"; sh -n "$stage/$item"; done
|
||||
test -s "$stage/luci"
|
||||
for item in cgi luci; do grep -F -q 'v116' "$stage/$item"; done
|
||||
for item in cgi luci; do grep -F -q 'v117' "$stage/$item"; done
|
||||
# curl is needed to preserve HTTPS SNI and bypass the local DNS/proxy.
|
||||
if ! command -v curl >/dev/null 2>&1; then
|
||||
if command -v opkg >/dev/null 2>&1; then opkg update && opkg install curl ca-bundle
|
||||
@@ -156,16 +156,16 @@ if [ '__ACTION__' = Install ]; then
|
||||
exit 1
|
||||
fi
|
||||
/usr/sbin/owrt-remote status
|
||||
grep -F -q 'v116' /www/cgi-bin/owrt-remote
|
||||
grep -F -q 'v116' /www/luci-static/resources/view/owrt_remote.js
|
||||
echo HOST_POLICY_ROUTER_V116_OK
|
||||
grep -F -q 'v117' /www/cgi-bin/owrt-remote
|
||||
grep -F -q 'v117' /www/luci-static/resources/view/owrt_remote.js
|
||||
echo HOST_POLICY_ROUTER_V117_OK
|
||||
else
|
||||
test -f "$backup/complete"
|
||||
save_files "$stage/previous"
|
||||
changed=1
|
||||
restore_files "$backup"
|
||||
/usr/sbin/owrt-remote status
|
||||
echo HOST_POLICY_ROUTER_V116_ROLLBACK_OK
|
||||
echo HOST_POLICY_ROUTER_V117_ROLLBACK_OK
|
||||
fi
|
||||
'@
|
||||
} else {
|
||||
@@ -180,7 +180,7 @@ fi
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-host-policy-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-host-policy-v117
|
||||
changed=0
|
||||
cleanup() {
|
||||
code=$?
|
||||
@@ -200,7 +200,7 @@ test -s "$target"
|
||||
python_bin=/opt/owrt-remote/venv/bin/python
|
||||
[ -x "$python_bin" ] || python_bin=python3
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
grep -F -q 'v116' "$stage/hub.py"
|
||||
grep -F -q 'v117' "$stage/hub.py"
|
||||
"$python_bin" -m py_compile "$stage/hub.py"
|
||||
OWRT_HOST_POLICY_HUB_SOURCE="$stage/hub.py" "$python_bin" "$stage/test_host_policy.py" HostPolicyTests -v
|
||||
OWRT_RESOURCE_HUB_SOURCE="$stage/hub.py" "$python_bin" "$stage/resources.py" -v
|
||||
@@ -226,7 +226,7 @@ for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
done
|
||||
test "$healthy" = 1
|
||||
cmp "$candidate" "$target"
|
||||
printf '\nHOST_POLICY_VPS_V116_%s_OK\n' '__ACTION__'
|
||||
printf '\nHOST_POLICY_VPS_V117_%s_OK\n' '__ACTION__'
|
||||
'@
|
||||
}
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ if ($Target -eq 'Router') {
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup=/root/owrt-remote-backup-before-hub-url-v116
|
||||
backup=/root/owrt-remote-backup-before-hub-url-v117
|
||||
chmod 700 "$stage"
|
||||
config=/etc/config/owrtremote
|
||||
changed=0
|
||||
@@ -99,7 +99,7 @@ if [ '__ACTION__' = Install ]; then
|
||||
grep -F -q 'heartbeat_set_url_if_changed()' "$stage/agent"
|
||||
grep -F -q 'reason=https-downgrade' "$stage/agent"
|
||||
for item in agent wan.sh; do grep -F -q -- '--post301' "$stage/$item"; done
|
||||
for item in cgi luci; do grep -F -q 'v116' "$stage/$item"; done
|
||||
for item in cgi luci; do grep -F -q 'v117' "$stage/$item"; done
|
||||
if [ ! -e "$backup" ]; then
|
||||
save_files "$stage/retained-backup"
|
||||
mv "$stage/retained-backup" "$backup"
|
||||
@@ -138,7 +138,7 @@ else
|
||||
changed=1
|
||||
restore_files "$backup"
|
||||
fi
|
||||
printf '\nBackup: %s\nHUB_URL_ROUTER_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nHUB_URL_ROUTER_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
} else {
|
||||
$Payloads = @{
|
||||
@@ -152,7 +152,7 @@ set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
chmod 700 "$stage"
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-hub-url-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-hub-url-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
cleanup() {
|
||||
@@ -174,7 +174,7 @@ python_bin=/opt/owrt-remote/venv/bin/python
|
||||
[ -x "$python_bin" ] || python_bin=python3
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
grep -F -q 'def router_hub_origins(' "$stage/hub.py"
|
||||
grep -F -q 'v116' "$stage/hub.py"
|
||||
grep -F -q 'v117' "$stage/hub.py"
|
||||
"$python_bin" -m py_compile "$stage/hub.py"
|
||||
OWRT_HOST_POLICY_HUB_SOURCE="$stage/hub.py" "$python_bin" "$stage/host_policy.py" HostPolicyTests -v
|
||||
OWRT_RESOURCE_HUB_SOURCE="$stage/hub.py" "$python_bin" "$stage/resources.py" IntegrationTests -v
|
||||
@@ -198,7 +198,7 @@ for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
done
|
||||
test "$healthy" = 1
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nHUB_URL_VPS_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nHUB_URL_VPS_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
}
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action).Replace('__HUB_URL__', $HubUrl)
|
||||
|
||||
@@ -14,7 +14,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-online-status-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-online-status-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
cleanup() {
|
||||
@@ -59,7 +59,7 @@ for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
done
|
||||
test "$healthy" = 1
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nONLINE_STATUS_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nONLINE_STATUS_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
try {
|
||||
|
||||
@@ -15,7 +15,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-password-logout-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-password-logout-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -105,7 +105,7 @@ if [ "$healthy" -ne 1 ]; then
|
||||
exit 1
|
||||
fi
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nPASSWORD_LOGOUT_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nPASSWORD_LOGOUT_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
param(
|
||||
[ValidateNotNullOrEmpty()][string]$Vps = 'root@193.233.82.38',
|
||||
[ValidateSet('Install', 'Rollback')][string]$Action = 'Install',
|
||||
[switch]$PrepareOnly
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
$UploadName = 'owrt-persistent-logs-vps-' + [guid]::NewGuid().ToString('N')
|
||||
$StageRoot = Join-Path ([System.IO.Path]::GetTempPath()) $UploadName
|
||||
$Utf8 = [System.Text.UTF8Encoding]::new($false)
|
||||
|
||||
$RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-persistent-logs-sync-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
show_failure() {
|
||||
echo '--- owrt-remote status ---' >&2
|
||||
systemctl --no-pager -l status owrt-remote >&2 || true
|
||||
echo '--- owrt-remote journal ---' >&2
|
||||
journalctl -u owrt-remote -n 80 --no-pager -l >&2 || true
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
code=$?
|
||||
trap - EXIT
|
||||
if [ "$code" -ne 0 ]; then
|
||||
echo "Remote step failed with rc=$code" >&2
|
||||
show_failure
|
||||
fi
|
||||
if [ "$code" -ne 0 ] && [ "$changed" = 1 ] && [ -s "$stage/previous.py" ]; then
|
||||
echo 'Restoring the previous Hub file...' >&2
|
||||
cp -p "$stage/previous.py" "$candidate"
|
||||
mv -f "$candidate" "$target"
|
||||
systemctl restart owrt-remote || true
|
||||
fi
|
||||
rm -f "$candidate"
|
||||
rm -rf "$stage"
|
||||
exit "$code"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
test -s "$target"
|
||||
python_bin=/opt/owrt-remote/venv/bin/python
|
||||
[ -x "$python_bin" ] || python_bin=python3
|
||||
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
test -s "$stage/hub.py"
|
||||
echo 'Validating candidate...'
|
||||
"$python_bin" -m py_compile "$stage/hub.py"
|
||||
"$python_bin" - "$stage/hub.py" <<'PY'
|
||||
from pathlib import Path
|
||||
import sys
|
||||
source = Path(sys.argv[1]).read_text(encoding='utf-8')
|
||||
required = (
|
||||
'def receive_router_logs(',
|
||||
'def router_log_storage(',
|
||||
'def router_log_diagnosis(',
|
||||
'def router_logs_html(',
|
||||
'def router_logs_api(',
|
||||
'data-global-logs-link=',
|
||||
'ROUTER_LOG_MAX_BYTES = 2 * 1024 * 1024',
|
||||
)
|
||||
missing = [marker for marker in required if marker not in source]
|
||||
if missing:
|
||||
for marker in missing:
|
||||
print('Missing marker:', repr(marker), file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
print('Candidate validation OK')
|
||||
PY
|
||||
# Read only the running Hub's relevant environment/argv; never print tokens.
|
||||
hub_pid=$(systemctl show -p MainPID --value owrt-remote)
|
||||
case "$hub_pid" in ''|*[!0-9]*|0) echo 'Cannot identify the running Hub storage.' >&2; exit 1 ;; esac
|
||||
"$python_bin" - "$stage/hub.py" "$hub_pid" <<'PY'
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
sys.dont_write_bytecode = True
|
||||
pid = sys.argv[2]
|
||||
environment = Path('/proc/' + pid + '/environ').read_bytes().split(b'\0')
|
||||
for item in environment:
|
||||
key, _, value = item.partition(b'=')
|
||||
if key in (b'OWRT_REMOTE_STATE_DIR', b'OWRT_REMOTE_DB'):
|
||||
os.environ[key.decode()] = os.fsdecode(value)
|
||||
spec = importlib.util.spec_from_file_location('storage_preflight', sys.argv[1])
|
||||
hub = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = hub
|
||||
spec.loader.exec_module(hub)
|
||||
db_path = hub.DB_PATH
|
||||
args = Path('/proc/' + pid + '/cmdline').read_bytes().split(b'\0')
|
||||
for index, arg in enumerate(args):
|
||||
if arg == b'--db' and index + 1 < len(args):
|
||||
db_path = Path(os.fsdecode(args[index + 1]))
|
||||
elif arg.startswith(b'--db='):
|
||||
db_path = Path(os.fsdecode(arg[5:]))
|
||||
if not db_path.is_absolute():
|
||||
db_path = Path('/proc/' + pid + '/cwd').resolve() / db_path
|
||||
storage = hub.router_log_storage(db_path)
|
||||
print('VPS log archive storage:', json.dumps(storage, ensure_ascii=False))
|
||||
if storage['persistent'] is not True or not storage.get('writable'):
|
||||
raise SystemExit('Archive is not confirmed on writable persistent disk. Nothing installed; check the Hub database mount.')
|
||||
if not db_path.is_file() or not os.access(db_path, os.W_OK):
|
||||
raise SystemExit('Hub database is missing or not writable. Nothing installed.')
|
||||
PY
|
||||
if [ ! -e "$backup" ]; then
|
||||
cp -p "$target" "$backup"
|
||||
fi
|
||||
source_file="$stage/hub.py"
|
||||
else
|
||||
test -s "$backup"
|
||||
"$python_bin" -m py_compile "$backup"
|
||||
source_file="$backup"
|
||||
fi
|
||||
|
||||
cp -p "$target" "$stage/previous.py"
|
||||
cp "$source_file" "$candidate"
|
||||
chmod 0755 "$candidate"
|
||||
changed=1
|
||||
mv -f "$candidate" "$target"
|
||||
echo 'Restarting owrt-remote...'
|
||||
if ! systemctl restart owrt-remote; then
|
||||
show_failure
|
||||
exit 1
|
||||
fi
|
||||
|
||||
healthy=0
|
||||
for attempt in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do
|
||||
if systemctl is-active --quiet owrt-remote && curl -fsS --max-time 5 http://127.0.0.1:8088/health; then
|
||||
healthy=1
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
if [ "$healthy" -ne 1 ]; then
|
||||
echo 'Health check did not become ready.' >&2
|
||||
show_failure
|
||||
exit 1
|
||||
fi
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nPERSISTENT_LOGS_VPS_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
|
||||
try {
|
||||
[System.IO.Directory]::CreateDirectory($StageRoot) | Out-Null
|
||||
[System.IO.File]::WriteAllText(
|
||||
(Join-Path $StageRoot 'run.sh'),
|
||||
$RemoteScript.Replace("`r`n", "`n") + "`n",
|
||||
$Utf8
|
||||
)
|
||||
|
||||
if ($Action -eq 'Install') {
|
||||
$Content = [System.IO.File]::ReadAllText((Join-Path $RepoRoot 'vps\owrt-remote-hub.py')).Replace("`r`n", "`n")
|
||||
[System.IO.File]::WriteAllText((Join-Path $StageRoot 'hub.py'), $Content, $Utf8)
|
||||
}
|
||||
|
||||
if ($PrepareOnly) {
|
||||
Write-Host "Prepared only: $StageRoot"
|
||||
return
|
||||
}
|
||||
|
||||
& scp -O -r -- $StageRoot "${Vps}:/tmp/"
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Upload failed. VPS was not changed.' }
|
||||
& ssh -T -o ConnectTimeout=10 $Vps "sh /tmp/$UploadName/run.sh"
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Action failed. See the output above; success was not confirmed." }
|
||||
Write-Host "Persistent logs VPS $Action finished successfully. Refresh Hub with Ctrl+F5."
|
||||
}
|
||||
finally {
|
||||
if (-not $PrepareOnly) {
|
||||
$Resolved = [System.IO.Path]::GetFullPath($StageRoot)
|
||||
$Expected = [System.IO.Path]::GetFullPath((Join-Path ([System.IO.Path]::GetTempPath()) $UploadName))
|
||||
if ($Resolved -ne $Expected -or (Split-Path -Leaf $Resolved) -ne $UploadName) {
|
||||
throw 'Unsafe staging cleanup path.'
|
||||
}
|
||||
if (Test-Path -LiteralPath $Resolved) {
|
||||
Remove-Item -LiteralPath $Resolved -Recurse -Force
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,195 @@
|
||||
param(
|
||||
[ValidateNotNullOrEmpty()][string]$Router = 'root@192.168.2.1',
|
||||
[ValidateSet('Install', 'Rollback')][string]$Action = 'Install',
|
||||
[switch]$PrepareOnly
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
$UploadName = 'owrt-persistent-logs-' + [guid]::NewGuid().ToString('N')
|
||||
$StageRoot = Join-Path ([System.IO.Path]::GetTempPath()) $UploadName
|
||||
$Utf8 = [System.Text.UTF8Encoding]::new($false)
|
||||
$RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup=/root/owrt-remote-backup-before-persistent-logs-v117
|
||||
items='cgi agent logs.sh logs-init luci'
|
||||
changed=0
|
||||
phase=preflight
|
||||
item_path() {
|
||||
case "$1" in
|
||||
cgi) echo /www/cgi-bin/owrt-remote ;;
|
||||
agent) echo /usr/sbin/owrt-remote ;;
|
||||
init) echo /etc/init.d/owrt-remote ;;
|
||||
logs.sh) echo /usr/lib/owrt-remote/logs.sh ;;
|
||||
logs-init) echo /etc/init.d/owrt-remote-logs ;;
|
||||
luci) echo /www/luci-static/resources/view/owrt_remote.js ;;
|
||||
esac
|
||||
}
|
||||
refresh_router_ui() {
|
||||
# Updated CGI and LuCI assets need no rpcd restart; keep management sessions alive.
|
||||
if [ -x /etc/init.d/uhttpd ]; then /etc/init.d/uhttpd reload || true; fi
|
||||
}
|
||||
save_files() {
|
||||
local dir item path
|
||||
dir=$1
|
||||
mkdir -p "$dir"
|
||||
chmod 700 "$dir"
|
||||
for item in $items; do
|
||||
path=$(item_path "$item")
|
||||
if [ -e "$path" ]; then cp -p "$path" "$dir/$item"; else : >"$dir/$item.absent"; fi
|
||||
done
|
||||
if [ -x /etc/init.d/owrt-remote-logs ] && /etc/init.d/owrt-remote-logs enabled; then : >"$dir/logs-enabled"; fi
|
||||
if ubus call service list '{"name":"owrt-remote-logs"}' 2>/dev/null | grep -q '"running": true'; then : >"$dir/logs-running"; fi
|
||||
: >"$dir/complete"
|
||||
}
|
||||
validate_backup() {
|
||||
local dir item
|
||||
dir=$1
|
||||
[ -f "$dir/complete" ] || return 1
|
||||
for item in $items; do
|
||||
[ -s "$dir/$item" ] || [ -f "$dir/$item.absent" ] || return 1
|
||||
done
|
||||
}
|
||||
restore_files() {
|
||||
local dir item path
|
||||
dir=$1
|
||||
validate_backup "$dir" || { echo "Incomplete backup: $dir" >&2; return 1; }
|
||||
if [ -x /etc/init.d/owrt-remote-logs ]; then
|
||||
/etc/init.d/owrt-remote-logs stop || true
|
||||
/etc/init.d/owrt-remote-logs disable || true
|
||||
fi
|
||||
for item in $items; do
|
||||
path=$(item_path "$item")
|
||||
if [ -f "$dir/$item.absent" ]; then
|
||||
rm -f "$path" || return 1
|
||||
else
|
||||
cp -p "$dir/$item" "$path.new-__UPLOAD_NAME__" && mv -f "$path.new-__UPLOAD_NAME__" "$path" || return 1
|
||||
fi
|
||||
done
|
||||
if [ -x /etc/init.d/owrt-remote-logs ]; then
|
||||
[ ! -f "$dir/logs-enabled" ] || /etc/init.d/owrt-remote-logs enable
|
||||
[ ! -f "$dir/logs-running" ] || /etc/init.d/owrt-remote-logs start
|
||||
else
|
||||
: # Preserve both journals for manual recovery.
|
||||
fi
|
||||
|
||||
refresh_router_ui
|
||||
}
|
||||
cleanup() {
|
||||
code=$?
|
||||
trap - EXIT
|
||||
if [ "$code" -ne 0 ]; then
|
||||
printf 'Failed during %s (rc=%s).\n' "$phase" "$code" >&2
|
||||
if [ "$changed" = 1 ]; then
|
||||
echo 'Restoring files from this attempt.' >&2
|
||||
restore_files "$stage/previous" || echo 'Restore failed; retained backup is available.' >&2
|
||||
fi
|
||||
fi
|
||||
for item in $items; do path=$(item_path "$item"); rm -f "$path.new-__UPLOAD_NAME__"; done
|
||||
rm -rf "$stage"
|
||||
exit "$code"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
for item in cgi agent logs.sh logs-init; do path=$(item_path "$item"); test -s "$path"; done
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
for item in $items; do test -s "$stage/$item"; done
|
||||
for item in cgi agent logs.sh logs-init; do sh -n "$stage/$item"; done
|
||||
command -v logread >/dev/null
|
||||
command -v ubus >/dev/null
|
||||
command -v mkfifo >/dev/null
|
||||
command -v base64 >/dev/null
|
||||
command -v jsonfilter >/dev/null
|
||||
if ! command -v sha256sum >/dev/null 2>&1 && ! command -v md5sum >/dev/null 2>&1 && ! command -v cksum >/dev/null 2>&1; then
|
||||
echo 'No supported checksum utility (sha256sum, md5sum or cksum). Nothing installed.' >&2
|
||||
exit 1
|
||||
fi
|
||||
grep -F -q 'globalLogsToggle' "$stage/cgi"
|
||||
for item in cgi luci; do grep -F -q 'v117' "$stage/$item"; done
|
||||
grep -F -q 'LOG_BUFFER_LIMIT=16384' "$stage/logs.sh"
|
||||
grep -F -q 'LOG_KEEP_SECONDS=86400' "$stage/logs.sh"
|
||||
grep -F -q 'log_sync_loop' "$stage/logs.sh"
|
||||
grep -F -q 'log_repair_legacy_locked' "$stage/logs.sh"
|
||||
grep -F -q 'sync_global_logs' "$stage/agent"
|
||||
phase=backup
|
||||
if [ ! -e "$backup" ]; then save_files "$stage/retained"; mv "$stage/retained" "$backup"; fi
|
||||
validate_backup "$backup" || { echo "Incomplete backup: $backup. Nothing changed." >&2; exit 1; }
|
||||
save_files "$stage/previous"
|
||||
phase=install
|
||||
changed=1
|
||||
if [ -x /etc/init.d/owrt-remote-logs ]; then /etc/init.d/owrt-remote-logs stop; fi
|
||||
mkdir -p /usr/lib/owrt-remote /www/luci-static/resources/view
|
||||
for item in $items; do
|
||||
path=$(item_path "$item")
|
||||
cp "$stage/$item" "$path.new-__UPLOAD_NAME__"
|
||||
case "$item" in logs.sh|luci) chmod 644 "$path.new-__UPLOAD_NAME__" ;; *) chmod 755 "$path.new-__UPLOAD_NAME__" ;; esac
|
||||
mv -f "$path.new-__UPLOAD_NAME__" "$path"
|
||||
done
|
||||
phase=legacy-migration
|
||||
sh /usr/lib/owrt-remote/logs.sh migrate
|
||||
/etc/init.d/owrt-remote-logs enable
|
||||
/etc/init.d/owrt-remote-logs start
|
||||
phase=verify
|
||||
ready=0
|
||||
for attempt in 1 2 3 4 5; do
|
||||
if ubus call service list '{"name":"owrt-remote-logs"}' | grep -q '"running": true'; then ready=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
if [ "$ready" != 1 ]; then echo 'Log collector did not start.' >&2; logread -e owrt-remote | tail -n 30 >&2; exit 1; fi
|
||||
# Generate UTF-8 on the router; Windows PowerShell 5.1 reads scripts as ANSI.
|
||||
marker="$(printf '\320\223\320\273\320\276\320\261\320\260\320\273\040\320\273\320\276\320\263\320\270\072') collector-__UPLOAD_NAME__"
|
||||
ready=0
|
||||
for attempt in 1 2 3 4 5; do
|
||||
logger -t owrt-remote "$marker"
|
||||
sleep 1
|
||||
if sh /usr/lib/owrt-remote/logs.sh read | grep -F -q "$marker"; then ready=1; break; fi
|
||||
done
|
||||
if [ "$ready" != 1 ]; then echo 'Log collector is running but did not capture the test event.' >&2; exit 1; fi
|
||||
for item in $items; do path=$(item_path "$item"); cmp "$stage/$item" "$path"; done
|
||||
phase=storage-check
|
||||
sh /usr/lib/owrt-remote/logs.sh flush force
|
||||
sh /usr/lib/owrt-remote/logs.sh status
|
||||
refresh_router_ui
|
||||
echo 'Log collector: running. Router settings preserved.'
|
||||
else
|
||||
validate_backup "$backup" || { echo "No complete backup: $backup. Nothing changed." >&2; exit 1; }
|
||||
save_files "$stage/previous"
|
||||
changed=1
|
||||
phase=rollback
|
||||
restore_files "$backup"
|
||||
fi
|
||||
printf 'Backup: %s\n' "$backup"
|
||||
printf '\nPERSISTENT_LOGS_ROUTER_%s_OK\n' '__ACTION__'
|
||||
'@
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
$Files = @{
|
||||
'cgi' = 'files\www\cgi-bin\owrt-remote'
|
||||
'agent' = 'files\usr\sbin\owrt-remote'
|
||||
'logs.sh' = 'files\usr\lib\owrt-remote\logs.sh'
|
||||
'logs-init' = 'files\etc\init.d\owrt-remote-logs'
|
||||
'luci' = 'files\www\luci-static\resources\view\owrt_remote.js'
|
||||
}
|
||||
try {
|
||||
[System.IO.Directory]::CreateDirectory($StageRoot) | Out-Null
|
||||
[System.IO.File]::WriteAllText((Join-Path $StageRoot 'run.sh'), $RemoteScript.Replace("`r`n", "`n") + "`n", $Utf8)
|
||||
if ($Action -eq 'Install') {
|
||||
foreach ($Item in $Files.GetEnumerator()) {
|
||||
$Content = [System.IO.File]::ReadAllText((Join-Path $RepoRoot $Item.Value)).Replace("`r`n", "`n")
|
||||
[System.IO.File]::WriteAllText((Join-Path $StageRoot $Item.Key), $Content, $Utf8)
|
||||
}
|
||||
}
|
||||
if ($PrepareOnly) { Write-Host "Prepared only: $StageRoot"; return }
|
||||
& scp -O -r -- $StageRoot "${Router}:/tmp/"
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Upload failed. The installation has not started.' }
|
||||
& ssh -T -o ConnectTimeout=10 $Router "sh /tmp/$UploadName/run.sh"
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Action failed. See the output above; success was not confirmed." }
|
||||
Write-Host "Persistent logs $Action finished successfully. Refresh the router panel with Ctrl+F5."
|
||||
} finally {
|
||||
if (-not $PrepareOnly) {
|
||||
$Resolved = [System.IO.Path]::GetFullPath($StageRoot)
|
||||
$Expected = [System.IO.Path]::GetFullPath((Join-Path ([System.IO.Path]::GetTempPath()) $UploadName))
|
||||
if ($Resolved -ne $Expected -or (Split-Path -Leaf $Resolved) -ne $UploadName) { throw 'Unsafe staging cleanup path.' }
|
||||
if (Test-Path -LiteralPath $Resolved) { Remove-Item -LiteralPath $Resolved -Recurse -Force }
|
||||
}
|
||||
}
|
||||
@@ -17,8 +17,8 @@ if ($Action -eq 'Install') {
|
||||
set -Eeuo pipefail
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-recaptcha-v3-v116
|
||||
captcha_backup=/opt/owrt-remote/captcha-before-recaptcha-v3-v116.json
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-recaptcha-v3-v117
|
||||
captcha_backup=/opt/owrt-remote/captcha-before-recaptcha-v3-v117.json
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
cleanup() {
|
||||
@@ -45,7 +45,7 @@ if [ ! -x "$python_bin" ]; then python_bin=python3; fi
|
||||
import ast, pathlib, sys
|
||||
source = pathlib.Path(sys.argv[1]).read_text(encoding='utf-8')
|
||||
tree = ast.parse(source)
|
||||
assert 'v116' in source
|
||||
assert 'v117' in source
|
||||
assert 'v108' not in source
|
||||
assert 'CAPTCHA_MODE_RECAPTCHA_V3' in source
|
||||
assert 'prepareLoginRecaptcha' in source
|
||||
@@ -88,7 +88,7 @@ pid=$(systemctl show owrt-remote -p MainPID --value)
|
||||
test "$pid" -gt 0
|
||||
printf 'PID=%s FD=%s\n' "$pid" "$(find /proc/$pid/fd -maxdepth 1 -type l | wc -l)"
|
||||
systemctl --no-pager --full status owrt-remote | sed -n '1,16p'
|
||||
echo RECAPTCHA_V3_V116_OK
|
||||
echo RECAPTCHA_V3_V117_OK
|
||||
'@
|
||||
} elseif ($Action -eq 'Rollback') {
|
||||
$RemoteScript = @'
|
||||
@@ -96,8 +96,8 @@ echo RECAPTCHA_V3_V116_OK
|
||||
set -Eeuo pipefail
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-recaptcha-v3-v116
|
||||
captcha_backup=/opt/owrt-remote/captcha-before-recaptcha-v3-v116.json
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-recaptcha-v3-v117
|
||||
captcha_backup=/opt/owrt-remote/captcha-before-recaptcha-v3-v117.json
|
||||
candidate=$target.rollback-__UPLOAD_NAME__
|
||||
changed=0
|
||||
restore_captcha() {
|
||||
|
||||
@@ -18,7 +18,7 @@ if ($Action -eq 'Install') {
|
||||
set -Eeuo pipefail
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-resource-fix-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-resource-fix-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
cleanup() {
|
||||
@@ -45,7 +45,7 @@ if [ ! -x "$python_bin" ]; then python_bin=python3; fi
|
||||
import ast, pathlib, sys
|
||||
source = pathlib.Path(sys.argv[1]).read_text(encoding='utf-8')
|
||||
tree = ast.parse(source)
|
||||
assert 'v116' in source
|
||||
assert 'v117' in source
|
||||
assert 'v108' not in source
|
||||
factory = next(node for node in tree.body if isinstance(node, ast.FunctionDef) and node.name == 'connect')
|
||||
assert any(isinstance(item, ast.Name) and item.id == 'contextmanager' for item in factory.decorator_list)
|
||||
@@ -78,7 +78,7 @@ pid=$(systemctl show owrt-remote -p MainPID --value)
|
||||
test "$pid" -gt 0
|
||||
printf 'PID=%s FD=%s\n' "$pid" "$(find /proc/$pid/fd -maxdepth 1 -type l | wc -l)"
|
||||
systemctl --no-pager --full status owrt-remote | sed -n '1,16p'
|
||||
echo RESOURCE_FIX_V116_OK
|
||||
echo RESOURCE_FIX_V117_OK
|
||||
'@
|
||||
} elseif ($Action -eq 'Rollback') {
|
||||
$RemoteScript = @'
|
||||
@@ -86,7 +86,7 @@ echo RESOURCE_FIX_V116_OK
|
||||
set -Eeuo pipefail
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-resource-fix-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-resource-fix-v117
|
||||
candidate=$target.rollback-__UPLOAD_NAME__
|
||||
changed=0
|
||||
cleanup() {
|
||||
|
||||
@@ -15,7 +15,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-router-card-ui-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-router-card-ui-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -106,7 +106,7 @@ if [ "$healthy" -ne 1 ]; then
|
||||
exit 1
|
||||
fi
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nROUTER_CARD_UI_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nROUTER_CARD_UI_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
|
||||
@@ -6,7 +6,7 @@ param(
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
$UploadName = 'owrt-version-v116-' + [guid]::NewGuid().ToString('N')
|
||||
$UploadName = 'owrt-version-v117-' + [guid]::NewGuid().ToString('N')
|
||||
$StageRoot = Join-Path ([System.IO.Path]::GetTempPath()) $UploadName
|
||||
$Utf8 = [System.Text.UTF8Encoding]::new($false)
|
||||
|
||||
@@ -15,7 +15,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -57,8 +57,8 @@ if [ '__ACTION__' = Install ]; then
|
||||
from pathlib import Path
|
||||
import sys
|
||||
source = Path(sys.argv[1]).read_text(encoding='utf-8')
|
||||
required = ('v116',)
|
||||
if 'v' + '115' in source:
|
||||
required = ('v117',)
|
||||
if 'v' + '116' in source:
|
||||
print('Old UI version remains in candidate', file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
missing = [marker for marker in required if marker not in source]
|
||||
@@ -103,7 +103,7 @@ if [ "$healthy" -ne 1 ]; then
|
||||
exit 1
|
||||
fi
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nHUB_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nHUB_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
@@ -130,7 +130,7 @@ try {
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Upload failed. VPS was not changed.' }
|
||||
& ssh -T -o ConnectTimeout=10 $Vps "sh /tmp/$UploadName/run.sh"
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Action failed. See the output above; success was not confirmed." }
|
||||
Write-Host "Hub v116 $Action finished successfully. Refresh the Hub panel."
|
||||
Write-Host "Hub v117 $Action finished successfully. Refresh the Hub panel."
|
||||
}
|
||||
finally {
|
||||
if (-not $PrepareOnly) {
|
||||
@@ -14,7 +14,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-vps-widgets-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-vps-widgets-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
cleanup() {
|
||||
@@ -37,7 +37,7 @@ python_bin=/opt/owrt-remote/venv/bin/python
|
||||
[ -x "$python_bin" ] || python_bin=python3
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
test -s "$stage/hub.py"
|
||||
grep -F -q 'v116' "$stage/hub.py"
|
||||
grep -F -q 'v117' "$stage/hub.py"
|
||||
"$python_bin" -m py_compile "$stage/hub.py"
|
||||
OWRT_VPS_WIDGETS_HUB_SOURCE="$stage/hub.py" "$python_bin" "$stage/tests.py" -v
|
||||
if [ ! -e "$backup" ]; then cp -p "$target" "$backup"; fi
|
||||
@@ -67,7 +67,7 @@ if [ '__ACTION__' = Install ]; then
|
||||
code=$(curl -sS --max-time 5 -o "$stage/response.json" -w '%{http_code}' http://127.0.0.1:8088/api/vps/resources)
|
||||
test "$code" = 401
|
||||
fi
|
||||
printf '\nVPS_WIDGETS_V116_%s_OK\n' '__ACTION__'
|
||||
printf '\nVPS_WIDGETS_V117_%s_OK\n' '__ACTION__'
|
||||
'@
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
try {
|
||||
|
||||
@@ -14,7 +14,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/usr/lib/owrt-remote/wan.sh
|
||||
backup_dir=/root/owrt-remote-backup-before-wan-dns-v116
|
||||
backup_dir=/root/owrt-remote-backup-before-wan-dns-v117
|
||||
changed=0
|
||||
phase=preflight
|
||||
heartbeat_pid() {
|
||||
|
||||
@@ -20,7 +20,7 @@ $RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup=/root/owrt-remote-backup-before-wan-multi-v116
|
||||
backup=/root/owrt-remote-backup-before-wan-multi-v117
|
||||
config=/etc/config/owrtremote
|
||||
changed=0
|
||||
item_path() {
|
||||
@@ -81,7 +81,7 @@ test -s "$config"
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
for item in agent wan.sh cgi hotplug; do test -s "$stage/$item"; sh -n "$stage/$item"; done
|
||||
test -s "$stage/luci"
|
||||
for item in cgi luci; do grep -F -q 'v116' "$stage/$item"; done
|
||||
for item in cgi luci; do grep -F -q 'v117' "$stage/$item"; done
|
||||
grep -F -q 'wan_interface_picker' "$stage/cgi"
|
||||
grep -F -q 'wan_select_interface()' "$stage/wan.sh"
|
||||
grep -F -q 'wan_reconcile_xray()' "$stage/agent"
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
param(
|
||||
[ValidateNotNullOrEmpty()][string]$Router = 'root@192.168.2.1',
|
||||
[ValidateSet('Install', 'Rollback')][string]$Action = 'Install',
|
||||
[switch]$PrepareOnly
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$RepoRoot = Split-Path -Parent $PSScriptRoot
|
||||
$UploadName = 'owrt-wan-recovery-' + [guid]::NewGuid().ToString('N')
|
||||
$StageRoot = Join-Path ([System.IO.Path]::GetTempPath()) $UploadName
|
||||
$Utf8 = [System.Text.UTF8Encoding]::new($false)
|
||||
$Payloads = @{
|
||||
'wan.sh' = 'files\usr\lib\owrt-remote\wan.sh'
|
||||
'agent' = 'files\usr\sbin\owrt-remote'
|
||||
}
|
||||
$RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup_dir=/root/owrt-remote-backup-before-wan-recovery-v117
|
||||
changed=0
|
||||
phase=preflight
|
||||
item_path() {
|
||||
case "$1" in
|
||||
wan.sh) printf /usr/lib/owrt-remote/wan.sh ;;
|
||||
agent) printf /usr/sbin/owrt-remote ;;
|
||||
esac
|
||||
}
|
||||
complete_backup() {
|
||||
[ -f "$1/complete" ] && [ -s "$1/wan.sh" ] && [ -s "$1/agent" ]
|
||||
}
|
||||
restore_files() {
|
||||
local dir item path
|
||||
dir=$1
|
||||
complete_backup "$dir" || return 1
|
||||
for item in agent wan.sh; do
|
||||
path=$(item_path "$item")
|
||||
cp -p "$dir/$item" "$path.new-__UPLOAD_NAME__" || return 1
|
||||
mv -f "$path.new-__UPLOAD_NAME__" "$path" || return 1
|
||||
done
|
||||
}
|
||||
heartbeat_pid() {
|
||||
local state
|
||||
state=$(ubus call service list '{"name":"owrt-remote"}') || return 1
|
||||
printf '%s' "$state" | jsonfilter -e '@["owrt-remote"].instances.heartbeat.pid' 2>/dev/null || true
|
||||
}
|
||||
refresh_heartbeat() {
|
||||
local old_pid new_pid attempt
|
||||
old_pid=$(heartbeat_pid) || return 1
|
||||
case "$old_pid" in ''|0) echo 'Heartbeat loop is stopped; files will be used at its next start.'; return 0 ;; *[!0-9]*) return 1 ;; esac
|
||||
# Signal only the supervised heartbeat shell; procd respawns it from the
|
||||
# new agent. A healthy Xray instance stays running; a crashed one can recover.
|
||||
ubus call service signal '{"name":"owrt-remote","instance":"heartbeat","signal":9}' >/dev/null || return 1
|
||||
attempt=0
|
||||
while [ "$attempt" -lt 20 ]; do
|
||||
sleep 1
|
||||
new_pid=$(heartbeat_pid) || return 1
|
||||
case "$new_pid" in ''|0|*[!0-9]*) ;; *)
|
||||
[ "$new_pid" = "$old_pid" ] || { echo 'Heartbeat loop reloaded; WAN recovery is active for enabled tunnels.'; return 0; } ;;
|
||||
esac
|
||||
attempt=$((attempt+1))
|
||||
done
|
||||
echo 'Heartbeat loop did not respawn. Code will be restored.' >&2
|
||||
return 1
|
||||
}
|
||||
cleanup() {
|
||||
code=$?
|
||||
trap - EXIT
|
||||
if [ "$code" -ne 0 ]; then
|
||||
printf 'Failed during %s (rc=%s).\n' "$phase" "$code" >&2
|
||||
if [ "$changed" = 1 ]; then
|
||||
echo 'Restoring the agent and WAN helper from this attempt.' >&2
|
||||
restore_files "$stage/previous" && refresh_heartbeat || echo 'Restore failed; retained backup is available.' >&2
|
||||
fi
|
||||
fi
|
||||
for item in agent wan.sh; do path=$(item_path "$item"); rm -f "$path.new-__UPLOAD_NAME__"; done
|
||||
rm -rf "$stage"
|
||||
exit "$code"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
for item in agent wan.sh; do
|
||||
path=$(item_path "$item")
|
||||
test -s "$path"
|
||||
done
|
||||
heartbeat_pid >/dev/null
|
||||
if [ '__ACTION__' = Install ]; then
|
||||
for item in agent wan.sh; do test -s "$stage/$item"; sh -n "$stage/$item"; done
|
||||
grep -F -q 'wan_recover_xray()' "$stage/agent"
|
||||
grep -F -q 'wait_for_selected_wan()' "$stage/agent"
|
||||
grep -F -q 'reason=https_required' "$stage/agent"
|
||||
grep -F -q 'remote_curl_post' "$stage/wan.sh"
|
||||
if [ ! -e "$backup_dir" ]; then
|
||||
mkdir -p "$stage/retained-backup"
|
||||
chmod 700 "$stage/retained-backup"
|
||||
for item in agent wan.sh; do path=$(item_path "$item"); cp -p "$path" "$stage/retained-backup/$item"; done
|
||||
touch "$stage/retained-backup/complete"
|
||||
mv "$stage/retained-backup" "$backup_dir"
|
||||
fi
|
||||
complete_backup "$backup_dir" || { echo "Incomplete backup: $backup_dir. Nothing changed." >&2; exit 1; }
|
||||
source_dir=$stage
|
||||
else
|
||||
complete_backup "$backup_dir" || { echo "No complete backup: $backup_dir. Nothing changed." >&2; exit 1; }
|
||||
for item in agent wan.sh; do sh -n "$backup_dir/$item"; done
|
||||
source_dir=$backup_dir
|
||||
fi
|
||||
printf 'Backup: %s\n' "$backup_dir"
|
||||
mkdir -p "$stage/previous"
|
||||
for item in agent wan.sh; do path=$(item_path "$item"); cp -p "$path" "$stage/previous/$item"; done
|
||||
touch "$stage/previous/complete"
|
||||
phase=install
|
||||
changed=1
|
||||
for item in agent wan.sh; do
|
||||
path=$(item_path "$item")
|
||||
cp "$source_dir/$item" "$path.new-__UPLOAD_NAME__"
|
||||
chmod 755 "$path.new-__UPLOAD_NAME__"
|
||||
mv -f "$path.new-__UPLOAD_NAME__" "$path"
|
||||
done
|
||||
phase=verify
|
||||
for item in agent wan.sh; do
|
||||
path=$(item_path "$item")
|
||||
sh -n "$path"
|
||||
cmp "$source_dir/$item" "$path"
|
||||
done
|
||||
phase=heartbeat-reload
|
||||
refresh_heartbeat
|
||||
# The reloaded heartbeat recovers an enabled, crashed Xray instance when WAN
|
||||
# returns. A healthy tunnel is not restarted by this deployment.
|
||||
printf '\nWAN_RECOVERY_ROUTER_%s_OK\n' '__ACTION__'
|
||||
|
||||
'@
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
try {
|
||||
[System.IO.Directory]::CreateDirectory($StageRoot) | Out-Null
|
||||
[System.IO.File]::WriteAllText((Join-Path $StageRoot 'run.sh'), $RemoteScript.Replace("`r`n", "`n") + "`n", $Utf8)
|
||||
if ($Action -eq 'Install') {
|
||||
foreach ($Item in $Payloads.GetEnumerator()) {
|
||||
$Content = [System.IO.File]::ReadAllText((Join-Path $RepoRoot $Item.Value)).Replace("`r`n", "`n")
|
||||
[System.IO.File]::WriteAllText((Join-Path $StageRoot $Item.Key), $Content, $Utf8)
|
||||
}
|
||||
}
|
||||
if ($PrepareOnly) { Write-Host "Prepared only: $StageRoot"; return }
|
||||
& scp -O -r -- $StageRoot "${Router}:/tmp/"
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Upload failed. The installation has not started.' }
|
||||
& ssh -T -o ConnectTimeout=10 $Router "sh /tmp/$UploadName/run.sh"
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Action failed. See the output above; success was not confirmed." }
|
||||
Write-Host "WAN recovery fix $Action finished successfully."
|
||||
} finally {
|
||||
if (-not $PrepareOnly) {
|
||||
$Resolved = [System.IO.Path]::GetFullPath($StageRoot)
|
||||
$Expected = [System.IO.Path]::GetFullPath((Join-Path ([System.IO.Path]::GetTempPath()) $UploadName))
|
||||
if ($Resolved -ne $Expected -or (Split-Path -Leaf $Resolved) -ne $UploadName) { throw 'Unsafe staging cleanup path.' }
|
||||
if (Test-Path -LiteralPath $Resolved) { Remove-Item -LiteralPath $Resolved -Recurse -Force }
|
||||
}
|
||||
}
|
||||
@@ -17,7 +17,7 @@ $RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup=/root/owrt-remote-backup-before-wan-stability-v116
|
||||
backup=/root/owrt-remote-backup-before-wan-stability-v117
|
||||
config=/etc/config/owrtremote
|
||||
changed=0
|
||||
fail() {
|
||||
|
||||
@@ -15,7 +15,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/opt/owrt-remote/owrt-remote-hub.py
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-web-push-v116
|
||||
backup=/opt/owrt-remote/owrt-remote-hub.py.bak-before-web-push-v117
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -72,7 +72,7 @@ for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
done
|
||||
test "$healthy" = 1
|
||||
cmp "$source_file" "$target"
|
||||
printf '\nBackup: %s\nWEB_PUSH_V116_%s_OK\n' "$backup" '__ACTION__'
|
||||
printf '\nBackup: %s\nWEB_PUSH_V117_%s_OK\n' "$backup" '__ACTION__'
|
||||
'@
|
||||
|
||||
$RemoteScript = $RemoteScript.Replace('__UPLOAD_NAME__', $UploadName).Replace('__ACTION__', $Action)
|
||||
|
||||
@@ -14,7 +14,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/www/cgi-bin/owrt-remote
|
||||
backup_dir=/root/owrt-remote-backup-before-xray-apply-v116
|
||||
backup_dir=/root/owrt-remote-backup-before-xray-apply-v117
|
||||
backup=$backup_dir/owrt-remote
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -14,7 +14,7 @@ $RemoteScript = @'
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
target=/www/cgi-bin/owrt-remote
|
||||
backup_dir=/root/owrt-remote-backup-before-xray-mobile-v116
|
||||
backup_dir=/root/owrt-remote-backup-before-xray-mobile-v117
|
||||
backup=$backup_dir/owrt-remote
|
||||
candidate=$target.new-__UPLOAD_NAME__
|
||||
changed=0
|
||||
|
||||
@@ -17,7 +17,7 @@ $RemoteScript = @'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
stage=/tmp/__UPLOAD_NAME__
|
||||
backup_dir=/root/owrt-remote-backup-before-xray-validation-v116
|
||||
backup_dir=/root/owrt-remote-backup-before-xray-validation-v117
|
||||
changed=0
|
||||
phase=preflight
|
||||
item_path() {
|
||||
|
||||
+55
-32
@@ -3087,6 +3087,38 @@ ROUTER_LOG_MAX_BYTES = 2 * 1024 * 1024
|
||||
ROUTER_LOG_BATCH_MAX_BYTES = 65536
|
||||
|
||||
|
||||
def router_log_storage(db_path):
|
||||
"""Describe the archive's real backing mount, including symlinked DB paths."""
|
||||
path = Path(db_path).resolve()
|
||||
result = {'path': str(path), 'filesystem': 'unknown', 'persistent': None,
|
||||
'limit_bytes': ROUTER_LOG_MAX_BYTES}
|
||||
if sys.platform != 'linux':
|
||||
return result
|
||||
def unescape(value):
|
||||
return re.sub(r'\\([0-7]{3})', lambda m: chr(int(m[1], 8)), value)
|
||||
try:
|
||||
mounts = []
|
||||
with open('/proc/self/mountinfo', encoding='utf-8') as source:
|
||||
for line in source:
|
||||
fields, separator, tail = line.partition(' - ')
|
||||
left, right = fields.split(), tail.split()
|
||||
if not separator or len(left) < 6 or len(right) < 3:
|
||||
continue
|
||||
mount = Path(unescape(left[4]))
|
||||
if path.is_relative_to(mount):
|
||||
mounts.append((len(mount.parts), right[0], left[5]))
|
||||
if mounts:
|
||||
_, filesystem, flags = max(mounts, key=lambda item: item[0])
|
||||
persistent = (False if filesystem in {'tmpfs', 'ramfs', 'rootfs'} else
|
||||
True if filesystem in {'ext2', 'ext3', 'ext4', 'xfs', 'btrfs', 'f2fs',
|
||||
'jfs', 'zfs', 'ubifs', 'jffs2', 'nfs', 'nfs4', 'ceph'} else None)
|
||||
result.update(filesystem=filesystem, persistent=persistent,
|
||||
writable='rw' in flags.split(','))
|
||||
except OSError:
|
||||
pass
|
||||
return result
|
||||
|
||||
|
||||
def redact_router_log_bytes(raw):
|
||||
"""Defense in depth: do not archive credentials supplied by older agents."""
|
||||
secret = re.compile(rb'authorization|hub_token|vless_uuid|ssh_vless_uuid|password|passwd|web[._ -]?key|[?&](?:key|token)=|"(?:uuid|privatekey)"|[0-9a-f]{8}-[0-9a-f-]{27,}', re.I)
|
||||
@@ -3137,6 +3169,12 @@ def receive_router_logs(conn, router_id, batch, state=None):
|
||||
conn.execute('update router_logs set last_sync = ? where router_id = ?', (ts, router_id))
|
||||
return {'journal_id': journal_id, 'offset': end, 'clear_request': pending}
|
||||
cursor = int(row['cursor']) if row['journal_id'] == journal_id else 0
|
||||
# A status-only packet cannot prove delivery of any journal bytes. Older
|
||||
# helpers sent from=to=total even when every chunk was skipped. Acknowledging
|
||||
# that number permanently hid the retained records and invented rotation gaps.
|
||||
if not raw and not pending:
|
||||
conn.execute('update router_logs set last_sync = ? where router_id = ?', (ts, router_id))
|
||||
return {'journal_id': journal_id, 'offset': cursor, 'clear_request': ''}
|
||||
if not pending and row['journal_id'] == journal_id and end <= cursor:
|
||||
conn.execute('update router_logs set last_sync = ? where router_id = ?', (ts, router_id))
|
||||
return {'journal_id': journal_id, 'offset': cursor, 'clear_request': ''}
|
||||
@@ -3145,7 +3183,7 @@ def receive_router_logs(conn, router_id, batch, state=None):
|
||||
cursor = start
|
||||
stored = b''
|
||||
elif row['journal_id'] and row['journal_id'] != journal_id:
|
||||
stored += ('[OK] ' + iso_time(ts) + ' Роутер начал новый журнал: перезагрузка или очистка в ОЗУ.\n').encode('utf-8')
|
||||
stored += ('[OK] ' + iso_time(ts) + ' Роутер начал новый журнал: очистка или восстановление сохранённой истории.\n').encode('utf-8')
|
||||
if start > cursor:
|
||||
stored += ('[WARN] ' + iso_time(ts) + ' Часть записей не доставлена до ротации журнала на роутере.\n').encode('utf-8')
|
||||
if end > cursor:
|
||||
@@ -4968,11 +5006,16 @@ body::before{{content:"";position:fixed;inset:-25%;z-index:0;pointer-events:none
|
||||
.brand{{display:flex;align-items:center;gap:14px;width:100%;min-width:0}}
|
||||
h1{{margin:0;font-size:29px;line-height:1.2;letter-spacing:0}}.appBanner{{position:relative;display:inline-flex;align-items:center;justify-content:center;gap:8px;min-height:36px;min-width:132px;padding:8px 14px;border:1px solid rgba(34,211,238,.38);border-radius:999px;background:linear-gradient(110deg,rgba(34,211,238,.14),rgba(124,58,237,.24),rgba(236,72,153,.14));color:#f3e8ff;text-decoration:none;font-weight:800;font-size:13px;line-height:1;white-space:nowrap;box-shadow:0 10px 24px rgba(124,58,237,.16),inset 0 1px 0 rgba(255,255,255,.10);overflow:hidden}}.appBanner::before{{content:"";position:absolute;inset:0;background:linear-gradient(90deg,transparent,rgba(255,255,255,.20),transparent);transform:translateX(-120%);animation:bannerShine 6.2s ease-in-out infinite}}.appBanner span{{position:relative}}.appBannerVersion{{color:#fb7185;text-shadow:0 0 12px rgba(251,113,133,.35)}}.muted{{color:var(--muted)}}.top p{{margin:4px 0 0}}.links,.headerActions{{display:flex;align-items:center;gap:8px}}.links{{margin-top:0;flex-wrap:nowrap}}.links a,.badge{{position:relative;display:inline-flex;align-items:center;justify-content:center;gap:8px;min-height:36px;min-width:132px;padding:8px 14px;border:1px solid var(--line);border-radius:999px;background:rgba(255,255,255,.08);color:#f3e8ff;text-decoration:none;font-weight:800;font-size:13px;line-height:1;white-space:nowrap;overflow:hidden}}.headerActions{{position:relative;display:grid;grid-template-columns:repeat(5,minmax(0,1fr));align-self:flex-start;justify-content:flex-start;align-content:flex-start;flex:1 1 auto;min-width:0;gap:8px;padding-top:0;max-width:none}}.headerActions .badge,.headerActions .btn{{width:100%;min-height:36px;min-width:0;padding:8px 10px;border-radius:999px;font-weight:800;font-size:12px;line-height:1;white-space:nowrap}}.headerActions .btn[href="/logout"]{{margin-left:0}}.badge{{background:rgba(255,255,255,.08);color:#f3e8ff;box-shadow:inset 0 1px 0 rgba(255,255,255,.06)}}.nethavenTop{{border-color:rgba(34,211,238,.46);background:linear-gradient(110deg,rgba(14,165,233,.20),rgba(168,85,247,.22),rgba(34,197,94,.14));color:#ecfeff;box-shadow:0 10px 24px rgba(14,165,233,.14),inset 0 1px 0 rgba(255,255,255,.10)}}.nethavenTop::before{{content:"";position:absolute;inset:0;background:linear-gradient(90deg,transparent,rgba(255,255,255,.24),transparent);transform:translateX(-120%);animation:bannerShine 6.2s ease-in-out infinite;pointer-events:none}}.authToggle{{cursor:pointer}}.dot{{width:9px;height:9px;border-radius:999px;background:var(--red);box-shadow:0 0 13px rgba(251,113,133,.72)}}.dot.on{{background:var(--green);box-shadow:0 0 13px rgba(34,197,94,.75)}}.dot.warn{{background:var(--amber);box-shadow:0 0 13px rgba(245,158,11,.75)}}
|
||||
.toolbar{{display:grid;grid-template-columns:minmax(0,1fr) minmax(0,1.2fr) minmax(84px,.52fr) minmax(100px,.65fr) minmax(0,1fr) minmax(0,1fr) minmax(116px,.68fr);gap:8px;margin:18px 0;padding:14px 16px;background:linear-gradient(180deg,rgba(255,255,255,.08),rgba(255,255,255,.045)),var(--panel);border:1px solid var(--line);border-radius:8px;box-shadow:0 18px 46px rgba(0,0,0,.20);backdrop-filter:blur(10px);width:100%;max-width:100%;box-sizing:border-box}}.toolbar>*{{width:100%;min-width:0}}.toolbar input,.toolbar select{{background:rgba(255,255,255,.08);border-color:var(--line);color:#f3e8ff;box-shadow:inset 0 1px 0 rgba(255,255,255,.05)}}.toolbar input::placeholder{{color:#b9adc9}}
|
||||
@media(min-width:681px){{
|
||||
#routerForm select,#routerGroupFilter{{color-scheme:dark}}
|
||||
#routerForm select option,#routerGroupFilter option{{background-color:#221a34;color:#f7f2ff}}
|
||||
#routerForm select option:checked,#routerGroupFilter option:checked{{background-color:#7c3aed;color:#fff}}
|
||||
}}
|
||||
.authMenu{{position:absolute;right:0;top:calc(100% + 10px);z-index:60;width:min(640px,calc(100vw - 44px));max-height:min(820px,calc(100svh - 120px));overflow:auto;padding:16px;background:linear-gradient(180deg,rgba(255,255,255,.09),rgba(255,255,255,.05)),rgba(19,14,32,.96);border:1px solid var(--line);border-radius:8px;box-shadow:0 24px 70px rgba(0,0,0,.36);backdrop-filter:blur(12px);scrollbar-width:thin}}.authMenu[hidden]{{display:none}}.authMenu>h2,.authMenu>p{{display:none}}.authMenuHead{{position:relative;display:block}}.authMenuHead>div{{min-width:0}}.authMenuHead p{{display:none}}.authMenu h2{{margin:0 0 4px;font-size:18px}}.authMenuClose{{display:none;position:absolute;top:14px;right:14px;min-height:34px;padding:7px 12px;border-radius:999px;white-space:nowrap}}.authMenu p{{margin:0 0 12px;color:var(--muted)}}.authGrid{{display:grid;grid-template-columns:1fr;gap:10px}}.authGrid .wide{{grid-column:1/-1}}.msg{{margin-top:10px;color:#bbf7d0;font-weight:750}}.msg.bad{{color:#fecdd3}}.formMsg{{margin:-8px 0 18px;padding:10px 12px;border:1px solid rgba(34,197,94,.34);border-radius:8px;background:rgba(34,197,94,.12);color:#bbf7d0;font-weight:800}}.formMsg.bad{{border-color:rgba(251,113,133,.4);background:rgba(251,113,133,.13);color:#fecdd3}}
|
||||
.authGroup{{margin-top:14px;border:1px solid rgba(34,211,238,.22);border-radius:14px;background:linear-gradient(180deg,rgba(255,255,255,.06),rgba(255,255,255,.03)),rgba(15,10,26,.78);box-shadow:inset 0 1px 0 rgba(255,255,255,.04)}}.authGroupSummary{{display:flex;align-items:center;justify-content:space-between;gap:12px;padding:12px 14px;cursor:pointer;list-style:none}}.authGroupSummary::-webkit-details-marker{{display:none}}.authGroupTitle strong{{display:block;color:#f7f2ff;font-size:15px;font-weight:900;line-height:1.15}}.authGroupTitle span{{display:block;margin-top:4px;color:var(--muted);font-size:12px;line-height:1.35}}.authGroupChevron{{display:inline-flex;align-items:center;justify-content:center;flex:0 0 28px;width:28px;height:28px;border:1px solid rgba(34,211,238,.24);border-radius:999px;background:rgba(34,211,238,.08);color:#dbeafe;font-size:16px;line-height:1;transition:transform .18s ease,background .18s ease,border-color .18s ease}}.authGroup[open] .authGroupChevron{{transform:rotate(180deg);background:rgba(34,197,94,.12);border-color:rgba(34,197,94,.24)}}.authGroupBody{{padding:0 14px 14px;border-top:1px solid rgba(255,255,255,.08)}}.authGroupBody>.authPasswordSection{{margin-top:14px;padding-top:0;border-top:0}}.authGroupBody>.authOverview{{margin-top:0;padding-top:14px;border-top:0}}.authGroupBody>.sessionBox{{margin-top:14px;padding-top:0;border-top:0}}.authGroupBody>.notifyBox{{margin-top:14px;padding-top:0;border-top:0}}.authGroupBody>.backupBox{{margin-top:14px;padding-top:0;border-top:0}}.authOverview{{display:grid;gap:10px;margin-top:14px;padding-top:12px;border-top:1px solid var(--line)}}.authPills{{display:flex;flex-wrap:wrap;gap:8px}}.authPill{{display:inline-flex;align-items:center;gap:7px;padding:7px 10px;border:1px solid rgba(34,211,238,.24);border-radius:999px;background:rgba(34,211,238,.08);color:#dbeafe;font-size:12px;font-weight:850}}.authPill.off{{border-color:rgba(255,255,255,.10);background:rgba(255,255,255,.06);color:#c4b5fd}}.authSection{{margin-top:14px;padding-top:12px;border-top:1px solid var(--line)}}.authSectionHead{{display:flex;align-items:flex-start;justify-content:space-between;gap:10px;margin-bottom:8px}}.authSectionHead h3{{margin:0;font-size:15px}}.authSectionHead p{{margin:4px 0 0;color:var(--muted);font-size:12px;line-height:1.35}}.authSectionState{{display:inline-flex;align-items:center;justify-content:center;min-height:30px;padding:6px 10px;border:1px solid rgba(34,197,94,.30);border-radius:999px;background:rgba(34,197,94,.12);color:#bbf7d0;font-size:12px;font-weight:850;white-space:nowrap}}.authSectionState.off{{border-color:rgba(255,255,255,.10);background:rgba(255,255,255,.06);color:#c4b5fd}}.authSectionState.warn{{border-color:rgba(251,191,36,.28);background:rgba(251,191,36,.11);color:#fde68a}}.authFields{{display:grid;grid-template-columns:1fr 1fr;gap:8px}}.authFields .wide{{grid-column:1/-1}}.authFields input,.authFields textarea{{width:100%;min-width:0}}.authFields textarea{{min-height:92px;resize:vertical}}.authActions{{display:flex;gap:8px;flex-wrap:wrap;margin-top:8px}}.authActions .sessionBtn,.authActions .btn,.authActions button{{flex:1 1 180px}}.authHint{{margin:8px 0 0;color:var(--muted);font-size:12px;line-height:1.35}}.authRememberToggle{{display:inline-flex;align-items:center;gap:8px;min-height:34px;padding:7px 11px;border:1px solid rgba(255,255,255,.10);border-radius:999px;background:rgba(255,255,255,.05);color:var(--muted);font-size:12px;font-weight:750;cursor:pointer;user-select:none}}.authRememberToggle input{{width:14px;height:14px;margin:0;flex:0 0 14px;accent-color:#22c55e}}.authSecretBox{{margin-top:10px;padding:10px;border:1px solid rgba(34,211,238,.24);border-radius:8px;background:rgba(34,211,238,.08)}}.authSecretBox strong{{display:block;margin-bottom:6px;font-size:12px;color:#f7f2ff}}.authSecretValue{{margin:0;padding:9px 10px;border:1px solid rgba(255,255,255,.08);border-radius:8px;background:rgba(0,0,0,.18);color:#c4b5fd;font:12px/1.4 ui-monospace,SFMono-Regular,Consolas,monospace;white-space:pre-wrap;word-break:break-word}}.authList{{display:grid;gap:8px;margin-top:10px}}.authRow{{display:grid;grid-template-columns:1fr auto;gap:8px;align-items:start;border:1px solid var(--line);border-radius:8px;background:rgba(255,255,255,.055);padding:9px}}.authRowAll{{border-style:dashed;background:linear-gradient(135deg,rgba(34,211,238,.10),rgba(34,197,94,.08),rgba(255,255,255,.04))}}.authRowTitle{{display:flex;align-items:center;gap:8px;flex-wrap:wrap;font-weight:900}}.authRowMeta{{margin-top:4px;color:var(--muted);font-size:12px;line-height:1.35;word-break:break-word}}.authRowKey{{margin-top:7px;padding:8px;border:1px solid rgba(255,255,255,.08);border-radius:7px;background:rgba(0,0,0,.18);color:#c4b5fd;font:11px/1.35 ui-monospace,SFMono-Regular,Consolas,monospace;white-space:pre-wrap;word-break:break-all}}.authRouterFlags{{justify-content:flex-end}}.authFlagPill{{gap:6px;min-height:30px;padding:5px 8px;cursor:pointer;user-select:nonLine truncated
|
||||
.sessionBox{{margin-top:14px;padding-top:12px;border-top:1px solid var(--line)}}.sessionHead{{display:flex;align-items:center;justify-content:space-between;gap:10px;margin-bottom:8px}}.sessionHead h3{{margin:0;font-size:15px}}.sessionList{{display:grid;gap:8px;max-height:260px;overflow:auto;padding-right:2px}}.sessionRow{{display:grid;grid-template-columns:1fr auto;gap:8px;align-items:center;border:1px solid var(--line);border-radius:8px;background:rgba(255,255,255,.055);padding:9px;text-align:left}}.sessionTitle{{display:flex;gap:7px;align-items:center;flex-wrap:wrap;font-weight:900}}.sessionMeta{{margin-top:3px;color:var(--muted);font-size:12px;line-height:1.35;word-break:break-word}}.sessionCurrent{{border:1px solid rgba(34,197,94,.38);border-radius:999px;padding:2px 7px;color:#bbf7d0;background:rgba(34,197,94,.13);font-size:11px}}.sessionBtn{{padding:7px 9px;font-size:12px;border-radius:999px}}.sessionEmpty{{padding:10px;border:1px dashed var(--line);border-radius:8px;color:var(--muted);text-align:center}}
|
||||
.notifyBox{{margin-top:14px;padding-top:12px;border-top:1px solid var(--line)}}.notifyActions{{display:flex;gap:7px;align-items:center;justify-content:flex-end;flex-wrap:wrap}}.notifyHint{{margin:-4px 0 10px;color:var(--muted);font-size:12px;line-height:1.35}}.notifyList{{display:grid;gap:8px;max-height:260px;overflow:auto;padding-right:2px}}.notifyRow{{border:1px solid var(--line);border-radius:8px;background:rgba(255,255,255,.055);padding:9px;text-align:left}}.notifyTitle{{display:flex;align-items:center;justify-content:space-between;gap:8px;font-weight:950}}.notifyTitle span:first-child{{min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}}.notifyTime{{color:var(--muted);font-size:11px;font-weight:750;white-space:nowrap}}.notifyBody{{margin-top:4px;color:#ddd6fe;font-size:12px;line-height:1.35;word-break:break-word}}.notifyDetails{{margin:7px 0 0;padding:8px;border:1px solid rgba(255,255,255,.08);border-radius:7px;background:rgba(0,0,0,.18);color:#c4b5fd;font:11px/1.35 ui-monospace,SFMono-Regular,Consolas,monospace;white-space:pre-wrap;max-height:92px;overflow:auto}}.notifyRow.warn{{border-color:rgba(245,158,11,.34);background:rgba(245,158,11,.08)}}.notifyRow.bad{{border-color:rgba(251,113,133,.38);background:rgba(251,113,133,.09)}}.notifyBtn.on{{border-color:rgba(34,197,94,.36);background:rgba(34,197,94,.15);color:#bbf7d0}}
|
||||
.backupBox{{margin-top:14px;padding-top:12px;border-top:1px solid var(--line)}}.backupGrid{{display:grid;grid-template-columns:1fr 1fr;gap:8px}}.backupGrid .wide{{grid-column:1/-1}}.backupGrid input[type=file]{{padding:8px;background:rgba(8,5,18,.72);border:1px solid var(--line);border-radius:8px;color:#ddd6fe;min-width:0}}.backupHint{{margin:0 0 10px;color:var(--muted);font-size:12px;line-height:1.35}}.backupCmds{{display:grid;gap:8px;margin-top:10px}}.backupCmd{{border:1px solid rgba(255,255,255,.10);border-radius:8px;background:rgba(0,0,0,.18);padding:9px}}.backupCmd strong{{display:block;margin-bottom:5px;color:#f7f2ff;font-size:12px}}.backupCmd pre{{margin:0;white-space:pre-wrap;word-break:break-word;color:#c4b5fd;font:11px/1.35 ui-monospace,SFMono-Regular,Consolas,monospace}}#backupRewrite{{border-color:rgba(251,191,36,.42);background:linear-gradient(135deg,rgba(251,191,36,.28),rgba(245,158,11,.22));color:#fff7cc;box-shadow:0 10px 24px rgba(245,158,11,.14),inset 0 1px 0 rgba(255,255,255,.12)}}#backupRewrite:hover{{border-color:rgba(251,191,36,.56);background:linear-gradient(135deg,rgba(251,191,36,.34),rgba(245,158,11,.28))}}.backupMsg{{margin-top:9px;color:#bbf7d0;font-size:12px;font-weight:800;line-height:1.35}}.backupMsg.bad{{color:#fecdd3}}
|
||||
.backupBox{{margin-top:14px;padding-top:12px;border-top:1px solid var(--line)}}.backupGrid{{display:grid;grid-template-columns:1fr 1fr;gap:8px}}.backupGrid .wide{{grid-column:1/-1}}.backupGrid input[type=file]{{padding:8px;background:rgba(8,5,18,.72);border:1px solid var(--line);border-radius:8px;color:#ddd6fe;min-width:0}}.backupHint{{margin:0 0 10px;color:var(--muted);font-size:12px;line-height:1.35}}#backupRewrite{{border-color:rgba(251,191,36,.42);background:linear-gradient(135deg,rgba(251,191,36,.28),rgba(245,158,11,.22));color:#fff7cc;box-shadow:0 10px 24px rgba(245,158,11,.14),inset 0 1px 0 rgba(255,255,255,.12)}}#backupRewrite:hover{{border-color:rgba(251,191,36,.56);background:linear-gradient(135deg,rgba(251,191,36,.34),rgba(245,158,11,.28))}}.backupMsg{{margin-top:9px;color:#bbf7d0;font-size:12px;font-weight:800;line-height:1.35}}.backupMsg.bad{{color:#fecdd3}}
|
||||
input,select{{min-width:0;border:1px solid var(--line);border-radius:8px;padding:10px 11px;background:rgba(8,5,18,.72);color:var(--text)}}button,.btn{{border:1px solid rgba(255,255,255,.10);border-radius:8px;padding:10px 13px;background:rgba(255,255,255,.10);color:#f7f2ff;font-weight:850;text-decoration:none;cursor:pointer;display:inline-flex;justify-content:center;align-items:center}}.authToggle{{border-radius:999px;padding:8px 14px;background:rgba(255,255,255,.08);color:#f3e8ff}}button.primary,.btn.primary{{background:var(--blue);color:#fff;box-shadow:0 10px 22px rgba(124,58,237,.22)}}button.bad,.btn.bad{{background:rgba(251,113,133,.16);color:#fecdd3}}.btn.good{{background:rgba(34,197,94,.16);color:#bbf7d0}}.btn.disabled{{opacity:.45;cursor:not-allowed}}
|
||||
.summary{{display:flex;gap:10px;flex-wrap:wrap;justify-content:flex-end}}.miniStat{{display:inline-flex;align-items:center;justify-content:center;gap:8px;min-height:36px;min-width:132px;border:1px solid var(--line);border-radius:999px;background:rgba(255,255,255,.07);padding:8px 12px;color:#ddd6fe;font-weight:800;font-size:13px;line-height:1;white-space:nowrap}}
|
||||
.routerStats{{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));align-items:start;gap:8px;margin:0 0 10px}}.statCard{{position:relative;min-height:54px;overflow:hidden;border:1px solid var(--line);border-radius:8px;padding:8px 10px 7px;background:linear-gradient(180deg,rgba(255,255,255,.075),rgba(255,255,255,.04)),var(--panel);box-shadow:0 8px 20px rgba(0,0,0,.14);text-align:center;box-sizing:border-box}}.statCard::before{{content:"";position:absolute;inset:0 0 auto 0;height:2px;background:var(--cyan)}}.statCard span{{display:block;color:var(--muted);font-size:10px;font-weight:850;text-transform:uppercase;letter-spacing:.035em}}.statCard strong{{display:block;margin-top:0;font-size:22px;line-height:1;color:#f7f2ff}}.statCard em{{display:block;min-width:0;margin-top:2px;padding:0 2px;color:#c4b5fd;font-style:normal;font-size:10px;line-height:1.15}}.statCard.online::before{{background:var(--green)}}.statCard.online strong{{color:#bbf7d0}}.statCard.offline::before{{background:var(--red)}}.statCard.offline strong{{color:#fecdd3}}.statCard.total::before{{background:var(--cyan)}}.statCard.total strong{{color:#a5f3fc}}.statCardHead{{display:flex;align-items:center;justify-content:center;min-height:14px}}.statCard.has-popover{{overflow:visible;z-index:12}}.statValueRow,.offlineStatRow{{position:relative;display:flex;align-items:center;justify-content:center;min-height:20px;margin-top:3px}}.offlineMoreBtn{{position:absolute;right:6px;top:50%;transform:translateY(-50%);display:inline-flex;align-items:center;justify-content:center;min-height:22px;padding:0 8px;border:1px solid rgba(251,113,133,.34);border-radius:999px;background:rgba(251,113,133,.12);color:#fecdd3;font-size:10px;font-weight:900;line-height:1;cursor:pointer;white-space:nowrap}}.offlineMoreBtn[aria-expanded="true"]{{background:rgba(251,113,133,.18);border-color:rgba(251,113,133,.48)}}.offlinePopover{{position:absolute;top:calc(100% + 8px);right:6px;width:min(340px,calc(100vw - 30px));padding:10px;border:1px solid rgba(251,113,133,.28);border-radius:12px;background:linear-gradient(180deg,rgba(49,28,44,.98),rgba(30,18,40,.98));box-shadow:0 22px 50px rgba(0,0,0,.34);text-align:left;z-index:25}}.offlinePopover[hidden]{{display:none!important}}.offlineList{{display:grid;gap:6px;max-height:170px;overflow:auto;padding-right:2px}}.offlineItem{{border:1px solid rgba(251,113,133,.18);border-radius:8px;padding:7px 8px;background:rgba(251,113,133,.08)}}.offlineName{{display:block;color:#fdf2f8;font-size:11px;font-weight:900;line-height:1.25;text-transform:none;letter-spacing:0}}
|
||||
@@ -5022,7 +5065,7 @@ input,select{{min-width:0;border:1px solid var(--line);border-radius:8px;padding
|
||||
<div class="brand">
|
||||
<div class="desktopHeader">
|
||||
<div class="desktopHeaderTop">
|
||||
<h1 class="appBanner"><span>OpenWrt Remote Hub <span class="appBannerVersion">v116</span></span></h1>
|
||||
<h1 class="appBanner"><span>OpenWrt Remote Hub <span class="appBannerVersion">v117</span></span></h1>
|
||||
<div class="routerSearchDock" id="routerSearchDock">
|
||||
<button class="routerSearchToggle" id="routerSearchToggle" type="button" aria-expanded="false" aria-controls="routerSearchPanel" data-active="false">
|
||||
<span>Поиск роутеров</span>
|
||||
@@ -5334,29 +5377,6 @@ input,select{{min-width:0;border:1px solid var(--line);border-radius:8px;padding
|
||||
<a class="sessionBtn btn good" id="backupDownload" href="/api/backup/download">Скачать</a>
|
||||
</div>
|
||||
<div class="backupHint">Архив переносит БД роутеров, логин, токены Hub/агента, push-ключи и уведомления. Для переезда на другой IP укажи новый VPS host/Public URL перед восстановлением.</div>
|
||||
<div class="backupCmds">
|
||||
<div class="backupCmd">
|
||||
<strong>1. Сделать backup на старой VPS</strong>
|
||||
<pre>/opt/owrt-remote/owrt-remote-hub.py backup --out /root/owrt-hub-backup.tar.gz</pre>
|
||||
</div>
|
||||
<div class="backupCmd">
|
||||
<strong>2. Поднять Hub на новой VPS</strong>
|
||||
<pre>curl -fsSL "https://raw.githubusercontent.com/kzolotarev95/luci-app-owrt-remote/main/vps/install-vps.sh?v=$(date +%s)" | sh</pre>
|
||||
</div>
|
||||
<div class="backupCmd">
|
||||
<strong>3. Перенести архив на новую VPS</strong>
|
||||
<pre>scp /root/owrt-hub-backup.tar.gz root@NEW_IP:/root/owrt-hub-backup.tar.gz</pre>
|
||||
</div>
|
||||
<div class="backupCmd">
|
||||
<strong>4. Восстановить на новой VPS с новым IP/domain</strong>
|
||||
<pre>/opt/owrt-remote/owrt-remote-hub.py restore --file /root/owrt-hub-backup.tar.gz --vps-host NEW_IP_OR_DOMAIN --public-url https://NEW_DOMAIN</pre>
|
||||
</div>
|
||||
<div class="backupCmd">
|
||||
<strong>5. Применить после restore</strong>
|
||||
<pre>systemctl restart owrt-remote
|
||||
systemctl restart owrt-remote-xray</pre>
|
||||
</div>
|
||||
</div>
|
||||
<div class="backupGrid">
|
||||
<input class="wide" id="backupFile" type="file" accept=".gz,.tgz,.tar.gz,application/gzip,application/x-gzip">
|
||||
<input id="backupVpsHost" placeholder="Новый VPS host/IP">
|
||||
@@ -5375,7 +5395,7 @@ systemctl restart owrt-remote-xray</pre>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mobileSearchDock" id="mobileRouterSearchDock">
|
||||
<span class="mobileSearchVersion">v116</span>
|
||||
<span class="mobileSearchVersion">v117</span>
|
||||
<button class="routerSearchToggle mobilePanelToggle primary" id="mobileRouterSearchToggle" type="button" aria-expanded="false" aria-controls="mobileRouterSearchPanel" data-active="false">
|
||||
<span>Поиск роутеров</span>
|
||||
</button>
|
||||
@@ -12203,10 +12223,10 @@ def router_logs_html(router):
|
||||
@media(max-width:680px),(pointer:coarse){body{background-attachment:scroll;background-image:linear-gradient(145deg,#07040f,#120a24 54%,#05030a)}.legend{justify-content:center;text-align:center;row-gap:8px}}
|
||||
</style></head><body><main class="wrap"><section class="panel">
|
||||
<div class="head"><h1>Глобал логи</h1><a class="btn" href="/">Вернуться в Hub</a></div><h2>__TITLE__</h2>
|
||||
<blockquote class="note">Действия панели, Heartbeat, WAN, предупреждения и ошибки Xray этого роутера. Копия доставляется на VPS каждые 3 секунды, пока доступна связь. Если роутер оффлайн, доступны последние доставленные записи. На роутере журнал хранится только в ОЗУ и очищается после перезагрузки и каждые 24 часа. Архив VPS сохраняется отдельно: до 2 МБ на роутер. Ключи и пароли скрываются.</blockquote>
|
||||
<blockquote class="note">Действия панели, Heartbeat, WAN, предупреждения и ошибки Xray этого роутера. Доставка проверяется каждые 3 секунды; новые записи обычно появляются после пакетной записи на Flash, до 60 секунд. На обновлённом роутере журнал сохраняется на проверенной Flash, временный буфер ограничен 16 КиБ. Если роутер оффлайн, доступны последние доставленные записи. Архив VPS хранится в SQLite на диске: до 2 МиБ на роутер. Ключи и пароли скрываются.</blockquote>
|
||||
<div class="legend" aria-label="Цвета журнала"><span class="ok">● Успешно</span><span class="warn">● Предупреждение</span><span class="error">● Ошибка</span></div>
|
||||
<div class="actions"><a class="btn" href="__DOWNLOAD__">Скачать логи</a><button class="btn" id="logsLive" type="button">Запустить в реальном времени</button><button class="btn bad" id="logsClear" type="button">Очистить логи</button></div>
|
||||
<div id="logsDiagnosis" class="warn" role="status">Получаю состояние роутера…</div><div id="logsStatus" role="status">Загружаю сохранённые записи с VPS…</div><pre id="logsText" tabindex="0">Загрузка записей журнала…</pre>
|
||||
<div id="logsDiagnosis" class="warn" role="status">Получаю состояние роутера…</div><div id="logsStorage" role="status"></div><div id="logsStatus" role="status">Загружаю сохранённые записи с VPS…</div><pre id="logsText" tabindex="0">Загрузка записей журнала…</pre>
|
||||
</section></main><script>
|
||||
(function(){
|
||||
const endpoint=__ENDPOINT__, output=document.getElementById('logsText'), status=document.getElementById('logsStatus'), diagnosis=document.getElementById('logsDiagnosis'), live=document.getElementById('logsLive'), clear=document.getElementById('logsClear');
|
||||
@@ -12231,6 +12251,8 @@ def router_logs_html(router):
|
||||
const data=await response.json();if(current!==generation)return;
|
||||
render(data.text||'');
|
||||
showDiagnosis(data);
|
||||
const storage=data.storage, storageLabel=document.getElementById('logsStorage');
|
||||
if(storage){storageLabel.className=storage.persistent===false?'error':storage.persistent===true?'ok':'warn';storageLabel.textContent=(storage.persistent===false?'Внимание: архив VPS находится в RAM и исчезнет после перезагрузки.':storage.persistent===true?'Архив VPS: постоянный диск.':'Архив VPS: тип хранилища не подтверждён.')+' '+storage.path+' · '+storage.filesystem+' · Лимит: 2 МиБ на роутер.';}
|
||||
status.textContent=(data.online?'Роутер онлайн':'Роутер оффлайн — показан архив VPS')+' · Последняя доставка: '+stamp(data.last_sync)+' · Объём архива: '+Math.ceil(data.bytes/1024)+' КБ'+(data.bytes>65536?' · На экране последние 64 КБ; полный журнал доступен через «Скачать логи».':'')+(data.pending_clear?' · Очистка журнала роутера ожидает связи.':'');
|
||||
}catch(error){if(current===generation)status.textContent='Не удалось получить логи: '+error.message+(running?'. Повтор через 3 секунды.':'');}
|
||||
finally{clearTimeout(timeout);if(controller===request)controller=null;if(current===generation&&running&&!document.hidden)timer=setTimeout(refresh,3000);}
|
||||
@@ -12815,7 +12837,7 @@ button:hover{{filter:brightness(1.06)}}
|
||||
<form class="login" method="post" action="/login">
|
||||
{error_html}
|
||||
<span class="brand">
|
||||
<h1 class="appBanner"><span>OpenWrt Remote Hub <span class="appBannerVersion">v116</span></span></h1>
|
||||
<h1 class="appBanner"><span>OpenWrt Remote Hub <span class="appBannerVersion">v117</span></span></h1>
|
||||
</span>
|
||||
<label for="hubUsername">Логин</label>
|
||||
<input id="hubUsername" name="username" autocomplete="off" autofocus required>
|
||||
@@ -13287,7 +13309,7 @@ body::after{{content:"";position:fixed;inset:0;pointer-events:none;background:li
|
||||
<circle cx="65" cy="59" r="3" fill="#E5F2FF"/>
|
||||
</svg>
|
||||
</div>
|
||||
<h2 class="brandTitle">OpenWrt Remote Hub <span class="brandVersion">v116</span></h2>
|
||||
<h2 class="brandTitle">OpenWrt Remote Hub <span class="brandVersion">v117</span></h2>
|
||||
</div>
|
||||
</div>
|
||||
<div class="brandBottom">
|
||||
@@ -15852,7 +15874,8 @@ a{{display:inline-flex;margin-top:18px;color:#93c5fd}}
|
||||
self.send_bytes(200, raw, 'text/plain; charset=utf-8',
|
||||
[('Cache-Control', 'no-store'), ('Content-Disposition', f'attachment; filename="{filename}"')])
|
||||
else:
|
||||
payload = {'ok': True, 'router_id': router_id, 'online': router['online'], **snapshot}
|
||||
payload = {'ok': True, 'router_id': router_id, 'online': router['online'],
|
||||
'storage': router_log_storage(self.app.db_path), **snapshot}
|
||||
if mode == 'clear':
|
||||
payload['message'] = 'Архив VPS очищен. Журнал роутера будет очищен при следующем соединении.'
|
||||
self.send_json(200, payload)
|
||||
|
||||
Reference in new issue
Block a user