Evan Hu
f9821d5867
Walking ~40+ plugin TOMLs via the GitHub Contents API from the worker exceeded the Workers Free 50-subrequest-per-invocation limit, leaving the daemon's signed plugins index either empty or partial after every forced refresh. Move the walk into the repo: scripts/build-plugins-index.mjs reads each plugins/<name>/plugin.toml directly from the checked-out tree and emits a sorted flat array (name, version?, description?, needs?) at plugins-index.json. The CI workflow regenerates and commits this file on every push under plugins/, then pokes the worker's /api/registry/refresh — which now fetches the single committed plugins-index.json (1 subrequest), validates the JSON shape, and re-signs it with Ed25519. Refresh cost is now constant in registry size, not linear. The dashboard's dict-shaped /api/registry payload is unchanged — that still rebuilds via the daily 02:00 UTC cron.
68 lines
2.3 KiB
YAML
68 lines
2.3 KiB
YAML
name: Refresh registry-worker cache
|
|
|
|
# On every push that changes plugins/, regenerate plugins-index.json
|
|
# (the daemon-shaped flat array the registry-worker signs and the
|
|
# librefang daemon installs from), commit it back, then poke the
|
|
# worker's forced-refresh endpoint so it re-signs the new bytes
|
|
# immediately — without this, daemon installs lag up to ~24h behind
|
|
# the next 02:00 UTC cron tick.
|
|
#
|
|
# Walking the on-disk repo (instead of the worker hitting GitHub
|
|
# Contents API per-file) keeps the worker's refresh path under the
|
|
# Workers Free 50-subrequest budget regardless of registry size.
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- 'plugins/**'
|
|
- 'scripts/build-plugins-index.mjs'
|
|
workflow_dispatch: # manual trigger for ops / first-deploy
|
|
|
|
permissions:
|
|
contents: write # needed to commit the regenerated index back
|
|
|
|
jobs:
|
|
refresh:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Rebuild plugins-index.json
|
|
run: node scripts/build-plugins-index.mjs
|
|
|
|
- name: Commit regenerated index if changed
|
|
run: |
|
|
git config user.name "github-actions[bot]"
|
|
git config user.email "github-actions[bot]@users.noreply.github.com"
|
|
git add plugins-index.json
|
|
if git diff --cached --quiet; then
|
|
echo "plugins-index.json already up-to-date"
|
|
else
|
|
git commit -m "chore: regenerate plugins-index.json"
|
|
git push
|
|
fi
|
|
|
|
- name: Trigger worker refresh
|
|
env:
|
|
REGISTRY_REFRESH_TOKEN: ${{ secrets.REGISTRY_REFRESH_TOKEN }}
|
|
run: |
|
|
if [ -z "$REGISTRY_REFRESH_TOKEN" ]; then
|
|
echo "::error::REGISTRY_REFRESH_TOKEN secret is not set on this repo"
|
|
exit 1
|
|
fi
|
|
response=$(curl -fsS -X POST \
|
|
-H "Authorization: Bearer $REGISTRY_REFRESH_TOKEN" \
|
|
-w "\nHTTP_CODE:%{http_code}" \
|
|
https://stats.librefang.ai/api/registry/refresh)
|
|
echo "$response"
|
|
code=$(echo "$response" | grep -oE 'HTTP_CODE:[0-9]+' | cut -d: -f2)
|
|
if [ "$code" != "200" ]; then
|
|
echo "::error::worker returned $code"
|
|
exit 1
|
|
fi
|