scrapper

A bus driver's own instance: it reads that driver's roster out of their employer's planning system, serves it to their phone, and — only if they turn it on — publishes it to friends they choose, end-to-end encrypted, over the Glong protocol.

One instance, one driver. Nobody operates it but them.

The point is the credentials. Reading a roster means logging into the employer's site as the driver, so a shared service would have to hold every user's work login. Here those credentials sit on a box the driver controls and are sent to exactly one place: the site they belong to.

How it works

employer's planning site
        │  scheduled scrape (05:00 + hourly for today/tomorrow)
        ▼
   PlanningSource  ──►  planning_legs  ──►  ShiftSegmentAssembler
   (one driver per                              │
    rostering vendor)                           ├──►  GET /v1/shift        → the driver's app
                                                ├──►  APNs Live Activity   → the driver's lock screen
                                                └──►  Glong publisher      → friends, encrypted

Everything under /v1 reads from the database. Nothing scrapes on request: a scrape logs in upstream and walks two weeks, so it belongs to the scheduler, not to a page load.

Requirements

• PHP 8.5, with ext-pdo and ext-sodium (Glong signing/sealing)
• Redis (queue + cache)
• SQLite by default; any database Laravel supports works
• A scheduler and a queue worker (see Running it)

No frontend build, no Node: this instance serves JSON and nothing else.

Install

git clone <this repo> && cd scrapper
composer setup          # install, .env, app key, migrate

Then fill in .env — at minimum PLANNING_BASE_URL, PLANNING_USERNAME and PLANNING_PASSWORD. php artisan env:check tells you if your .env has drifted from .env.example.

php artisan planning:scrape          # first scrape: today + 14 days
php artisan planning:scrape today tomorrow   # or a window
curl localhost:8000/api/v1/shift     # what the app will see

Running it

Three processes:

php artisan schedule:work            # or a crontab entry for schedule:run
php artisan queue:work               # scrapes and publishes triggered from the API
php artisan serve                    # in production: nginx/caddy + php-fpm

Without the scheduler nothing scrapes. The schedule (routes/console.php):

When	What
05:00 daily	Full scrape, today + 14 days
Hourly, 04:00–23:59 Europe/Paris, on a jittered minute	Re-scrape today and tomorrow — the upstream site pushes roster changes only to browsers, so this instance re-reads instead
Every minute	live-activity:push, so Live Activity updates land on segment boundaries

The jitter is derived from the hour and the app key: stable within an hour (the scheduler re-reads the file every minute) but not a machine-looking fixed tick upstream.

Behind a reverse proxy, set TRUSTED_PROXIES — otherwise every throttle keys on the proxy's address instead of the caller's.

The roster API

There is no authentication. One instance serves one driver and is expected to sit somewhere only that driver can reach — a home network, a Tailscale address, a VPS behind a firewall. Throttles protect the upstream site and this box, not an identity. Do not put a real instance on the open internet; a demo-mode one holds nothing to protect and is meant to be reachable.

GET /v1/shift returns the current-or-next duty as an ordered, contiguous ShiftSegment[]: breaks materialised as explicit pause segments, a planned free day as a single rest segment, a day whose roster is not published yet as [].

That shape is fixed by glong-relay/glong-protocol.md §6 and is shared with the driver app and with Glong readers, so it is not this repo's to change unilaterally — see §6.2 there.

Route	Purpose
GET /v1/shift	The ShiftSegment[] contract.
GET /v1/message	Day-level notice shown above the legs.
GET /v1/documents	Documents listed on a day's planning.
GET /v1/documents/{id}	Streamed live from upstream with the server's session; never stored here.
POST /v1/live-activity/tokens	APNs token registration.
GET/PATCH /v1/glong	Sharing on/off and this instance's identity.
GET/POST/DELETE /v1/glong/friends	Friend management for the app's settings screen.
POST /v1/planning/scrape	Out-of-band refresh; queues the scheduled job.

OpenAPI lives at /docs/api (and /docs/api.json), generated from the actions.

Demo mode

App-store review and screenshots need a working server, not a real driver's work login. PLANNING_SOURCE=demo swaps the scraper for one that invents the roster:

PLANNING_SOURCE=demo   # PLANNING_BASE_URL / USERNAME / PASSWORD stay empty
php artisan planning:scrape

It reaches no network and reads no credential. Every day is generated from its own date, so any day answers instantly and answers the same way twice — a screenshot taken today still matches the app tomorrow, and re-scraping updates the same rows instead of duplicating them.

What a reviewer sees: an early or late duty on alternating days, two to four commercial runs with lines, girouette codes and itineraries, a break between them (materialised as a pause, so that state is reachable), a deadhead back to the depot, and Sunday as a rest day. Documents list one entry and download as a generated PDF. Places, lines, plates and the driver's name are invented.

A demo instance is the one deployment that can safely face the internet: there is no roster and no credential on it.

Glong sharing

Off by default. When enabled, every scrape takes the roster from today on (every day scraped, about two weeks), projects it down to the level granted per friend, encrypts and signs it, and publishes one envelope per friend to a relay. A friend receives as many leading days as fit the relay's envelope limit (GLONG_MAX_ENVELOPE_BYTES), whole days only: a full friend sees a few days ahead, a line-times one the whole fortnight. The relay stores ciphertext under an opaque stream id and can decrypt nothing; the author is inside the ciphertext, not on the envelope.

php artisan glong:identity                       # this instance's id + fingerprint (generated on first run)
php artisan glong:friend:add glong:z… --name=Bob --level=line-times
php artisan glong:publish                        # publish the roster from today on, on change only
php artisan glong:retract                        # withdraw every envelope the relay still holds

Share levels are full, line-times (default and recommended) and free-busy. What each level keeps is specified in §6.1 — including the deliberate calls: a commercial run's terminus survives at line-times because it is written on the front of the bus, while origin never does; and a rest day is withheld entirely at free-busy rather than published as a busy block.

The id to hand a friend is this instance's, not the one the phone shows. They are different keypairs, and pairing against the wrong one fails silently — the friend derives a real stream nobody writes to and sees nothing, forever, with no error anywhere.

Turning sharing off retracts every envelope and clears the stored versions, so turning it back on republishes.

Configuration

Beyond the usual Laravel keys:

Env	Default	Meaning
PLANNING_SOURCE	abc	Which rostering system to read from. demo invents one — see Demo mode.
PLANNING_BASE_URL	(empty)	Upstream roster site.
PLANNING_USERNAME / PLANNING_PASSWORD	(empty)	The driver's own upstream login. Sent nowhere but upstream.
TRUSTED_PROXIES	(empty)	Comma-separated IPs/CIDRs, or *. Empty trusts none.
APNS_KEY_ID, APNS_TEAM_ID, APNS_KEY_PATH or APNS_KEY_CONTENT, APNS_BUNDLE_ID, APNS_SANDBOX	—	Live Activity push. Leave empty to disable pushing.
GLONG_ENABLED	false	Publish to the relay after each scrape.
GLONG_RELAY_URL	(empty)	Base URL of the relay.
GLONG_MAX_ENVELOPE_BYTES	65536	The relay's envelope size limit; days are dropped from the far end until the envelope fits.
GLONG_HANDLE	(empty)	Advisory handle; the reference relay has no directory, so it is inert today.
GLONG_IDENTITY_DISK / GLONG_IDENTITY_PATH	local / glong/identity.json	Where the identity keypair lives. Kept off the database so it never lands in a dump. Back it up — losing it means re-pairing with every friend.

Adding a rostering system

abc is one implementation of App\Services\Planning\PlanningSource, not the architecture. To support another employer's system:

1. Write a driver in app/Services/Planning/Drivers/ implementing PlanningSource — login(), scrapeDay(), documents(), message(), fetchDocument().
2. Add a createYourVendorDriver() method to PlanningManager.
3. Add its credentials under planning.sources.your-vendor and set PLANNING_SOURCE.

Nothing downstream changes: storage, assembly, the API and Glong all work off the returned rows.

One caveat, stated in PlanningSource's docblock: the ScrapedLeg shape still speaks the first vendor's vocabulary — leg ids are idserv, and type uses its REG/TA/ABS_* codes, which ShiftSegmentAssembler maps to a ShiftKind. A second driver translates into those names. When one genuinely cannot, that mapping is what should move behind the interface — and a real second driver is the right thing to design it against.

Development

composer test        # 100% type coverage, tests (min 90% line coverage), lint, phpstan
composer lint        # Rector then Pint
php artisan test --compact --filter=Glong

Static analysis runs at PHPStan level max. Tests are Pest; every behaviour change is expected to come with one.

Conventions worth knowing before a first patch:

• Actions (app/Actions/V1) are single-purpose classes; routes point straight at them.
• Comments explain why, not what. A ponytail: comment marks a deliberate shortcut and names its upgrade path.
• Times upstream are Europe/Paris wall-clock; the database stores UTC. No duty crosses midnight.

What is stored, and where

Thing	Where	Note
Roster legs, documents index, day notices	Database	Rewritten on every scrape
Upstream credentials	.env	Used only against the roster site
Glong identity keypair	Storage disk (not the DB)	Back it up
Friends' public keys and share levels	Database	Pinned out of band, never trusted from the relay
Documents (the files)	Nowhere	Proxied live, streamed through, never written to disk

License

GNU AGPL v3 or later — see LICENSE.

Copyright © 2026 Nicolas Perraut.

The network clause is the point: run a modified version as a service and its users are entitled to that version's source. A driver hosting this for themselves owes nobody anything; someone offering it to other drivers has to hand them the code they are actually running.