Cron, jobs and workers
Three ways to make the system work when nobody is watching: an HTTP cron, an internal job queue, a long-running process. They're for different things, and picking the wrong one costs.
1. Which of the three
| Need | Tool | Why |
|---|---|---|
| "every N minutes do X" | HTTP cron (§2) | needs no shell access: works on any hosting with a curl scheduler |
| "do X, but not now and not in front of the user" | job queue (§4) or Jobs::async | the user doesn't wait; the effect arrives shortly after |
| "always listen / poll an external service at sub-minute cadence" | CLI worker (§5) | cron has 1-minute granularity and pays the bootstrap on every run |
Rule of thumb: start with cron. A worker is a service to install, monitor and restart; introduce it when cron proves not to be enough, not before.
2. HTTP cron: a job is a page
Scheduled jobs are pages like any other (see 02 §1):
contents/cronjob/<name>.php → GET /cronjob/<name>?key=<secret>
The contents/cronjob/ folder already exists in the repo, empty: that's the intended place.
Pattern to copy:
<?php
use_cache(false);
// 1 — SECRET: the job is reachable over HTTP, so by anyone
$cronSecret = (string) (defined("JC_CRON_SECRET") ? JC_CRON_SECRET : "");
if ($cronSecret === "" || !hash_equals($cronSecret, (string)($_GET["key"] ?? ""))) {
http_response_code(403); echo "forbidden"; exit;
}
$db = ffDB_Sql::factory();
// 2 — LOCK: two overlapping runs are the norm, not the exception
$db->query("SELECT GET_LOCK('jc_coda', 0) AS l");
$db->nextRecord();
if ((int)$db->getField("l", "Number", true) !== 1) { echo "busy"; exit; }
try {
// 3 — limited BATCH: never "everything there is"
$limite = 200;
// ... work ...
// 4 — LOG the outcome
// INSERT INTO cm_jc_cron_log (job, esito, elementi, eseguito_il) VALUES (...)
} finally {
$db->query("SELECT RELEASE_LOCK('jc_coda')");
}
echo "ok";
exit;
The four safeguards, and why
- Secret in the query string compared with
hash_equals. A job without a secret is a public write endpoint. - MySQL
GET_LOCKwith timeout0. If the previous run hasn't finished, the new one exits immediately instead of doubling the work. One lock per job, not a global one: different jobs mustn't contend. - Limited batch with
LIMIT. A job trying to clear the whole backlog times out, and the timeout leaves it half-done. Better N short runs. - Log to a table with outcome and count. Without it, you can't answer "is the cron running?".
SELECT * FROM <log_table> ORDER BY ID DESC LIMIT 20should be enough.
Other tricks you learn in production
- Explicit skip if the migration isn't applied. A job running on an old schema must exit with "migration missing", not hit a fatal error every minute.
- Degrade, don't block. If an external provider doesn't answer, log it and retry on the next run.
- Quiet hours, if the job sends communications: don't send at 3 a.m.
- Off switch in configuration (
abilitatoon/off), which doesn't require a deploy. - Idempotency: the job may be re-run on the same interval (restart, double scheduling, scheduler retry). An anti-duplicate register (one row per
(recipient, type, period)with a unique key) is more reliable than "count and compare".
Scheduling
From the hosting panel (cron via curl):
*/5 * * * * curl -fsS "https://YOUR-DOMAIN/cronjob/<name>?key=<secret>" > /dev/null
Document in a table, in the repo, which jobs exist, how often they run and what happens if they don't: it's the information you need on the day you move to another hosting.
| Job | Purpose | Frequency | If it doesn't run |
|---|---|---|---|
| (none in the base repo) |
3. Inherited system jobs
conf/gallery/job/ contains four jobs from upstream VGallery:
| Job | What it does | Status |
|---|---|---|
ping | notifies search engines of the updated sitemap | legacy: several of the pinged services no longer exist |
update | queries the remote updater | inert: MASTER_SITE is neutralised |
diagnostic | checks cache, dictionary, config, uploads, thumbs | useful as a manual check |
reset_carts | cleans up abandoned carts | relevant only with e-commerce active |
They're not scheduled by default and write their outcome as a backend notification (write_notification). There's also an autoload-by-convention mechanism: check_function("job_<name>") loads library/gallery/job/<name>.php or modules/<module>/library/job/<name>.php (library/gallery/common.php, lines ~138-146).
4. The internal job queue (Jobs)
vendor/hcore/jobs/Jobs.php implements a persistent queue with scheduling and repetition.
Jobs::getInstance()->add($source, $params, $schedule, $repeat, $priority);
Jobs::getInstance()->run(); // runs due jobs
Jobs::getInstance()->ps(); // queue status
Jobs::getInstance()->forceRun($kid); // forces a job
Jobs::setScript($callback, $schedule, $repeat);
Executors per type: internal API, service, HTTP request, script, function, object (controller_api, controller_service, controller_request, controller_script, controller_func, controller_obj).
Guards in the class: SOCKET = 10, TIMEOUT = 60, ANTIFLOOD = 4 seconds between two runs, DELAY_SCHEDULE = 3600 for rescheduling after an error, REPEAT = "1week".
And the fire-and-forget primitive:
Jobs::async("/api/cache/refresh", array("url" => $_SERVER["HTTP_REFERER"]));
opens a socket to its own host, writes the request and doesn't wait for the response. In the repo it's used by library/gallery/system/trace.php and by the service bus (see 03 §4).
When to use it and when not.
Jobs::asyncis perfect for "refresh the cache after answering". For work that must get done — sends, charges, notifications — you need something that leaves a trace and can be retried: the persistent queue, or your own queue table drained by a cron. The fire-and-forget socket guarantees nothing: if the called process dies, nobody notices.
There's also library/rabbitMQ/ among the libraries: an external broker, not wired in this repo. A project that really needs it can hook it in, but for the vast majority of cases a queue table + cron is simpler to manage and easier to inspect.
5. Long-running worker (CLI)
When you need a sub-minute cadence, or in-memory state between runs, you leave the web: a PHP script in bin/ started by systemd (or by a fallback cron).
Lessons that apply to any worker on this stack:
- A dedicated CLI bootstrap. From the command line the page cycle doesn't run: classes come from the autoloader, but the functions in
ff/common.php(ffCommon_dirname,ffCommon_jsonenc) don't. They must be included explicitly. Don't includelibrary/gallery/common.php, which assumes acminstance. HTTP_HOSTmust be set beforeconfig.php: it decides whether to loadconfig.local.phporconfig.remote.php, and half the framework reads$_SERVERwithout checking keys.FF_DISK_PATHis the server path. The installer writes/var/www/…into it, which doesn't exist locally: a worker building paths on that constant fails off the server. Better an own constant derived from__DIR__, preferred when present.- Load only the module you need: the others hook events into the page cycle and from the CLI find no
cmto hook into. - Static caches must be discarded periodically. Configuration classes are designed for the lifetime of an HTTP request: in a process living an hour they'd stay frozen at the value read at start-up, and changing a threshold from the backend would have no effect.
- The worker exits by itself after a maximum time (e.g. an hour) and
Restart=alwaysbrings it back. An exit with code 0 every hour is the intended behaviour: it's the simplest way not to have a memory problem two months from now in a long-running PHP process. - It doesn't run as root: it needs to read
config.remote.php(0640, owned by the site user). Running as that user makes permissions work without widening them. - Identical cron fallback in code:
--durata=55plusflock -n.flockis the part that matters: two workers in parallel double the calls.
If the worker maintains state read by the front end (state cache in a table), two tricks pay off straight away:
- heartbeat and read in the same call: a single endpoint saying "this screen is open" and returning the state. Separating them doubles traffic for the same information;
- always return
letto_il+ora_server: without knowing when it was read, a value from three seconds ago is unusable for a client that has to interpolate.
6. Maintenance: what gets pruned and what doesn't
A running system accumulates. It's worth having a maintenance job that prunes, with judgement:
- prune application logs (from the tail, keeping the last N rows), expired temporary states, expired OAuth tokens, caches of external content;
- don't prune anti-duplicate registers (an
email_logisn't history: it's what prevents sending the same communication again), audits of movements and payments, rows another table points to; - don't automatically close objects in an intermediate state (abandoned orders, drafts): count them and show them. The decision belongs to whoever manages them, not to the job.
⚠️ AUTO_INCREMENT: deleting rows doesn't rewind the counter. A test creating and deleting a thousand rows leaves the next ID at 1001. It's not a problem, but it's a question that comes up sooner or later.