Skip to content
JoeCMS Download 3.1.5
Manual · 05

Cron, jobs and workers

Three ways to make the system work when nobody is watching: an HTTP cron, an internal job queue, a long-running process. They're for different things, and picking the wrong one costs.

1. Which of the three

NeedToolWhy
"every N minutes do X"HTTP cron (§2)needs no shell access: works on any hosting with a curl scheduler
"do X, but not now and not in front of the user"job queue (§4) or Jobs::asyncthe user doesn't wait; the effect arrives shortly after
"always listen / poll an external service at sub-minute cadence"CLI worker (§5)cron has 1-minute granularity and pays the bootstrap on every run

Rule of thumb: start with cron. A worker is a service to install, monitor and restart; introduce it when cron proves not to be enough, not before.

2. HTTP cron: a job is a page

Scheduled jobs are pages like any other (see 02 §1):

contents/cronjob/<name>.php    →    GET /cronjob/<name>?key=<secret>

The contents/cronjob/ folder already exists in the repo, empty: that's the intended place.

Pattern to copy:

<?php
use_cache(false);

// 1 — SECRET: the job is reachable over HTTP, so by anyone
$cronSecret = (string) (defined("JC_CRON_SECRET") ? JC_CRON_SECRET : "");
if ($cronSecret === "" || !hash_equals($cronSecret, (string)($_GET["key"] ?? ""))) {
    http_response_code(403); echo "forbidden"; exit;
}

$db = ffDB_Sql::factory();

// 2 — LOCK: two overlapping runs are the norm, not the exception
$db->query("SELECT GET_LOCK('jc_coda', 0) AS l");
$db->nextRecord();
if ((int)$db->getField("l", "Number", true) !== 1) { echo "busy"; exit; }

try {
    // 3 — limited BATCH: never "everything there is"
    $limite = 200;
    // ... work ...

    // 4 — LOG the outcome
    // INSERT INTO cm_jc_cron_log (job, esito, elementi, eseguito_il) VALUES (...)
} finally {
    $db->query("SELECT RELEASE_LOCK('jc_coda')");
}

echo "ok";
exit;

The four safeguards, and why

  1. Secret in the query string compared with hash_equals. A job without a secret is a public write endpoint.
  2. MySQL GET_LOCK with timeout 0. If the previous run hasn't finished, the new one exits immediately instead of doubling the work. One lock per job, not a global one: different jobs mustn't contend.
  3. Limited batch with LIMIT. A job trying to clear the whole backlog times out, and the timeout leaves it half-done. Better N short runs.
  4. Log to a table with outcome and count. Without it, you can't answer "is the cron running?". SELECT * FROM <log_table> ORDER BY ID DESC LIMIT 20 should be enough.

Other tricks you learn in production

  • Explicit skip if the migration isn't applied. A job running on an old schema must exit with "migration missing", not hit a fatal error every minute.
  • Degrade, don't block. If an external provider doesn't answer, log it and retry on the next run.
  • Quiet hours, if the job sends communications: don't send at 3 a.m.
  • Off switch in configuration (abilitato on/off), which doesn't require a deploy.
  • Idempotency: the job may be re-run on the same interval (restart, double scheduling, scheduler retry). An anti-duplicate register (one row per (recipient, type, period) with a unique key) is more reliable than "count and compare".

Scheduling

From the hosting panel (cron via curl):

*/5 * * * *  curl -fsS "https://YOUR-DOMAIN/cronjob/<name>?key=<secret>" > /dev/null

Document in a table, in the repo, which jobs exist, how often they run and what happens if they don't: it's the information you need on the day you move to another hosting.

JobPurposeFrequencyIf it doesn't run
(none in the base repo)

3. Inherited system jobs

conf/gallery/job/ contains four jobs from upstream VGallery:

JobWhat it doesStatus
pingnotifies search engines of the updated sitemaplegacy: several of the pinged services no longer exist
updatequeries the remote updaterinert: MASTER_SITE is neutralised
diagnosticchecks cache, dictionary, config, uploads, thumbsuseful as a manual check
reset_cartscleans up abandoned cartsrelevant only with e-commerce active

They're not scheduled by default and write their outcome as a backend notification (write_notification). There's also an autoload-by-convention mechanism: check_function("job_<name>") loads library/gallery/job/<name>.php or modules/<module>/library/job/<name>.php (library/gallery/common.php, lines ~138-146).

4. The internal job queue (Jobs)

vendor/hcore/jobs/Jobs.php implements a persistent queue with scheduling and repetition.

Jobs::getInstance()->add($source, $params, $schedule, $repeat, $priority);
Jobs::getInstance()->run();          // runs due jobs
Jobs::getInstance()->ps();           // queue status
Jobs::getInstance()->forceRun($kid); // forces a job
Jobs::setScript($callback, $schedule, $repeat);

Executors per type: internal API, service, HTTP request, script, function, object (controller_api, controller_service, controller_request, controller_script, controller_func, controller_obj).

Guards in the class: SOCKET = 10, TIMEOUT = 60, ANTIFLOOD = 4 seconds between two runs, DELAY_SCHEDULE = 3600 for rescheduling after an error, REPEAT = "1week".

And the fire-and-forget primitive:

Jobs::async("/api/cache/refresh", array("url" => $_SERVER["HTTP_REFERER"]));

opens a socket to its own host, writes the request and doesn't wait for the response. In the repo it's used by library/gallery/system/trace.php and by the service bus (see 03 §4).

When to use it and when not. Jobs::async is perfect for "refresh the cache after answering". For work that must get done — sends, charges, notifications — you need something that leaves a trace and can be retried: the persistent queue, or your own queue table drained by a cron. The fire-and-forget socket guarantees nothing: if the called process dies, nobody notices.

There's also library/rabbitMQ/ among the libraries: an external broker, not wired in this repo. A project that really needs it can hook it in, but for the vast majority of cases a queue table + cron is simpler to manage and easier to inspect.

5. Long-running worker (CLI)

When you need a sub-minute cadence, or in-memory state between runs, you leave the web: a PHP script in bin/ started by systemd (or by a fallback cron).

Lessons that apply to any worker on this stack:

  • A dedicated CLI bootstrap. From the command line the page cycle doesn't run: classes come from the autoloader, but the functions in ff/common.php (ffCommon_dirname, ffCommon_jsonenc) don't. They must be included explicitly. Don't include library/gallery/common.php, which assumes a cm instance.
  • HTTP_HOST must be set before config.php: it decides whether to load config.local.php or config.remote.php, and half the framework reads $_SERVER without checking keys.
  • FF_DISK_PATH is the server path. The installer writes /var/www/… into it, which doesn't exist locally: a worker building paths on that constant fails off the server. Better an own constant derived from __DIR__, preferred when present.
  • Load only the module you need: the others hook events into the page cycle and from the CLI find no cm to hook into.
  • Static caches must be discarded periodically. Configuration classes are designed for the lifetime of an HTTP request: in a process living an hour they'd stay frozen at the value read at start-up, and changing a threshold from the backend would have no effect.
  • The worker exits by itself after a maximum time (e.g. an hour) and Restart=always brings it back. An exit with code 0 every hour is the intended behaviour: it's the simplest way not to have a memory problem two months from now in a long-running PHP process.
  • It doesn't run as root: it needs to read config.remote.php (0640, owned by the site user). Running as that user makes permissions work without widening them.
  • Identical cron fallback in code: --durata=55 plus flock -n. flock is the part that matters: two workers in parallel double the calls.

If the worker maintains state read by the front end (state cache in a table), two tricks pay off straight away:

  • heartbeat and read in the same call: a single endpoint saying "this screen is open" and returning the state. Separating them doubles traffic for the same information;
  • always return letto_il + ora_server: without knowing when it was read, a value from three seconds ago is unusable for a client that has to interpolate.

6. Maintenance: what gets pruned and what doesn't

A running system accumulates. It's worth having a maintenance job that prunes, with judgement:

  • prune application logs (from the tail, keeping the last N rows), expired temporary states, expired OAuth tokens, caches of external content;
  • don't prune anti-duplicate registers (an email_log isn't history: it's what prevents sending the same communication again), audits of movements and payments, rows another table points to;
  • don't automatically close objects in an intermediate state (abandoned orders, drafts): count them and show them. The decision belongs to whoever manages them, not to the job.

⚠️ AUTO_INCREMENT: deleting rows doesn't rewind the counter. A test creating and deleting a thousand rows leaves the next ID at 1001. It's not a problem, but it's a question that comes up sooner or later.