Skip to content
JoeCMS Download 3.1.5
Manual · 01

Request lifecycle and routing

What happens between the URL and the PHP file that answers. This is the chapter to read first: almost all of the stack's surprises start here.

1. The boot chain

HTTP request
  └─ .htaccess            path not on the filesystem → rewrites to /index.php
       ▼
  index.php
    ├─ config.php ────────► FIRST-INSTALL GUARD (host not ".local" and
    │                        themes/site/conf/config.remote.php missing → includes install/index.php, exits)
    ├─ themes/site/conf/config.{local|remote}.php   environment constants
    ├─ conf/gallery/config.php                      framework + vgallery config
    ├─ vendor/autoload.php                          composer
    ├─ hook.php                                     platform helpers (~1,900 lines);
    │    └─ hook.local.php                          at the end, if present: the site's code
    └─ Cms::getInstance("kernel")->run()            router → controller → ffPage → output

Not every entry point goes through index.php. These have their own bootstrap:

Entry pointPurpose
router.phpalternative dispatch
cm/showfiles.phpserving and resizing images (see 11 — Media and files)
cm/static.phpstatic assets (/themes, /uploads, /asset)
api/1.0/index.phppublic API, physical folder (see 04 — Public API)
install/index.phpfirst-install wizard

Rule of thumb: anything that must apply "always" (error handling, constants, correlation id) has to hook into a common boot, not into index.php.

2. Routing has three levels, plus the kernel

Rules are collected from several sources and sorted by priority (TOP, VERY_HIGH, HIGH, NORMAL, LOW, VERY_LOW, BOTTOM):

LevelFileContents
kernelvendor/wolfgan/cms/services/kernel.php (internal array)static assets and the reserved paths that answer 404
coreconf/routing_table.xmlsitemap, feed, error, install, updater, /md, /frame, /api, /srv, /notify, /search, /user/*
CMcm/conf/routing_table.xmlat the end, the rule that makes pages exist: (.*) → /contents$0, priority BOTTOM
modulesmodules/<name>/conf/routing_table.xmlloaded automatically for every folder in /modules

Anatomy of a rule (conf/routing_table.xml):

<rule id="services">
    <priority>NORMAL</priority>
    <source>^(?!/updater/check/file(.*))/srv(/.*)?$</source>
    <reverse>/srv</reverse>
    <destination><url>/conf/gallery/sys/services$2</url></destination>
    <accept_path_info/>
</rule>
  • <source> is a regexp, not a glob, and it matches anywhere in the path: without ^ a rule written /srv(.*) also catches /blog/srvizio. Folder rules start with ^ and end with (/.*)?$; the group inside (?!…) counts in the numbering (here the destination uses $2);
  • <reverse> is the prefix used to rebuild the public URL;
  • <accept_path_info/> allows a path residue (without it, a residue makes the rule fail);
  • <blocking>true</blocking> stops evaluation of the following rules.

⚠️ CM_ENABLE_MEM_CACHING is false in conf/gallery/config.php: the router rereads the XML on every request. Handy (routing changes take effect immediately), but it's also the file people most often forget to upload in a manual deploy. It can be turned on from the environment config, but it only helps with a redis or memcached adapter (12 §3).

When a rule points to a file that doesn't exist

cmsRouter::run() (vendor/wolfgan/cms/services/router.php) substitutes the regexp groups ($1, $2…) in the destination even when the destination is a file — not only in redirects. Then Cms::execute() (vendor/wolfgan/cms/Cms.php) checks, before including it, that the file really exists and lies inside the document root.

The two situations where you see this:

  • a rule asking for a group the path didn't produce — the destination arrives with a literal $1 inside;
  • a badly written destination (for example the <source> regexp copied into the <url> as well).

In both cases the answer is 404, with a line in log_error. They used to be a fatal error require(): Failed opening required — a 500 across the site, reported as a crash instead of a missing page, and that's how automated scans calling paths like /install/make-config.php or /…/sources/nginx.conf show up in Sentry.

A detail of _getDiskPath() that explains odd paths in the logs: a destination without an extension gets /index.php appended (/conf/gallery/install → /conf/gallery/install/index.php).

Three particulars of cmsRouter::regexp(), which apply to kernel routes and page-schema routes (not to routing_table.xml, which are real regexps read by another router):

  • matching is case-insensitive (#…#i): /INSTALL.md catches ^/install(.*) just like /install/ would. That's how a documentation file tried by a crawler ended up in the installer route;
  • the match is not anchored at the end: ^/conf(.*) catches everything that starts with conf, not just the folder. For a system folder the form is ^/name(/|$): the folder and what lies under it, nothing more. With '^/conf*' a /confronto page answered 404 with an empty body, before it was even resolved, and /errore ended up on the CMS error page.
  • * is expanded to (.*) only when it is a real wildcard, that is outside a hand-written group; inside a group it stays as it is ((*) is the only exception, and counts as a wildcard group). Before this fix the expansion was blind: ^/install(.*) became ^/install(.(.*)), which requires at least one character after /install. The kernel recognised the page (it reads the regex without touching it) but its route didn't fire on the "short" URL, and the request ended in the actex case of system_init_on_before_routing(): an E_USER_ERROR, i.e. a 500, on every GET /install. If a rule seems not to fire on the short URL, this is still the first suspect.

The route declared by the page schema

In themes/site/settings.php a page can declare its own route:

$schema["page"]["/scheda-dettaglio"] = array(
      /* ... */
    , "router" => array(
          "source"      => "^/scheda(.*)"
        , "destination" => "/scheda$0"
    )
);

In this form the route identifies the page, it doesn't route: the kernel uses it to work out which $schema["page"] entry matches the requested URL (get_page_properties() in vendor/wolfgan/cms/services/kernel.php, and library/gallery/system/cache.php for the cache key). What actually executes it is library/gallery/system/init.php, with addRule(), and only for the service and updater groups — that's how /install reaches conf/gallery/install/index.php, without going through the check in conf/gallery/config.php (see the note on the legacy installer further down, in the route map). If that route doesn't close the request (source that doesn't match, destination that doesn't exist) the request carries on and the Content Manager router takes it; before this fix the service/updater case fell instead into the actex case of system_init_on_before_routing(), which raises an E_USER_ERROR (500).

⚠️ The kernel table, on the other hand, is a regexp => destination map. Merging the form above with array_replace() slipped in two routes called source and destination: two literal, unanchored regexps matching any URL containing those words, with the regexp itself as destination. /esperto-risponde/sources/nginx.conf matched #source#i and ended up in a require() of the path ^/esperto-risponde(.*) — a fatal error on screen, not a 404. Now mergePageRouter() only merges the map form, and the {source, destination} form stays page data. To add real routes from the page schema, use the map: "router" => array('^/my-path(/|$)' => '/cm/main.php'). The folder-rule convention applies here too — ^/name(/|$), never '^/name*', which grabs every URL that starts that way.

XHR requests on a public page

A request with X-Requested-With: XMLHttpRequest on a public page does not necessarily reach the module of its URL. library/gallery/system/init.php (groups public, user and default) first tries to render the sections of the Referer page with system_layer_gallery(): if anything comes out, it saves it in the cache, prints it and ends the request. It is the ajax reload of the current page's blocks. Left out are /user, /services, the VG_SITE_USER, VG_SITE_CART and VG_SITE_NOTIFY areas, calls with XHR_DIALOG_ID or XHR_COMPONENT (the ff.ajax forms) and, since 3.1.3, every POST.

Before 3.1.3 an XHR POST sent from a page with its own sections, such as the home page, received those sections instead of the module's response. The case found was the registration form on joecms.it, which answered with the home page HTML. From the /registrazione page it worked, because there the sections are empty and the request goes on. An endpoint called via ajax with GET from another page is still exposed: call it with XHR_COMPONENT, or put it under /srv.

3. Paths reserved by the kernel

vendor/wolfgan/cms/services/kernel.php decides before the XML rules:

PatternOutcome
^/applets(/|$), ^/cache(/|$), ^/cm(/|$), ^/conf(/|$), ^/contents(/|$), ^/ff(/|$), ^/installer(/|$), ^/library(/|$), ^/modules(/|$), ^/vendor(/|$)404
^/themes(/|$), ^/uploads(/|$), ^/asset(/|$)served by cm/static.php
/restricted(/|$)reserved area: /cm/main.php
^/static/*, ^/media/*, ^/showfiles/*files: cm/static.php / cm/showfiles.php (the asterisk comes after the slash: the folder and what's below)
^/api/1.0/public API, physical folder (/cm/main.php if Apache doesn't serve it)
^/api(?=/|$)…routed to the framework's internal REST (Api::request)
^/srv(/|$)left to the XML rules
wp-login*401 (scan noise)

The kernel's 404 applies to requests that reach index.php. A file that exists on disk would be served by Apache on its own, PHP included: the PHP files and data of the internal folders are closed by the .htaccess (14 §3).

After routing, right after Auth::check(), the kernel also resolves the preferred language (language_pref_resolve()). It may answer with a 302 to the prefixed URL before any output: see 12 — Cache, §Multilingual.

Practical consequences:

  • a public page cannot be called /conf, /cache, /library…: the folder is reserved, with or without a trailing slash;
  • a name that starts like a system folder, on the other hand, is free — /conferma, /errore, /cms, /assetto, /ffmpeg resolve as normal pages. It wasn't so: with the '^/conf*' form the prefix was enough, and /confronto answered 404 without anything saying so;
  • /app stays free: ^/applets(/|$) doesn't catch it;
  • /api is reserved: a public API must be served from a physical folder, see 04 — Public API. It is reserved only as the first segment: a module's /jd/api/me, /apicoltura, /en/api/… reach the normal rules. Up to 3.1.0 the kernel's two /api rules and the api rule in conf/routing_table.xml weren't anchored: any path containing /api ended up in Api::request() and answered 404 with an empty body, before the page or the module. The same went for /srv in the services rule (/blog/srvizio).

4. File resolution walks backwards (which is why you never see a 404)

cm/cm.php (around lines 920-1000) tries, for /a/b/c:

  1. contents/a/b/c.php
  2. contents/a/b/c/index.php
  3. walks up with dirname(): contents/a/b.php, passing /c in $cm->real_path_info
  4. …up to contents/index.php, which always exists

The unconsumed residue ends up in real_path_info and the response is 200.

This is the mechanism that makes pages with a parameter in the path work — /md/<permalink> is the example in the repo. But on a site made of pages it means /made-up-page shows the home page with status 200: for a search engine, N different addresses with the same content.

The safeguard, if the project is "page-based", is a single guard at the top of the page renderer:

// if path is left over, the requested page doesn't exist
if (class_exists("cm", false)) {                       // false avoids instantiating the CMS from CLI
    $residuo = trim((string)cm::getInstance()->real_path_info, "/");
    if ($residuo !== "") { /* render the 404 and exit */ }
}

Four details that look like nitpicks and aren't:

  • a residue made only of slashes doesn't count: /prices/ has real_path_info = "/" and is the same page as /prices;
  • the guard goes into the common renderer, not into individual pages: that way a new page inherits it, and paths that really use the residue (/md, the /srv endpoints) don't go through it;
  • $cm is not a global variable: cm/main.php runs inside Cms::execute() (vendor/wolfgan/cms/Cms.php), a require in a static method, so its $cm is local. In the contents/ scripts it exists because callScript() defines it before the require; in a function or in a class method called from there it doesn't. $GLOBALS["cm"] is empty, the residue comes out empty and the guard never fires, without an error: on joecms.it every chapter of the Guide showed the index. Use cm::getInstance(), like get_pathinfo() in library/gallery/common.php;
  • the residue arrives in lowercase: cm/cm.php passes the path through ffCommon_url_normalize() (ff/common.php), which ends with mb_strtolower(). A residue to be compared with names that have capitals (00-INDICE.md) must be compared case-insensitively.

5. The request schema: the parameter whitelist

The kernel filters the query string against $schema["request"] (vendor/wolfgan/cms/services/kernel.php, line ~116). If a non-XHR GET carries undeclared parameters, the kernel doesn't ignore them: it classifies them unknown and redirects to the same path, rebuilding the query with only the valid parameters (lines ~127-134).

In practice: an OAuth callback, a UTM link, a confirmation ?token= lose their parameters and fail without any visible error, if they're not declared.

They're declared in themes/site/settings.php:

$schema["request"]["/"] = array(
    "get" => array("utm_source", "utm_medium", "utm_campaign", "optin")
);
$schema["request"]["/area-utente"] = array(
    "get"  => array("force_id"),
    "post" => array("name")
);

Two useful facts:

  • rules cascade along the path (dirname() in a loop): the /app entry also covers /app/cerca, /app/coda, …;
  • modules' conf/schema.php is NOT read. library/gallery/system/cache.php (lines ~91-94) only loads library/gallery/settings.php (core) and themes/<theme>/settings.php (project). A schema.php inside a module is dead code: all declarations go in the theme.

In the same entry you can turn off caching for that path:

$schema["request"]["/OneSignalIdentity"] = array("get" => true, "nocache" => true);
$schema["request"]["/cerca"]             = array("get" => array("q"), "nocache" => array("q"));

"nocache" => true defines DISABLE_CACHE for the request; with an array it does so only if one of those parameters is in the query. It's for responses that depend on the user and must not end up in a shared cache (see 12).

The page cache is declared in the same file:

$schema["page"]["/"] = array("name" => "public", "cache" => "guest");

See 12 — Cache and performance.

6. Map of core routes

From conf/routing_table.xml (repo state):

URLDestinationNotes
/install(.*)conf/gallery/installlegacy installer; the physical install/ dir exists only in the repo — at the end of the installation install/cleanup.php deletes it, so on the site /install reaches the kernel. On an installed site it answers 404 (since 3.1.5): conf/gallery/install/index.php only opens from a URL under /admin/ (Configuration → Basic, /admin/configuration/install, behind login), because the form is prefilled with the site's config, passwords included
/updater/check(.*)conf/gallery/updater/checkupstream updater, effectively retired
/error/(.*)conf/gallery/errorCMS error pages
(.*)/sitemap(.*), (.*)/feedconf/gallery/sitemaplegacy engine; JoeCMS uses modules/cms-common/models/Sitemap.php (see GUIDA_SITEMAP.md). It catches any path containing /sitemap, backend included: an /admin/…/sitemap page is never reached (that's why the Utility → Sitemap menu entry opens /sitemap.xml)
/manifest.(.*)PWA manifest
/md(/.*)?contents/md/index.phpMarkdown version of content (GUIDA_INDICIZZAZIONE_AI.md)
/frame(.*)conf/gallery/sys/frame.phpframed rendering
^/api/<ver>(.*)conf/gallery/api/...framework's internal REST; only as the first segment
^/srv(/.*)?$conf/gallery/sys/servicessystem services; rewritten by the cms-common module (see below); only as the first segment
/notify(.*)conf/gallery/modulesnotifications
/search(.*)conf/gallery/modules/searchsearch
/user/vgallery, /user/resources, /user/draft, /user/menu, /user/managecontent management areas
/user/wishlist, /user/cart, /user/mpaye-commercesee 10 — Modules
/ext/modules(.*)module resources

From modules:

ModuleRule
restricted^/restricted(.*) → /layout$0; ^/admin/settings(.*) → /settings$1
cms-common/srv/(.*) → /applets/services/$1 — this is the rule that makes project services reachable; ^/srv/email-preview has a dedicated rule
auth^/oauth2/(.+), ^/apiserver/(.*), ^/services/oauth2/(.+) — ⚠️ the destination folders don't exist in this repo: rules inherited from upstream, inert today

7. Checklist when a URL "doesn't answer as it should"

  1. Does the path start with a prefix reserved by the kernel? → 404 decided before the XML rules.
  2. Does the rule exist but the path residue makes it fail? → <accept_path_info/> is missing.
  3. Does the page show other content with a 200? → the router walked up (§4). If it's a page that uses the residue (/guida/<chapter>) and always shows the base page, the residue is read from $GLOBALS["cm"], which is empty: it must be read from cm::getInstance() (§4).
  4. Does a query parameter vanish and there's a redirect? → it's not in $schema["request"] (§5).
  5. Does the page declare "router" in the schema but no route fires? → in the {source, destination} form it identifies the page and doesn't route (§2).
  6. Changed an XML file and nothing changes? → it's a modified file, not a new one: in a manual deploy it's the first one forgotten.
  7. Stale response? → page cache, see 12.