About Cache

Preliminary note: this page needs rewriting to be more 100% correct and easier to understand. Please ignore it for the moment.

How to refresh the cache in pages when http caching is enabled

How to refresh the cache in pages when http caching is enabled

First see more about caching and how to enable it on https://github.com/joomlatools/joomlatools-pages/wiki/Caching

Configuration on the present site

The following config.php file was created in the joomlatools-pages folder:

    <?php

return array(

    'http_cache' => true,

    ## Cache is cached for 1day, revalidated every day
    'http_cache_time' => '1day',

    ## Browser cache is always revalidated, changes made will appear immediately
    'http_cache_time_browser' => 0,

    ## Turn on collection caching for collection data fetches over http
    'http_client_cache' => true,

    //'page_cache' => false,
    //'data_cache' => false,
    //'template_cache' => false,
);

Is the cache also working on this website for the gdoc and gsheet demo on the present site?

'http_client_cache' => true, is on so the cache for the gdoc and gsheet is working: you will find a new dir called /joomlatools-pages/cache/responses, containing two files, one for the gdoc and one for the gsheet

The gdoc is cached always, aka pages will not recheck of the original file has changed, the gsheet uses pages webservice caching which is a smart cache, it will know when you update the csv.

The http cache can be disabled per page and the cache time can also be set. The frontmatter property is:

@process:
    cache: [true, false,  time]

The time is specified in seconds.

It is important to understand how the Pages caching works. Pages cache is smart, meaning Pages knows when data that is used on a page has changed.

For example, when you have a list of Joomla articles and you edit the article Pages will regenerate the page, when it does that still depends a bit on your cache settings but it knows the page is no longer valid.
=> This only works if you are using collections

However, if you use the Joomla menu system Pages doesn't know when you edited a menu item, and it should regenerate the page, so it keeps returning the page from cache.

Adding support for the menu system is probably something I could add as part of the Joomla extension, to get around this issue, but the issue exists for any other Joomla components. That is why collections are key to making it all work.

How does Pages'cache work

As a rule of thumb you should never need to manually delete the /joomlatools-pages/cache folder. This is a last resort kind of measure just in case, but it shouldn't be a action you ever need to take if everything is working correctly.

You can enable the http cache as follows:

return array(
    'http_cache' => true
);

Out of the box, once enabled the pages cache uses a mechanism called heuristic freshness: https://timkadlec.com/saved/2018/03/http-heuristic-caching-missing-cache-control-and-expires-headers-explained-paul-calvano/

In short this means that the cache lifetime is 10% of the last modified date of the page. So if your page is 1 day old it will be cached for 2,4h. The older the page gets the longer it will be cached. Note: There are additional settings to define caching behaviour that give you complete control how the cache works.

Once a page is cached it will be automatically regenerated if it expires aka when cache age > the heuristic time. If however you want to regenerate the page faster you can manually refresh the cache. Here is how:

Normal reload

cmd + R
hitting browser refresh button

This will make pages check if the collections used on the page are still valid, aka have not been changed, if one of them is then pages will regenerate the page.

=> This is called cache revalidation

Note: This only works for data coming from collections, it doesn't work for changes to templates, partials, config, or for changes to data that is not part of a collection like menu items

Hard reload

shift + cmd + R
right click on browser refresh button and choose hard reload (only works if inspector is open on chrome)

This will make pages bypass your cache and regenerate the page. This makes sure that changes in templates, partials, config, and Joomla menu items are being considered too.

=> This is called cache regeneration

When developing

Always hitting the browser refresh to do a hard reload is cumbersome so when developing with pages you do the following:

Enable inspector in chrome, a good developer always has inspector open this allows you to see what is happening
In inspector go to the network tab, you will see a [disable cache] checkbox => enable it

With disable cache enabled, chrome will always do a hard reload, and you are sure you page is always regenerated.

When in production

If you are running a production site and depending on your cache settings you might want to revalidate all the cached pages. This is possible too using Mason. Mason is able to incrementally revalidate or regenerate the cache of a site, or even single URL.

For more information see https://github.com/joomlatools/joomlatools-pages/pull/435

When doing a "hard reload", this means that Cache is then Regenerated. Does that mean that all visitors will get the refreshed version of the page (or is it just me)?

The answer to this it, it depends and i will need to deep dive a bit more into how web caching works.

First of its important to understand that caching works on a http level, so the same rules apply o any type of page, html, css, js, ... etc.

By default a browser will cache a page as long as you tell it too and if you don't tell it how long it will use heuristics to calculate this himself.

Static assets

For static assets like css, js, image, your webserver decides the cache time by setting the max-age header.

Usually this is 1day at least or more. You can configure this through htaccess.In case of css file.

The longer the max-age, the longer the browser will cache the css file and even if you make changes the client will not see them.

Unless he cleans his cache, or the cache expires.To solve this problem (for static assets) we use a technique called cache busting: https://css-tricks.com/strategies-for-cache-busting-css/

The approach is simple, add a unique id to the url of the css, based on the content or last modified date of the file.

If the file changes the url will change and it will need to be refetched, regardless how it's cached.

Cache busting is the defacto standard that you should use for static assets, you then server your assets with following Cache-Control header:

cache-control: public,max-age=31536000,immutable

Here we are telling all caches, including the browser to cache the css file for 1year.

Immutable gives a hint to web browsers that this assets at this url will never change.

I have a template filter for Pages that implements cache busting, still need to move this into the core, just ask if you need it.

You still need to configure your cache rules in your browser yourself using mod_expire to get the desired results.

Generated pages

For generated pages like html, rss, json etc the cache busting technique doesn't work since we cannot change the url each time the content changes, the url needs to stay the same.To solve this problem, we can tell the browser that it can cache the page but it should always re-check with the server if the page it has in cache is still fresh. We do this by following config:

'http_cache_time' => '1day',
'http_cache_time_browser' => 0,

Here you are telling the browser to use a different cache time then the server, server will cache for a day.

The 0 here means the browser is allowed to cache but needs to revalidate it's cache with each request.

Meaning that it needs to go back to the server to check if the page is still valid.If the page has changed on the server it will be served the new page, if the page hasn't changed the browser will receive a 304 not modified response and he will use the page he has in cache.

Note though that this has a performance impact:

If the browser needs to revalidate the page on each request it means that it can never serve it from it's cache, and that means you will loose the benefits of the prefetcher in Pages, which make your site will like a PWA (progressive web app).
With the smart caching in pages this impact is much smaller then regenerating the page for each request (which is what Joomla does). You should get a +/- 100msec request on a proper server, which is still super fast.

Revalidating for each request is often not needed. You could for example also set a smaller cache time for the browser:

'http_cache_time_browser' => '1hour',

This would make the browser cache your page for one hour, usually this means that during the same session if the page is visited it will be retrieved from the local cache.This gives you a nice trade-off between freshness and caching.