Automate backup restoration
Joomla :fa-joomla: | WordPress | Drupal | Prestashop | Magento | Moodle | phpBB
present slides https://slides.woluweb.be
video (FR) YouTube : rJbYrQQlRP4
« An untested backup is as good as no backup at all »
your life !
Think local, act global
This presentation has initially been made
for our local Joomla User Group (JUG Wallonie)
https://www.jugwallonie.be Facebook : joomla.belgique
But it was also streamed in order to remain available 24/7 on YouTube so that everybody can enjoy it virtually ;)
Since then, it has been updated and also presented a.o. at different JoomlaDays :
At your own risks
Look at this presentation at your own risks
This presentation contains no allergens but the (physical/virtual) presence at Joomla events can induce a dependence.
If you feel the symptoms after this presentation, follow instructions : * go to more JUG meetings, JoomlaDays, Conferences etc * whatever your level, starter or pro, share your knowledge * contribute to the forum, write articles, make presentations, ... * start volunteering https://volunteers.joomla.org * spread the Joomla Love
In this presentation I just share my enthousiasm about a nice solution so that everyone can benefit from my own experience
But note that I am not sponsored in any way by the developer of the mentioned Restoration Tool (all I have is a free sticker )
Like many I just happen to be a happy subscriber of his Backup Tool...
... and his Restoration Tool is available at no extra cost
In the previous episode...
Automatic backups & Amazon S3
In a previous presentation I explained in detail "how to secure your automatic backups by transferring them to the cloud on Amazon S3 with Akeeba Backup"
The previous presentation
One script to automate them all
In this presentation, I will explain in detail the next step, namely how to use a script that will automatically 1. restore the backups of all your sites (for ex. on your local server) 2. upgrade Joomla (experimental) 3. update all extensions (experimental)
When is it useful ?
Different cases :
1. test whether backups are indeed OK
2. test before updating extensions/plugins (or installing new ones)
3. test before upgrading your CMS
so now you can easily contribute testing Beta & RC ! ;) 4. mass site creation 5. maintaining demo sites
Only for Joomla ?
The automated restoration solution works for all the major CMS. The automated upgrade/update is nevertheless specific to Joomla, unless you know of a script that can do that job. * Joomla! :fa-joomla: * WordPress * Drupal * Prestashop * Magento * Moodle * phpBB
Which tools ?
- Akeeba Backup (free or pro) to take the backups
- Akeeba UNiTE (no extra cost if you have any subscription)
- A central script + 1 XML file for each website to handle
- A local server (Win/Mac/Linux. UNiTE is agnostic)
Akeeba Backup ?
Akeeba Backup for Joomla :fa-joomla:
Download https://www.akeebabackup.com/products/akeeba-backup.html Documentation https://www.akeebabackup.com/documentation/akeeba-backup-documentation.html
Akeeba Backup for WordPress
Download https://www.akeebabackup.com/products/akeeba-backup-wordpress.html Documentation https://www.akeebabackup.com/documentation/akeeba-solo.html
Akeeba "SOLO" for all other CMS or PHP applications Download https://www.akeebabackup.com/products/akeeba-solo.html Documentation https://www.akeebabackup.com/documentation/akeeba-solo.html
Akeeba UNiTE ?
To quote the site :
"A command-line application created for power users. Automatically restore one or multiple sites. Useful for mass site creation or maintaining demo sites. Exclusively available to AKEEBABACKUP, BACKUPWP, SOLOPHP, ESSENTIALS and JOOMLADELUXE subscribers."
A few screenshots
Let's suppose you manage plenty of websites.
As you know, you should definitely check the integrity & completeness of your backups on a regular basis.
Besides this, before upgrading Joomla and updating all extensions, you would like to be sure that everything will be OK.
Last, now that it becomes so easy to restore/upgrade/update all your sites, you will be very happy to test the future Beta and Release Candidate releases of Joomla and to contribute !
Let's see how to automate the whole process very easily.
One double-click on a script and all your sites are restored/upgraded/updated.
See the "exciting" video of the script doing your work on the next slide (1m32s for 2 sites in this example).
All you need to do after is to check on your local server if all websites are fine...
Download and install Akeeba UNiTE
First of all, please RTFM :) Everything is explained in detail in the following official documentation https://www.akeebabackup.com/documentation/unite/
Download Akeeba UNiTE and unzip it on your local server. For this presentation, I have uncompressed it in a folder named
My local server being Xampp for Windows, the root is
C:\xampp\htdocs and so here is the complete path in my case :
For every site, create an XML file that you typically put in the same folder as Akeeba UNiTE.
You could create a subfolder of course.
Note that before version 4.x of Akeeba UNiTe, the XML files were typically in a
The XML file for a local .jpa backup file
Example of XML file : local
<?xml version="1.0" encoding="UTF-8"?> <unite scripting="02_angie"> <siteInfo> <package>/demo_backups/1demo.jpa</package> <deletePackage>0</deletePackage> <localLog>test.log</localLog> <emailSysop>0</emailSysop> <name>Restored with Akeeba Unite</name> <email>firstname.lastname@example.org</email> <absolutepath>C:\xampp\htdocs\akeeba_unite_output\1demo</absolutepath> <adminID>79</adminID> <adminUser>admin</adminUser> <adminEmail>email@example.com</adminEmail> <adminPassword><![CDATA[admin]]></adminPassword> <force_ssl>0</force_ssl> <removephpini>1</removephpini> <removehtpasswd>1</removehtpasswd> <replacehtaccess>1</replacehtaccess> </siteInfo> <databaseInfo> <database name="site"> <changecollation>0</changecollation> <dbdriver>mysqli</dbdriver> <dbhost>localhost</dbhost> <dbuser>root</dbuser> <dbpass></dbpass> <dbname>1demo</dbname> <dbprefix>test_</dbprefix> <existing>drop</existing> <foreignkey>1</foreignkey> <noautovalue>1</noautovalue> <replace>0</replace> <utf8db>0</utf8db> <utf8tables>0</utf8tables> <utf8mb4>0</utf8mb4> </database> </databaseInfo> </unite>
In my case, all .jpa backup files are stored in a subfolder called
Therefore, for a filename being 1demo.jpa, the
<package> tag is as follows :
I want the restoration to be put in a folder called
<absolutepath> tag is as follows :
Comments : tags related to the Super User
:fa-joomla: == These 'admin' Tags are specific to Joomla
but you have similar Tags for all the other CMS==
, , , ... : see https://www.akeebabackup.com/documentation/unite/
<adminID>79</adminID> <adminUser>admin</adminUser> <adminEmail>firstname.lastname@example.org</adminEmail> <adminPassword><![CDATA[admin]]></adminPassword>
adminID : you HAVE to put the right ID of the Super User bc UNiTE has no other way to know
adminUser : you can change the Name of that user... but that won't change its Username (used to login)
adminEmail : if you feel like, you can change it
adminPassword : very handy to quickly login on your Local Server
Comments : ANGIE parameters 1/2
<force_ssl>0</force_ssl> <removephpini>1</removephpini> <removehtpasswd>1</removehtpasswd> <replacehtaccess>1</replacehtaccess>
Akeeba UNiTE was already very efficient when it comes to install on a local server the backups of all sites. But when I discovered it, I still had to do a little series of manual interventions : * all my sites have SSL enforced... but I have no SSL on my local server * some of my sites have php.ini at the root, which would have side-effects on my local server * all my sites have their administration protected by an htpsswd. I had to delete those manually on the local server * all my sites have long htaccess (optimisation+security), which had side-effects on the local server (like adding www)
Comments : ANGIE parameters 2/2
Then I contacted Nicholas (the author of Akeeba)... and a few weeks later he came with a new version of Akeeba UNiTE, offering those 4 new options (these being already available when you restore manually btw)
Txs Nicholas :)
For more information, see the release notes of version 3.1.0
Just a little tip : for each site, you have to choose its database name.
If you give the same "prefix" (like
s3_ in the following example), then all those datatabases will be gathered in PhpMyAdmin, which makes it easier to manage (for example if you want to delete them all)
The XML file for a .jpa backup file from AmazonS3
Example of XML file : AS3
<?xml version="1.0" encoding="UTF-8"?> <unite scripting="02_angie"> <s3> <accesskey>ABCDEF</accesskey> <secretkey>123456</secretkey> <signature>v4</signature> <bucket>mybucket</bucket> <region>eu-central-1</region> <ssl>1</ssl> <filename>path/to/site-backup.jpa</filename> </s3> <siteInfo> <package from="s3"></package> <deletePackage>0</deletePackage> <localLog>test.log</localLog> <emailSysop>0</emailSysop> <name>Restored with Akeeba Unite</name> <email>email@example.com</email> <absolutepath>C:\xampp\htdocs\akeeba_unite_output\3demo-amazons3</absolutepath> <adminID>79</adminID> <adminUser>admin</adminUser> <adminEmail>firstname.lastname@example.org</adminEmail> <adminPassword><![CDATA[admin]]></adminPassword> <force_ssl>0</force_ssl> <removephpini>1</removephpini> <removehtpasswd>1</removehtpasswd> <replacehtaccess>1</replacehtaccess> </siteInfo> <databaseInfo> <database name="site"> <changecollation>0</changecollation> <dbdriver>mysqli</dbdriver> <dbhost>localhost</dbhost> <dbuser>root</dbuser> <dbpass></dbpass> <dbname>3demo</dbname> <dbprefix>test_</dbprefix> <existing>drop</existing> <foreignkey>1</foreignkey> <noautovalue>1</noautovalue> <replace>0</replace> <utf8db>0</utf8db> <utf8tables>0</utf8tables> <utf8mb4>0</utf8mb4> </database> </databaseInfo> </unite>
Amazon S3 v4 API integration
When I discovered Akeeba UNiTE, the restoration directly from Amazon S3 was already present, but only with signature v2. Therefore, the tool would not be able to access my backups, which are stored in EU (Frankfurt) and use v4.
Following an exchange with Nicholas, again he upgraded UNiTE and amended the Documentation to integrate Amazon S3 v4 !
Txs Nicholas :)
In particular, a new parameter is now needed, ie the Region
For more information, see
'The' Script to automate them all
The principle for restoration
The principle for restoration is very simple : simply execute the following command from the Akeeba UNiTE folder on your local server
Note : before version 4.x released in 2018, the command was
About version 4.x of Akeeba UNiTE
Advantage of the new phar format introduced in version 4.x of Akeeba UNiTE: everything is in 1 file :)
Note : before version 4, the XML files were by default in a subfolder called '''inbox'''
But anyway you are of course free to place your xml files whereever you want. Example from the Manual :
/usr/bin/php-cli /path/to/unite.phar /home/myuser/backups/mysite.xml
Command line on a local server on Linux/Mac
If you are a Linux user, no doubt you don't need my/any help to run the script
If you are a Mac user, macOS comes with PHP pre-installed.
So you just need to do php ./unite.phar mywebsite.xml
Command Line on a local server on Windows
Paradoxically, on a live server running Unix at my hosting companies I knew how to use the Command Line... but on my Windows local server I had no clue :) So here the answer :
Search and launch CMD (the Command Line)
Go to the PHP directory of your local server. In my case
- Type the following command,
mywebsite.xmlbeing in the same folder as
php C:\xampp\htdocs\akeeba_unite\unite.phar mywebsite.xml
On a local server on Windows with a Batch file
Of course, when you have many website to restore, you want to create a Batch file in order to process all your websites in one double-click
- create a
.bat file, for example in
- copy-paste the batch file shown hereafter
- and adapt it of course according to the names of your folders and XML files
The principle for upgrade/update
There is a script out there allowing to upgrade Joomla and update its extensions from the Command Line. It is a proof of concept so - do not use it on Production site (see next slide) - but for what we are doing (testing purposes), it is just fine
and put that file in the same folder as '''unite.phar''' NB: afterwards we will copy this file on each restored website in order to be executed, see hereafter
update.php is 'experimental'
But please be aware that
update.php should not be used on Production sites, as it has drawbacks :
See for example https://github.com/joomla-projects/cli-update/issues/1
Also, read here the feedback by Nicholas : I had written the same kind of script. Eh, I wouldn't recommend updating Joomla! from CLI. The last bit which cleans up old files is still a hit and miss and we've only been able to make it work through the web interface. We being Michael Babker, George Wilson and yours truly. The update is based on Akeeba Restore which is also the heart of Kickstart, UNiTE and the integrated restoration feature in my backup software. The trick is that we have a post-extraction script which loads a thin layer of Joomla! to execute the post-update tasks. This needs to load the NEW version of Joomla! you have just installed. When running from CLI you end up with some of the loaded classes being from the PREVIOUSLY installed version of Joomla! which can make things go boom.
NB : see the future of CLI update here:
Three parts in the Batch file
The full Batch file is detailed in 2 versions on the next 2 slides (Windows example), but here are the key commands/steps : 1. Restore the website
php unite.phar 1demo.xml
- Copy the update script into the CLI folder of the website
copy "C:\xampp\htdocs\akeeba_unite\update.php" "C:\xampp\htdocs\akeeba_unite_output\1demo\cli\update.php"
- Update extensions and upgrade Joomla
php C:\xampp\htdocs\akeeba_unite_output\1demo\cli\update.php --extensions php C:\xampp\htdocs\akeeba_unite_output\1demo\cli\update.php --core
REM This Script (i) automates installation of backups (ii) upgrades Joomla (iii) updates all extensions set STARTTIME=%TIME% REM Needed for Windows: adds the folder containing php.exe to the path (Xampp being here the Local Server) PATH=%PATH%;C:\xampp\php REM Change Directory to the folder containing your script CD C:\xampp\htdocs\akeeba_unite\ REM Restore all websites php unite.phar 1demo.xml php unite.phar 2demo.xml REM Copy the update.php script in the CLI folder of each website copy "C:\xampp\htdocs\akeeba_unite\update.php" "C:\xampp\htdocs\akeeba_unite_output\1demo\cli\update.php" copy "C:\xampp\htdocs\akeeba_unite\update.php" "C:\xampp\htdocs\akeeba_unite_output\2demo\cli\update.php" REM Update all extensions php C:\xampp\htdocs\akeeba_unite_output\1demo\cli\update.php --extensions php C:\xampp\htdocs\akeeba_unite_output\2demo\cli\update.php --extensions REM Upgrade Joomla php C:\xampp\htdocs\akeeba_unite_output\1demo\cli\update.php --core php C:\xampp\htdocs\akeeba_unite_output\2demo\cli\update.php --core echo Start Time : %STARTTIME% echo End Time : %time% pause
Batch file with a loop - if many websites
REM This Script (i) automates installation of backups (ii) upgrades Joomla (iii) updates all extensions set STARTTIME=%TIME% REM Needed for Windows: adds the folder containing php.exe to the path (Xampp being here the Local Server) PATH=%PATH%;C:\xampp\php REM Change Directory to the folder containing your script CD C:\xampp\htdocs\akeeba_unite\ for %%x in (1demo 2demo 3demo 4demo) do ( REM Restore backup php unite.phar %%x.xml REM Copy the update.php script in the CLI folder of each website copy "C:\xampp\htdocs\akeeba_unite\update.php" "C:\xampp\htdocs\akeeba_unite_output\%%x\cli\update.php" REM Update all extensions php C:\xampp\htdocs\amazons3\%%x\cli\update.php --extensions REM Upgrade Joomla php C:\xampp\htdocs\amazons3\%%x\cli\update.php --core ) echo Start Time : %STARTTIME% echo End Time : %time% pause
After running the script
If you restore many websites, after the script has run you might want to copy-paste the contents of your Command Line window to your favourite Notepad.
That way, you can quickly check whether all restorations (and updates/upgrades) went fine.
NB : the script show the Start Time and End Time. This is of course optional. In my case, each restore takes on average less than 1 minute.
Run a customised query after restoration
Note : you can even create a SQL file to suit you own needs and point to it in your XML file in UNiTE
That's BTW how you can automate everything with UNiTE, for example * disable misbehaving plugins (see examples in Troubleshooting) * disable all email / live services plugins * anonymize the users (names and emails) when restoring a backup of the site locally
Fine-tuning & troubleshooting
Your website forces SSL (https) but your local server does not handle https ?
5 alternative solutions :
- manual solution : set $force_ssl to 0 in your configuration.php file
- just force your browser to show that "insecure website" (you're on YOUR local server anyway)
- have fun and install an SSL certificate on your local server
- use a local server with like MAMP and its built-in SSL support (the best option for Windows and macOS)
- ... or enjoy the
<force_ssl>0</force_ssl>parameter that you can put in your XML file
If the configuration.php is not part of the backup, the automatic restoration will not work (although a manual restoration with Akeeba would work !).
So be sure that configuration.php is part of your backup...
configuration.php > memcache
If you use Memcache on your live site, the locally restored website will not work.
Simply edit configuration.php and change the
$cache_handler as follows :
$cache_handler = 'file'
configuration.php > customisation
More generally about anything related to configuration.php :
If you are restoring the same site on your local / demo / dev / whatever server all the time you can create a small ZIP file with a prefabricated configuration.php file and have UNiTE extract it to the site's root.
That's what Nicholas himself does with his continuous restoration of my live site's backups to my local server :)
Redirected to "www.localhost"
When you check your sites, you get redirected from
You probably left the original .htaccess file forcing
Just delete it and replace it by the native .htaccess file
Even better : use the option in the XML file allowing to automate it !
Another cause can be some "conflict" with your antivirus software.
Example from a forum :
"My Avast ! Free Antivirus redirects localhost:8080 to www.localhost.com:8080. I reset the web redirection setting in Avast!"
SiteGround Cache Plugin
If you use SiteGround's Cache Plugin, you might get an error message on the front-end of your website.
Simply disable that plugin from the backend in order to test your site effectively.
You can also automate this by creating a SQL file that disable misbehaving plugins and point to it in your XML file in UNiTE.
Restore phase freezes
Your process freezes in the middle of the restore phase ?
Just delete the stealth .htaccess file and finish the installation manually (
You see no images on your local website ? You probably left the original .htaccess file. Just delete it and replace it by the native .htaccess file
Even better : use the option in the XML file allowing to automate it !
In the case of two websites "sharing some tables" (like the Users table) with a View, the restoration would not work for the website having the View instead of the Table.
Name of site
The XML file allows to change the Name of the site.
Well, for multilingual sites, if you have setup a different Name for some language, of course this is left as it is.
So if you really need to have that changed as well, you can also play with Akeeba UNiTE in order to run some SQL query automatically.
Backup would not be restored
Well, there could be many reasons to that one. But what I have experienced from a couple of sites is that there were some files with special character at the root of the original website, like
(even though that question mark had never caused me any issue during manual backup/restore !)
Issue with Acymailing and local restoration
This issue has in fact nothing to do in itself with Akeeba UNiTE, but when you restore a website having an Acymailing module on the front-end, typically the page will throw a popup telling that "localhost is not allowed".
In order to fix that, go to
Acymailing > Configuration > Subscription > Allowed domains for the redirection via the module
|localhost to the field
How to handle the backup files
Option 1 : directly from the Cloud
Actually you don't even have to download your backups in the first place.
In your XML file, you can directly specify your credentials.
This can be done with the following Tag for Amazon S3 :
Or with the following Tag if you connect directly to your website :
The disadvantage : you have to edit each XML file to adapt for the name of the backup file because (unless you configure it otherwise) the latter typically contains a timestamp.
Option 2 : download first all backup files from the Cloud
To download all your backups from Amazon S3, use for example Cyberduck
NB : contrary to other softwares like FileZilla, Cyberduck can indeed login to AS3
(Or download them from each website, but that will take much more time...)
Option 3 : backups already available locally
Well, if your backups are already available for example on the local server where you want to restore them all, no need to download them.
Be efficient/lazy : manual solution
In Options 2 and 3, your backups are on your local server. But as already mentionned for Option 1, the name of the backup files typically contain a timestamp.
One could of course simply adapt each XML file accordingly everytime.
But I guess a better solution is to first rename the backup files manually.
Be efficient/lazy : automatic solution
An even more efficient solution is to rename the backup files automatically.
Using Windows, this can be done for example with a tool like Bulk Rename Utility : http://www.bulkrenameutility.co.uk/Download.php
In my case, I remove the first 9 characters (site-www.) and the last 16 characters (timestamp of the backup) :
As suggested by Nicholas, you can even automate the whole process (download from AS3/rename/restore) with shell scripts.
With other words, a kind of mix of all previous Options :
- use s3cmd to list the backups in the bucket and awk to extract the name of the latest backup
- then s3cmd again to download it and rename it to site.jpa
- then launch UNiTE
FYI that bit works perfectly on a Raspberry Pi 2, a really underpowered ARM thingie with barely 512M RAM.
I'm not sure I would need that, but on Windows you could use the Task Scheduler in order to launch the restoration of all backups automatically at regular intervals.
But this would of course first require to also automate the Download of the backups.
If you want to test your websites further, you might want to install a MailCatcher. That way, you can also simulate the sending of a newsletter, the transactional emails etc.
See for example http://nilhcem.com/FakeSMTP
Other backup & archive tools
See https://github.com/Amaury/Arkiv :
Arkiv is an easy-to-use backup and archive tool.
Arkiv is designed to backup local files and MySQL databases, and archive them on Amazon S3 and Amazon Glacier. Backup files are removed (locally and from Amazon S3) after defined delays.
Arkiv could backup your data on a daily or an hourly basis (you can choose which day and/or which hours it will be launched). It is written in pure shell, so it can be used on any Unix/Linux machine.
Control, repair or convert your zip, jpa, rar files
Presentation in French :
Someone has kindly told me about another solution to backup/restore/upgrade websites.
I haven't tested it, but feel free to see what it can do for yourself :
Other DevOps tools
I discovered the existence of update.php in an excellent presentation made by Andy Gaskell at #jd17uk
for even more interesting DevOps tools :)
The previous episode was about automating backups in the cloud
The present episode is about automating restoration (and updates/upgrades) on your local server
In the next episode, why would you not do the final step and learn how to become a Joomla Tester ?
Spread the Joomla Love
Joomla! GSoC 18 - CLI Update
The project (published on 2 June 2018)
The aim of this project is to add a good CLI Application to the Joomla! CMS, that can perform the updates without going to the administrative panel. Administrators can just go via SSH to update somethings within the CMS.
The idea for it to be core feature in Joomla! 4. See the code here : https://github.com/joomla-projects/gsoc18_cli_update
Watch the video (FR) : YouTube : rJbYrQQlRP4
Any suggestion about this presentation ?
Please feel free to contact me. I'll be happy to keep improving it :)
Marc Dechèvre | @woluweb
+32 474 37 13 12 | +32 2 772 58 69