These last weeks I had to upgrade a large ownCloud installation. It was the community edition, so there was little to none information about how to upgrade such big instances. Many ownCloud community installations seem to be running on small organizations, so most recommendations about upgrading were not valid for this instance. I had to upgrade 3 major releases, from ownCloud 4.5 to ownCloud 7.
This ownCloud instance used MySQL 5.5 as the database manager.
How large was this instance? Let’s see some numbers:
- 6 TB of information stored
- Almost 4 million files managed
- More than 11.000 active users
Using the standard update method on a testing environment showed a terrible slow speed when updating the database schema. After some calculations, seemed that the upgrade would take more than 24 hours to complete, supposing there were no errors during the update. Remember that I had to upgrade two more times, so the service would be unavailable for several days. Crazy, isn’t it?
I found out this to be a common issue when upgrading ownCloud. The database has to be updated row by row, generating a lot of subsequent read and write queries.
On a relatively small installation this could take just some minutes, but as the number of files increases, the required time to complete is way too much.
Fortunately, this instance met two conditions that made me able to shorten the upgrade process:
- It didn’t use encryption (important)
- Was using user names (
uid) as internal identifiers inside the data directory
So I came up with an alternate ownCloud upgrade process.
Does this method actually improve the upgrade time? Absolutely! It only took about 2 hours to upgrade the instance described above (~4.000.000 files). And how can this be possible? There is an easy answer to this question: given that ownCloud takes a very long time when upgrading a populated database, why not work on a clean database and restore everything at the end?
Let’s do it!
Your users should not be using ownCloud while you are in the middle of the upgrade. Deny public access to your instance using your firewall or your frontend web servers.
Before proceeding to upgrade make sure to backup the ownCloud directory and the
data directory, and dump the database. In case you use MySQL I suggest using
mysqldump like this:
Preparing the database
Instead of working on the same ownCloud database, using a copy of the database gives you an easy fall-back plan, so proceed to create a new user and database:
And then import the old ownCloud data into this new database:
Note that you should use proper user/database names instead of
suggest using ownCloud major version target (e.g.
ETags and shares
We are going to wipe our new database, so we need to keep a copy of some important data. Most important data is:
- File ETags, so ownCloud desktop clients recognize currently synchronized files
- User configured file/directory shares
Knowing exactly how to backup and restore this information depends on your source and target ownCloud versions. Have a look at some ownCloud upgrade scripts that I have released. Contributions are welcome!
Once you have both ETags and shares current data backed up, put them in a safe place until the last step.
Wipe the database
The following SQL sentences will put your ownCloud database to a mostly clean state:
Note: if you are upgrading from an old ownCloud version (e.g. 4.0.x), then
oc_filecache table will not exist, because it was called
oc_fscache. In that case, work with the
oc_fscache table instead of
Perform the ownCloud upgrade
Upgrade ownCloud as their documentation explains. Remember to copy your old
file and do not forget updating the database connection credentials to use your new
Rescan all user files
Now it’s time to let ownCloud to know about your user files. I used GNU Parallel to make it faster.
Before launching the scan, get a list of current users from the data directory, and then loop on it. Adapt the following command to your case:
The command above launches 8 parallel file scan jobs. You can control its progress
by having a look at
/tmp/scan_log and using the
This step will take most of the upgrade process time. Depending on your machine specs (storage, memory and CPU) it can take more or less time. Adjust the number of parallel processes to your system.
Restore ETags and shares
Use the data from the step ETags and shares to restore the old ownCloud state.
The ownCloud upgrade scripts project can be helpful.
Et voilá! Now you are ready to go!
About upgrading multiple releases
If you have to upgrade your ownCloud instance more than one major release, apply the database cleaning process just once (in the beginning), and do the file scan and ETags/shares restore just after upgrading to the latest ownCloud. Do not worry about intermediate versions.