typo3_contrib.md 7.2 KB
Retour d'expérience - 2021-10-07
Retour d'expérience partagé avec la communauté Typo3 via Slack
I wanted to share our feedback about the use we make of Typo3 in our company.
Our company uses Typo3 since more than 12 years now to provide a prebuild website to each of our clients, which means that our typo3 instance hosts about 3500 websites, with specific admin accesses for each client, different editing rights according to the product they bought, a templating system, and some custom widgets we provide to them.
We recently upgraded this instance to Typo3 10.4
I wanted to share with you the pros of this system, and the limits we found by hosting this huge amount of websites on a single Typo3 instance.
- Why we chose Typo3
As far as we could find, Typo3 is the only CMS that allowed us to host hundreds of websites in a single place. The page and content edition suited our clients needs, the user and group system allowed us to manage the different access levels, and the other centralized functionnalities, builtin or extension based (files, forms, news...), satisfied our needs.
The CMS is also well maintained and get regular updates since the beginning.
- What our instance looks like
We host around 3500 websites. Most of them are really basic, composed of 10-20 pages and a contact form. Some others are bigger, with news system, private pages, custom forms, and 30+ pages.
As I said, we're now using Typo3 10.4. This installation is built with composer, uses php7.4 and mariadb 10.3, and is hosted on a dedicated apache server.
We developed 6 extensions:
- a core extension giving among other things access to our API
- a templating extension that allows admins to choose between two templates (more to come)
- a statistics extension, integrating matomo tracking for websites and a module that gives access to the websites admins to the statistics of their websites
- an optimization extension, whose only purpose is to make use of xclass / middlewares to overthrow the performance struggles caused by the huge number of websites
- an authentication ext, implementing our custom authentication system
- an admin ext, providing API and CLI commands to create, delete, or update websites automatically.
These extensions depends on the fluidtypo3/flux, fluidtypo3/vhs and typo3fluid/fluid exts. We're also using the georgringer/news and the friendsoftypo3/frontend-editing extensions.
- The performance struggles
The main challenges we faced were about perfs.
First of all, the website configuration system based on one yaml file per websites and introduced with Typo3 9 caused a huge performance loss. The time needed to parse 3500 files is really long, and opening the "Sites" backend module can last something like 30sec. We also had to rise the php limit about the max number of files that it can maitain open at the same time.
The website and page resolution were also problematics. In their primitive form, they triggered one or two db query per site, meaning that each page displayed made around 7000 db queries each time! Not only the loading time was near to 6secs, but our hosting machine had some bad times...
We fixed these issues by:
- rebuilding an inbase website configuration (we've got a 'website' table hosting those informations). The 'pages' table got a new foreign key linking it to this new table.
- overriding the \TYPO3\CMS\Frontend\Middleware\PageResolver middleware to resolve the website first with one db query on this 'website' table, then a second query in the 'pages' table to find the suited page. From 7000 queries, we're now to only 2.
- also, xclassing the now named TYPO3\CMS\Core\Routing\PageSlugCandidateProvider class, precisely the getPagesFromDatabaseForCandidates method. the way this method is designed makes the while loop to call getSiteByPageId once for each page matching the given 'slug'. But with 3500 websites, we've also got 3500 pages with the '/' slug...
The last problems we could'nt solve at this time are related to the backend when accessed by the super-admin. Because this admin can see all of the pages, sites, files, or users, the backend performances are really overstretched. I could give more feedback on this matter if needed.
I'll be glad to answer to any questions you could have about how Typo3 behaves in our use case.
Retour d'expérience, version technique abregée pour le canal des core-dev - 2021-10-07
Hi everyone, @Volker Graubaum suggested me to share this feedback with you.
My company uses typo3 to provide a website to each of our clients, meaning that we host around 3500 websites (i.e. around 120.000 pages) on a single Typo3 installation. We're now using Typo3 10.4. This installation is built with composer, uses php7.4 and mariadb 10.3, and is hosted on a dedicated apache server.
The genereal behaviour is really satisfying, but hosting that many websites and pages also presents some difficulties.
The main challenges we faced were about perfs.
First of all, the website configuration system based on one yaml file per websites and introduced with Typo3 9 caused a huge performance loss. The time needed to parse 3500 files is really long, and opening the "Sites" backend module can last something like 30sec. We also had to rise the php limit about the max number of files that it can maitain open at the same time.
The website and page resolution were also problematics. In their primitive form, they triggered one or two db query per site, meaning that each page displayed made around 7000 db queries each time! Not only the loading time was near to 6secs, but our hosting machine had some bad times...
We fixed these issues by:
- rebuilding an inbase website configuration (we've got a 'website' table hosting those informations). The 'pages' table got a new foreign key linking it to this new table.
- overriding the \TYPO3\CMS\Frontend\Middleware\PageResolver middleware to resolve the website first with one db query on this 'website' table, then a second query in the 'pages' table to find the suited page. From 7000 queries, we're now to only 2.
- also, xclassing the now named TYPO3\CMS\Core\Routing\PageSlugCandidateProvider class, precisely the getPagesFromDatabaseForCandidates method. the way this method is designed makes the while loop to call getSiteByPageId once for each page matching the given 'slug'. But with 3500 websites, we've also got 3500 pages with the '/' slug...
The last problems we could'nt solve at this time are related to the backend when accessed by the super-admin. Because this admin can see all of the pages, sites, files, or users, the backend performances are really overstretched. I could give more feedback on this matter if needed.
I would be glad to share more informations about the workarounds we found to solve these issues.
Projet publique de démonstration
Pour permettre de reproduire la problématique du grand nombre de sites et de représenter l'effet des correctifs, un container docker est créé et est publié sur github pour être partagé avec la communauté:
- Dépôt gitlab (
origin): https://gitlab.2iopenservice.com/olivier/multisites_optim - Dépôt github (
github): https://github.com/Opentalent/Typo3MultisitesOptim