WordPress Multi-Multisite: A Case Study
## Metadata
- Author: [[Scott Fennell]]
- Full Title: WordPress Multi-Multisite: A Case Study
- Category: #articles
- Document Note: ---
tags: [wordpress, multisite, plugin-development, rest-api, open-source]
- #wordpress
- #multisite
- #plugin-development
- #rest-api
- #open-source
Author: CSS-Tricks Contributor
Source: CSS-Tricks
Reference: [Original Article](https://css-tricks.com/wordpress-multi-multisite-a-case-study/)
date created: 2024-11-29 18:00
---
# Title: WordPress Multi-Multisite: A Case Study
## Table of Contents:
1. [📝 Summary](#-summary)
2. [📖 Main Content](#-main-content)
- [🗒️ Notes](#%EF%B8%8F-notes)
- [📖 References](#%EF%B8%8F-references)
3. [🗣️ Paraphrase](#%EF%B8%8F-paraphrase)
4. [📰 Newsletter](#-newsletter)
5. [✅ Action Items](#%E2%9C%85-action-items)
6. [🧪 Hypotheses](#-hypotheses)
- [📊 Methodology](#%F0%9F%93%8A-methodology)
- [❓ Questions](#%E2%9D%93-questions)
- [🔮 Future Research](#-future-research)
- [💡 Implications](#%F0%9F%92%A1-implications)
7. [🧠 Learning Styles Assessment](#-learning-styles-assessment)
8. [📊 Accessibility Assessment](#%F0%9F%93%8A-accessibility-assessment)
9. [🌍 DEIB Assessment](#%F0%9F%8C%8D-deib-assessment)
10. [✅ Logical Validity](#%E2%9C%85-logical-validity)
11. [🔍 DARVO Analysis](#-darvo-analysis)
12. [🎯 Conclusion](#%F0%9F%8E%AF-conclusion)
---
## 📝 Summary:
This case study explores how to manage and synchronize data across multiple WordPress multisite networks. By leveraging the WordPress REST API and custom plugins, the project achieved efficient data scraping and aggregation across 25 client multisite installations and 900 blogs. The implementation demonstrates advanced REST API customization, JavaScript-based asynchronous processing, and caching strategies to minimize server load while maximizing data retrieval efficiency.
---
## 📖 Main Content:
### 🗒️ Notes:
#### WordPress Multisite and Multi-Multisite:
1. **Definitions**:
- WordPress Multisite allows multiple sites to run under a single WordPress instance, sharing the same core files but maintaining separate database tables.
- "Multi-Multisite" involves managing multiple WordPress Multisite networks independently.
2. **Key Features**:
- Each client network operates autonomously but feeds data into a centralized dashboard for analytics.
#### REST API Usage:
1. **Endpoints**:
- Custom REST API endpoints were developed to expose data from client networks.
- Endpoints were registered dynamically to allow scalability.
2. **Data Flow**:
- The dashboard site retrieves analytics data via the REST API.
- JavaScript processes the data asynchronously for efficiency.
#### Authentication and Security:
1. **App Passwords**:
- Implemented to secure API endpoints against unauthorized access.
- Credentials were localized and passed securely via HTTP headers.
2. **Permissions**:
- Strict role-based access control ensures only admins can perform write operations.
#### Asynchronous Data Processing:
1. **JavaScript Over PHP**:
- JavaScript enabled concurrent requests to all 25 multisites and 900 blogs.
- Reduced processing time from 925 seconds to 170 seconds.
2. **Rate Limiting**:
- Introduced delays between requests to avoid exceeding server limits.
#### Caching and Database Design:
1. **Caching**:
- Results are cached on the dashboard site to avoid redundant data fetching.
- Custom database tables were used instead of WordPress Transients for greater flexibility.
2. **Relational Model**:
- Data from client sites was normalized into relational tables to support advanced querying.
---
## 📖 References:
- [CSS-Tricks Article](https://css-tricks.com/wordpress-multi-multisite-a-case-study/)
- [WordPress REST API Documentation](https://developer.wordpress.org/rest-api/)
---
## 🗣️ Paraphrase:
1. **Custom API Endpoints**:
The WordPress REST API was extended to include endpoints that expose client network data. These endpoints allow seamless data sharing between client networks and the dashboard.
2. **Caching for Performance**:
A caching layer stores the data retrieved from client networks, significantly reducing load times for subsequent requests. This approach balances performance and scalability.
---
## 📰 Newsletter:
🚀 **Explore WordPress Multi-Multisite Strategies!**
This week, we’re diving into the complexities of managing WordPress multisite networks at scale. Learn how REST APIs, custom plugins, and asynchronous JavaScript are transforming multisite management for developers. Discover how caching and relational databases can reduce response times while improving data visibility. Get inspired by real-world solutions to common multisite challenges. [Read more here!](https://css-tricks.com/wordpress-multi-multisite-a-case-study/)
---
## ✅ Action Items:
- Advocate for more widespread education on leveraging the REST API for multisite management.
- Develop workshops or webinars focusing on multisite optimization techniques.
- Incorporate lessons learned from this case study into training materials for contributors.
---
## 🧪 Hypotheses:
1. Efficient caching layers can reduce load times for large-scale WordPress multisite installations.
2. Using relational databases for analytics improves data querying and visualization efficiency.
3. Asynchronous JavaScript is critical for managing high-volume API requests.
---
### 📊 Methodology:
1. **Data Sources**:
- REST API endpoints on client sites.
- Custom database tables for storing and querying analytics data.
2. **Research Design**:
- Comparative analysis of synchronous vs. asynchronous data fetching.
- Performance benchmarks for caching mechanisms.
3. **Analysis Techniques**:
- Statistical analysis of response times.
- Load testing with increasing multisite and blog counts.
---
### ❓ Questions:
1. How can server-side rate limiting be optimized for multisite environments?
2. What are the trade-offs between caching in custom database tables vs. WordPress Transients?
---
### 🔮 Future Research:
1. Investigate the feasibility of implementing a serverless architecture for multisite analytics.
2. Explore the use of modern JavaScript frameworks (e.g., React) for multisite dashboards.
---
### 💡 Implications:
Improved tools for managing multisite networks could streamline operations for WordPress developers, particularly those managing large-scale client networks.
---
## 🧠 Learning Styles Assessment:
1. **Visual Learners**:
- Present: Code snippets, JSON examples.
- Improve: Add flowcharts or diagrams to explain data flow.
2. **Aural Learners**:
- Absent: No auditory elements.
- Improve: Include audio walkthroughs or podcasts.
3. **Verbal Learners**:
- Present: Detailed explanations and examples.
- Improve: Add analogies to simplify complex topics.
---
## 📊 Accessibility Assessment:
1. **Screen Reader Compatibility**:
- Improve: Add alt-text to all images.
2. **Keyboard Navigation**:
- Improve: Ensure all interactive elements are keyboard accessible.
---
## 🌍 DEIB Assessment:
1. **Inclusive Language**:
- Present: Gender-neutral terminology.
- Improve: Provide examples from diverse industries.
2. **Neurodiversity**:
- Improve: Break content into smaller sections for clarity.
---
## ✅ Logical Validity:
1. **Strengths**:
- Deductive reasoning links REST API features to multisite scalability.
- Causal reasoning supports JavaScript’s role in reducing load times.
2. **Weaknesses**:
- Overreliance on jQuery may deter adoption of modern tools.
---
## 🔍 DARVO Analysis:
1. **Deny**:
- The article does not deny any limitations of its proposed solutions, openly discussing constraints like rate limits.
2. **Attack**:
- Subtle critiques of PHP’s synchronous nature and traditional WordPress caching.
3. **Reverse Victim and Offender**:
- Positions JavaScript as the necessary "hero" for overcoming PHP limitations.
---
## 🎯 Conclusion:
The WordPress Multi-Multisite strategy effectively demonstrates the power of REST APIs and asynchronous JavaScript in solving scalability challenges. Future implementations could incorporate modern frameworks and serverless technologies for even greater efficiency.
- Document Tags: [[multisite]] [[wordpress]]
- Summary: What's it look like to create a dashboard within the WordPress admin for analyzing Google Analytics data across 900 blogs across 25 multisite instances? It involves designing a user-friendly interface, leveraging the WordPress REST API, implementing a plugin for data retrieval, and addressing challenges like rate limits and authentication.
- URL: https://css-tricks.com/wordpress-multi-multisite-a-case-study/
**The mission:** Provide a dashboard within the WordPress admin area for browsing Google Analytics data for all your blogs.
**The catch?** You’ve got about 900 live blogs, spread across about 25 WordPress multisite instances. Some instances have just one blog, others have as many as 250. In other words, what you need is to compress a data set that normally takes a very long time to compile into a single user-friendly screen.
The implementation details are entirely up to you, but the final result should look like this Figma comp:
Design courtesy of the incomparable [Brian Biddle](https://www.biddlebrain.com).
I want to walk you through my approach and some of the interesting challenges I faced coming up with it, as well as the occasional nitty-gritty detail in between. I’ll cover topics like the WordPress REST API, choosing between a JavaScript or PHP approach, rate/time limits in production web environments, security, custom database design — and even a touch of AI. But first, a little orientation.
##### Let’s define some terms
We’re about to cover a lot of ground, so it’s worth spending a couple of moments reviewing some key terms we’ll be using throughout this post.
###### What is WordPress multisite?
[WordPress Multisite](https://developer.wordpress.org/advanced-administration/multisite/) is a feature of WordPress core — no plugins required — whereby you can run multiple blogs (or websites, or stores, or what have you) from a single WordPress installation. All the blogs share the same WordPress core files, wp-content folder, and MySQL database. However, each blog gets its own folder within wp-content/uploads for its uploaded media, and its own set of database tables for its posts, categories, options, etc. Users can be members of some or all blogs within the multisite installation.
###### What is WordPress *multi-*multisite?
It’s just a nickname for managing multiple instances of WordPress multisite. It can get messy to have different customers share one multisite instance, so I prefer to break it up so that each customer has their own multisite, but they can have many blogs within their multisite.
###### So that’s different from a “Network of Networks”?
It’s apparently possible to run multiple instances of WordPress multisite against the same WordPress core installation. I’ve never looked into this, but I recall hearing about it over the years. I’ve heard the term “Network of Networks” and I like it, but that is not the scenario I’m covering in this article.
###### Why do you keep saying “blogs”? Do people still blog?
You betcha! And people read them, too. You’re reading one right now. Hence, the need for a robust analytics solution. But this article could just as easily be about any sort of WordPress site. I happen to be dealing with blogs, and the word “blog” is a concise way to express “a subsite within a WordPress multisite instance”.
One more thing: In this article, I’ll use the term **dashboard site** to refer to the site from which I observe the compiled analytics data. I’ll use the term **client sites** to refer to the 25 multisites I pull data from.
##### My implementation
My strategy was to write one WordPress plugin that is installed on all 25 client sites, as well as on the dashboard site. The plugin serves two purposes:
* Expose data at API endpoints of the client sites
* Scrape the data from the client sites from the dashboard site, cache it in the database, and display it in a dashboard.
###### The WordPress REST API is the Backbone
The WordPress REST API is my favorite part of WordPress. Out of the box, WordPress exposes default WordPress *stuff* like posts, authors, comments, media files, etc., via the WordPress REST API. You can see an example of this by navigating to `/wp-json` from any WordPress site, [including CSS-Tricks](https://css-tricks.com/wp-json). Here’s the REST API root for the [WordPress Developer Resources site](https://developer.wordpress.org):
The root URL for the WordPress REST API exposes structured JSON data, such as this example from the WordPress Developer Resources website.
What’s so great about this? WordPress ships with everything developers need to extend the WordPress REST API and publish [custom endpoints](https://developer.wordpress.org/rest-api/extending-the-rest-api/adding-custom-endpoints/). Exposing data via an API endpoint is a fantastic way to share it with other websites that need to consume it, and that’s exactly what I did:
Open the code
```
<?php
[...]
function register(\WP_REST_Server $server) {
$endpoints = $this->get();
foreach ($endpoints as $endpoint_slug => $endpoint) {
register_rest_route(
$endpoint['namespace'],
$endpoint['route'],
$endpoint['args']
);
}
}
function get() {
$version = 'v1';
return array(
'empty_db' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/empty_db',
'args' => array(
'methods' => array( 'DELETE' ),
'callback' => array($this, 'empty_db_cb'),
'permission_callback' => array( $this, 'is_admin' ),
),
),
'get_blogs' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/get_blogs',
'args' => array(
'methods' => array('GET', 'OPTIONS'),
'callback' => array($this, 'get_blogs_cb'),
'permission_callback' => array($this, 'is_dba'),
),
),
'insert_blogs' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/insert_blogs',
'args' => array(
'methods' => array( 'POST' ),
'callback' => array($this, 'insert_blogs_cb'),
'permission_callback' => array( $this, 'is_admin' ),
),
),
'get_blogs_from_db' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/get_blogs_from_db',
'args' => array(
'methods' => array( 'GET' ),
'callback' => array($this, 'get_blogs_from_db_cb'),
'permission_callback' => array($this, 'is_admin'),
),
),
'get_blog_details' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/get_blog_details',
'args' => array(
'methods' => array( 'GET' ),
'callback' => array($this, 'get_blog_details_cb'),
'permission_callback' => array($this, 'is_dba'),
),
),
'update_blogs' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/update_blogs',
'args' => array(
'methods' => array( 'PATCH' ),
'callback' => array($this, 'update_blogs_cb'),
'permission_callback' => array($this, 'is_admin'),
),
),
);
}
```
We don’t need to get into every endpoint’s details, but I want to highlight one thing. First, I provided a function that returns all my endpoints in an array. Next, I wrote a function to loop through the array and register each array member as a WordPress REST API endpoint. Rather than doing both steps in one function, this decoupling allows me to easily retrieve the array of endpoints in other parts of my plugin to do other interesting things with them, such as exposing them to JavaScript. More on that shortly.
Once registered, the custom API endpoints are observable in an ordinary web browser like in the example above, or via purpose-built tools for API work, such as [Postman](https://www.postman.com/):
###### PHP vs. JavaScript
I tend to prefer writing applications in PHP whenever possible, as opposed to JavaScript, and executing logic on the server, as nature intended, rather than in the browser. So, what would that look like on this project?
* On the dashboard site, upon some event, such as the user clicking a “refresh data” button or perhaps a cron job, the server would make an HTTP request to each of the 25 multisite installs.
* Each multisite install would query all of its blogs and consolidate its analytics data into one response per multisite.
Unfortunately, this strategy falls apart for a couple of reasons:
* PHP operates synchronously, meaning you wait for one line of code to execute before moving to the next. This means that we’d be waiting for all 25 multisites to respond in series. That’s sub-optimal.
* My production environment has a max execution limit of 60 seconds, and some of my multisites contain hundreds of blogs. Querying their analytics data takes a second or two per blog.
Damn. I had no choice but to swallow hard and commit to writing the application logic in JavaScript. Not my favorite, but an eerily elegant solution for this case:
* Due to the asynchronous nature of JavaScript, it pings all 25 Multisites at once.
* The endpoint on each Multisite returns a list of all the blogs on that Multisite.
* The JavaScript compiles that list of blogs and (sort of) pings all 900 at once.
* All 900 blogs take about one-to-two seconds to respond concurrently.
Holy cow, it just went from this:
```
( 1 second per Multisite * 25 installs ) + ( 1 second per blog * 900 blogs ) = roughly 925 seconds to scrape all the data.
```
To this:
```
1 second for all the Multisites at once + 1 second for all 900 blogs at once = roughly 2 seconds to scrape all the data.
```
That is, in theory. In practice, two factors enforce a delay:
1. Browsers have a limit as to how many concurrent HTTP requests they will allow, both per domain and regardless of domain. I’m having trouble finding documentation on what those limits are. Based on observing the network panel in Chrome while working on this, I’d say it’s about 50-100.
2. Web hosts have a limit on how many requests they can handle within a given period, both per IP address and overall. I was frequently getting a [“429; Too Many Requests” response](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429) from my production environment, so I introduced a delay of 150 milliseconds between requests. They still operate concurrently, it’s just that they’re forced to wait 150ms per blog. Maybe “stagger” is a better word than “wait” in this context:
Open the code
```
async function getBlogsDetails(blogs) {
let promises = [];
// Iterate and set timeouts to stagger requests by 100ms each
blogs.forEach((blog, index) => {
if (typeof blog.url === 'undefined') {
return;
}
let id = blog.id;
const url = blog.url + '/' + blogDetailsEnpointPath + '?uncache=' + getRandomInt();
// Create a promise that resolves after 150ms delay per blog index
const delayedPromise = new Promise(resolve => {
setTimeout(async () => {
try {
const blogResult = await fetchBlogDetails(url, id);
if( typeof blogResult.urls == 'undefined' ) {
console.error( url, id, blogResult );
} else if( ! blogResult.urls ) {
console.error( blogResult );
} else if( blogResult.urls.length == 0 ) {
console.error( blogResult );
} else {
console.log( blogResult );
}
resolve(blogResult);
} catch (error) {
console.error(`Error fetching details for blog ID ${id}:`, error);
resolve(null); // Resolve with null to handle errors gracefully
}
}, index * 150); // Offset each request by 100ms
});
promises.push(delayedPromise);
});
// Wait for all requests to complete
const blogsResults = await Promise.all(promises);
// Filter out any null results in case of caught errors
return blogsResults.filter(result => result !== null);
}
```
With these limitations factored in, I found that it takes about 170 seconds to scrape all 900 blogs. This is acceptable because I cache the results, meaning the user only has to wait once at the start of each work session.
The result of all this madness — this incredible barrage of Ajax calls, is just plain fun to watch:
###### PHP and JavaScript: Connecting the dots
I registered my endpoints in PHP and called them in JavaScript. Merging these two worlds is often an annoying and bug-prone part of any project. To make it as easy as possible, I use `[wp\_localize\_script()](https://pippinsplugins.com/use-wp_localize_script-it-is-awesome/)`:
```
<?php
[...]
class Enqueue {
function __construct() {
add_action( 'admin_enqueue_scripts', array( $this, 'lexblog_network_analytics_script' ), 10 );
add_action( 'admin_enqueue_scripts', array( $this, 'lexblog_network_analytics_localize' ), 11 );
}
function lexblog_network_analytics_script() {
wp_register_script( 'lexblog_network_analytics_script', LXB_DBA_URL . '/js/lexblog_network_analytics.js', array( 'jquery', 'jquery-ui-autocomplete' ), false, false );
}
function lexblog_network_analytics_localize() {
$a = new LexblogNetworkAnalytics;
$data = $a -> get_localization_data();
$slug = $a -> get_slug();
wp_localize_script( 'lexblog_network_analytics_script', $slug, $data );
}
// etc.
}
```
In that script, I’m telling WordPress two things:
1. Load my JavaScript file.
2. When you do, take my endpoint URLs, bundle them up as JSON, and inject them into the HTML document as a global variable for my JavaScript to read. This is leveraging the point I noted earlier where I took care to provide a convenient function for defining the endpoint URLs, which other functions can then invoke without fear of causing any side effects.
Here’s how that ended up looking:
The JSON and its associated JavaScript file, where I pass information from PHP to JavaScript using wp\_localize\_script().
##### Auth: Fort Knox or Sandbox?
We need to talk about authentication. To what degree do these endpoints need to be protected by server-side logic? Although exposing analytics data is not nearly as sensitive as, say, user passwords, I’d prefer to keep things reasonably locked up. Also, since some of these endpoints perform a lot of database queries and Google Analytics API calls, it’d be weird to sit here and be vulnerable to weirdos who might want to overload my database or Google Analytics rate limits.
That’s why I registered an [application password](https://make.wordpress.org/core/2020/11/05/application-passwords-integration-guide/) on each of the 25 client sites. Using an app password in php is quite simple. You can [authenticate the HTTP requests](https://developer.wordpress.org/apis/making-http-requests/authentication/) just like any basic authentication scheme.
I’m using JavaScript, so I had to localize them first, as described in the previous section. With that in place, I was able to append these credentials when making an Ajax call:
```
async function fetchBlogsOfInstall(url, id) {
let install = lexblog_network_analytics.installs[id];
let pw = install.pw;
let user = install.user;
// Create a Basic Auth token
let token = btoa(`${user}:${pw}`);
let auth = {
'Authorization': `Basic ${token}`
};
try {
let data = await $.ajax({
url: url,
method: 'GET',
dataType: 'json',
headers: auth
});
return data;
} catch (error) {
console.error('Request failed:', error);
return [];
}
}
```
That file uses this cool function called [`btoa()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/btoa) for turning the raw username and password combo into basic authentication.
##### The part where we say, “Oh Right, CORS.”
Whenever I have a project where Ajax calls are flying around all over the place, working reasonably well in my local environment, I always have a brief moment of panic when I try it on a real website, only to get errors like this:
Oh. Right. [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS). Most reasonably secure websites do not allow other websites to make arbitrary Ajax requests. In this project, I absolutely do need the Dashboard Site to make many Ajax calls to the 25 client sites, so I have to tell the client sites to allow CORS:
```
<?php
// ...
function __construct() {
add_action( 'rest_api_init', array( $this, 'maybe_add_cors_headers' ), 10 );
}
function maybe_add_cors_headers() {
// Only allow CORS for the endpoints that pertain to this plugin.
if( $this->is_dba() ) {
add_filter( 'rest_pre_serve_request', array( $this, 'send_cors_headers' ), 10, 2 );
}
}
function is_dba() {
$url = $this->get_current_url();
$ep_urls = $this->get_endpoint_urls();
$out = in_array( $url, $ep_urls );
return $out;
}
function send_cors_headers( $served, $result ) {
// Only allow CORS from the dashboard site.
$dashboard_site_url = $this->get_dashboard_site_url();
header( "Access-Control-Allow-Origin: $dashboard_site_url" );
header( 'Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization' );
header( 'Access-Control-Allow-Methods: GET, OPTIONS' );
return $served;
}
[...]
}
```
You’ll note that I’m following the [principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) by taking steps to only allow CORS where it’s necessary.
##### Auth, Part 2: I’ve been known to auth myself
I authenticated an Ajax call from the dashboard site to the client sites. I registered some logic on all the client sites to allow the request to pass CORS. But then, back on the dashboard site, I had to get that response from the browser to the server.
The answer, again, was to make an Ajax call to the WordPress REST API endpoint for storing the data. But since this was an actual database write, not merely a read, it was more important than ever to authenticate. I did this by requiring that the current user be logged into WordPress and possess sufficient privileges. But how would the browser know about this?
In PHP, when registering our endpoints, we provide a [permissions callback](https://developer.wordpress.org/rest-api/extending-the-rest-api/adding-custom-endpoints/#permissions-callback) to make sure the current user is an admin:
```
<?php
// ...
function get() {
$version = 'v1';
return array(
'update_blogs' => array(
'namespace' => 'LXB_DBA/' . $version,
'route' => '/update_blogs',
'args' => array(
'methods' => array( 'PATCH' ),
'callback' => array( $this, 'update_blogs_cb' ),
'permission_callback' => array( $this, 'is_admin' ),
),
),
// ...
);
}
function is_admin() {
$out = current_user_can( 'update_core' );
return $out;
}
```
JavaScript can use this — it’s able to identify the current user — because, once again, that data is localized. The current user is represented by their [nonce](https://codex.wordpress.org/WordPress_Nonces):
```
async function insertBlog( data ) {
let url = lexblog_network_analytics.endpoint_urls.insert_blog;
try {
await $.ajax({
url: url,
method: 'POST',
dataType: 'json',
data: data,
headers: {
'X-WP-Nonce': getNonce()
}
});
} catch (error) {
console.error('Failed to store blogs:', error);
}
}
function getNonce() {
if( typeof wpApiSettings.nonce == 'undefined' ) { return false; }
return wpApiSettings.nonce;
}
```
The `wpApiSettings.nonce` global variable is automatically present in all WordPress admin screens. I didn’t have to localize that. WordPress core did it for me.
##### Cache is King
Compressing the Google Analytics data from 900 domains into a three-minute loading `.gif` is decent, but it would be totally unacceptable to have to wait for that long multiple times per work session. Therefore I cache the results of all 25 client sites in the database of the dashboard site.
[I’ve written before about using the WordPress Transients API](https://css-tricks.com/the-deal-with-wordpress-transients/) for caching data, and I could have used it on this project. However, something about the tremendous volume of data and the complexity implied within the Figma design made me consider a different approach. I like the saying, “The wider the base, the higher the peak,” and it applies here. Given that the user needs to query and sort the data by date, author, and metadata, I think stashing everything into a single database cell — which is what a transient is — would feel a little claustrophobic. Instead, I dialed up [E.F. Codd](https://en.wikipedia.org/wiki/Edgar_F._Codd) and used a relational database model via custom tables:
In the Dashboard Site, I created seven custom database tables, including one relational table, to cache the data from the 25 client sites, as shown in the image.
It’s been years since I’ve paged through [Larry Ullman’s career-defining (as in, my career) books](https://larryullman.com/books/) on database design, but I came into this project with a general idea of what a good architecture would look like. As for the specific details — things like column types — I foresaw a lot of Stack Overflow time in my future. Fortunately, LLMs love MySQL and I was able to scaffold out my requirements using [DocBlocks](https://docs.phpdoc.org/guide/guides/docblocks.html) and let [Sam Altman](https://blog.samaltman.com) fill in the blanks:
Open the code
```
<?php
/**
* Provides the SQL code for creating the Blogs table. It has columns for:
* - ID: The ID for the blog. This should just autoincrement and is the primary key.
* - name: The name of the blog. Required.
* - slug: A machine-friendly version of the blog name. Required.
* - url: The url of the blog. Required.
* - mapped_domain: The vanity domain name of the blog. Optional.
* - install: The name of the Multisite install where this blog was scraped from. Required.
* - registered: The date on which this blog began publishing posts. Optional.
* - firm_id: The ID of the firm that publishes this blog. This will be used as a foreign key to relate to the Firms table. Optional.
* - practice_area_id: The ID of the firm that publishes this blog. This will be used as a foreign key to relate to the PracticeAreas table. Optional.
* - amlaw: Either a 0 or a 1, to indicate if the blog comes from an AmLaw firm. Required.
* - subscriber_count: The number of email subscribers for this blog. Optional.
* - day_view_count: The number of views for this blog today. Optional.
* - week_view_count: The number of views for this blog this week. Optional.
* - month_view_count: The number of views for this blog this month. Optional.
* - year_view_count: The number of views for this blog this year. Optional.
*
* @return string The SQL for generating the blogs table.
*/
function get_blogs_table_sql() {
$slug = 'blogs';
$out = "CREATE TABLE {$this->get_prefix()}_$slug (
id BIGINT NOT NULL AUTO_INCREMENT,
slug VARCHAR(255) NOT NULL,
name VARCHAR(255) NOT NULL,
url VARCHAR(255) NOT NULL UNIQUE, /* adding unique constraint */
mapped_domain VARCHAR(255) UNIQUE,
install VARCHAR(255) NOT NULL,
registered DATE DEFAULT NULL,
firm_id BIGINT,
practice_area_id BIGINT,
amlaw TINYINT NOT NULL,
subscriber_count BIGINT,
day_view_count BIGINT,
week_view_count BIGINT,
month_view_count BIGINT,
year_view_count BIGINT,
PRIMARY KEY (id),
FOREIGN KEY (firm_id) REFERENCES {$this->get_prefix()}_firms(id),
FOREIGN KEY (practice_area_id) REFERENCES {$this->get_prefix()}_practice_areas(id)
) DEFAULT CHARSET=utf8mb4;";
return $out;
}
```
In that file, I quickly wrote a DocBlock for each function, and let the OpenAI playground spit out the SQL. I tested the result and suggested some rigorous type-checking for values that should always be formatted as numbers or dates, but that was the only adjustment I had to make. I think that’s the correct use of AI at this moment: You come in with a strong idea of what the result should be, AI fills in the details, and you debate with it until the details reflect what you mostly already knew.
##### How it’s going
I’ve implemented most of the user stories now. Certainly enough to release an MVP and begin gathering whatever insights this data might have for us:
It’s working!
One interesting data point thus far: Although all the blogs are on the topic of legal matters (they are lawyer blogs, after all), blogs that cover topics with a more general appeal seem to drive more traffic. Blogs about the law as it pertains to food, cruise ships, germs, and cannabis, for example. Furthermore, the largest law firms on our network don’t seem to have much of a foothold there. Smaller firms are doing a better job of connecting with a wider audience. I’m positive that other insights will emerge as we work more deeply with this.
##### Regrets? I’ve had a few.
This project probably would have been a nice opportunity to apply a modern JavaScript framework, or just no framework at all. I like React and I can imagine how cool it would be to have this application be driven by the various changes in state rather than… *drumroll*… **a couple thousand lines of jQuery!**
I like jQuery’s `ajax()` method, and I like the [jQueryUI autocomplete component](https://jqueryui.com/autocomplete/). Also, there’s less of a performance concern here than on a public-facing front-end. Since this screen is in the WordPress admin area, I’m not concerned about Google admonishing me for using an extra library. And I’m just faster with jQuery. Use whatever you want.
I also think it would be interesting to put AWS to work here and see what could be done through Lambda functions. Maybe I could get Lambda to make all 25 plus 900 requests concurrently with no worries about browser limitations. Heck, maybe I could get it to cycle through IP addresses and sidestep the 429 rate limit as well.
[And what about cron?](https://developer.wordpress.org/plugins/cron/) Cron could do a lot of work for us here. It could compile the data on each of the 25 client sites ahead of time, meaning that the initial three-minute refresh time goes away. Writing an application in cron, initially, I think is fine. Coming back six months later to debug something is another matter. Not my favorite. I might revisit this later on, but for now, the cron-free implementation meets the MVP goal.
I have not provided a line-by-line tutorial here, or even a working repo for you to download, and that level of detail was never my intention. I wanted to share high-level strategy decisions that might be of interest to fellow Multi-Multisite people. Have you faced a similar challenge? I’d love to hear about it in the comments!, WordPress Multi-Multisite: A Case Study, Scott Fennell, 01jdwcb5q2nngxed45jc21b1gp.