Earlier this month WordPress.org meta contributors removed the active install growth chart from plugins, sending plugin developers who relied on this data into a state of dismay and indignation. The commit cited "insufficient data obfuscation" but there was no clear communication on when and where this decision was made. The developers asked for more transparency on the removal of the graphics, but received no clear answers.
Multiple opportunities to disclose the details underlying the decision were deliberately given up as speculations increased. Several contributors not directly involved in the conversations prematurely insisted that he was removed due to security or privacy concerns, but Samuel Otto Wood unequivocally confirmed that it was none of those things.
In a recent appearance on the WPwatercooler podcast , Wood worked out the decision, which he says was made in May through private channels
"The reason is really very simple," Wood said. “It was removed because in general no one was using them. No one was using the chart itself. In general, the chart was not useful to the majority and it did not really fit the purpose we had in mind when we implemented it ”.
Wood said the active growth chart was meant to show only a plugin's growth or decline on a weekly basis, but the data didn't work as expected:
People wanted that feedback on whether the plugin was growing, shrinking, etc. And this is valuable information that developers need to have, it's valuable information that users need to know. But it really didn't work that way.
The data provided were percentage based data and were very low percentage based data. So, in general, most of the use of that data has been people scraping the data and using it to work backwards to the exact quote, to the exact numbers
That was entirely the problem was that people mostly used it to get those numbers. Now, that's not bad in itself, but the reverse math didn't work. It was wrong for a variety of reasons, mainly because we were doing it in such a way as to obfuscate the data in a way that made that number wrong.
Second, it's actually pretty funny. It actually always gave the numbers a little too high, so it gave people the wrong impression. Thirdly, actually, people trusted it as an active number, like a number of active cells to the point where, to the point where they relied on it to make decisions and things like that. It wasn't a good idea.
Although Otto was not involved in the work on the project at the time, he was privy to the discussion and reported some details:
I read all that discussion and we worked, they worked on it for a long time, Scott and several people tried various things before removing it. They adjusted the values, they adjusted the numbers. They went through a ridiculous amount of iterations and in the end, none worked. People were still using him even though he was basically giving them garbage. So in the end removing it was the only thing to do. We had a plan to replace it. We just didn't have a plan to replace it immediately. However, we have felt that giving them incorrect active install count numbers is more detrimental to the interests of users and developers than simply not providing them at all. That's why it was removed.
The concern that podcast host Sé Reed and host Matt Cromwell highlighted is that the decision was communicated in a way that suggests it was safety related. Since this wasn't a sensitive security or privacy issue, Reed asked why it was handled in a private chat instead of the meta channel when the decision had such a profound impact on developers being able to track the trajectory of their plugs. -in.
Because the inaccuracy of the graphs was well known to those who knew the problem most intimately, Wood said its removal "wasn't really a big deal" as everyone else ended up feeling it was. They did not expect the firestorm that the removal of the graphics would create in trac ticket where the developers asked to restore them.
“The physical visual chart itself isn't that crucial to how I handle things,” said GiveWP founder Matt Cromwell. “But it's the act of removing it without any conversation whatsoever.
“And what does this mean for the long series of data on plugins on.org and the viability of theirs, of us, to continue to have them? This is the real question. It is an indicator of an underlying problem that is not improving ”.
This incident sparked discussions about the type of partnership that develops himuPlugin owners should expect WordPress.org and whether it's time for them to seek mutual support instead of platform, as suggested by Eric Karkovack on The WP Minute. In light of plugin developers losing more valuable data that hasn't been replaced, Alex Denning, chief executive of Ellipsis, a digital marketing agency, argues that WordPress.org is ineffective for plugin distribution in 2022. He claims that the new WordPress plugins aren't surpassing installation thresholds of 100.000, 500.000, or 1 million and more, and the directory doesn't offer organic plug-in reach.
The ticket's goal has changed from inviting WordPress.org to report active growth charts to focus more on brainstorming helpful stats and insights into plugins that plugin developers would like to see. It is still receiving angry and frustrated comments from developers who believe the data should belong to the community.
“I can't stress enough that conversations about what to replace the active growth chart should take place in a public Slack channel or on a Trac ticket,” said Amber Hinds, CEO of Equalize Digital. "This data should belong to the community and the community should be able to participate in deciding how (or not) to display it."
The reasons allegedly requiring obfuscation have not been clearly explained, but many participants in the discussion urged WordPress.org to simply publish the raw data so that it can be accessed and processed independently of the platform. @Starbuck suggests that the community would then be able to create sites that render the data in meaningful and interesting ways.
WordPress developers want a lot more data than previously available. Hinds he asked an assortment of data points that may or may not be possible:
Things that tell us if our readme and other ranking factors are on track:
- Number of searches (or impressions) for target keywords
- Average ranking for target keywords by time period (month)
- Conversion rates from impression to install for target keywords
Things that tell us if we might have a problem with our plugin:
- Number of deactivations per time slot (month, preferably week)
- Number of eliminations per time interval (month, preferably week)
- Average time from activation to deactivation or cancellation
Things for better version testing:
- You also activate the first 20 plugins
- The first 20 themes are also active
- PHP versions (percentage)
- WordPress versions (percentage)
Atarim CEO Vito Peleg he suggested some other tools to monitor growth / decline, to which Matt Mullenweg replied that some of the ideas were "very feasible:"
... I think that perhaps instead we should have deeper tools for monitoring growth / decline:
- time to churn (to deactivate) signals good / bad onboarding, UI / UX
- repeat installs - how many users (anonymised) install on multiple sites for community opp & advocacy
...- Vito Peleg (@VitoPeleg) October 5, 2022
- Time to churn (to disable) good / bad onboarding signals, UI / UX
- Repeated installations: how many (anonymized) users install on multiple sites for community opp and advocacy activities
- Time to result: The developer can choose 1 single hook to fire as a “result” and the calculation controls how long from installation to get there. By changing the positioning of the hooks, developers can optimize entire streams.
- Internal page tracker: which / how many internal plugin pages users visit
- PHP ver distribution, country based general installations, active install / revision ratio
Wood confirmed that the active install growth charts are not reverting to their previous form and that the endpoints people were scraping earlier will remain disabled. You mentioned that those involved in the private discussion are monitoring the trac ticket for feedback.
"What will happen is that the active install count will change instead of being rounded to the nearest number," Wood said. “I don't know the exact limits of the breakpoints, but for example it shows an individual up to 50, so it rounds to the nearest 10 up to a thousand and the nearest hundred up to 10.000, for example. In this way we are making sure that the active installation contains a much finer grain than it has been. So, in that sense, yes, we are providing you with the data. They won't be exact numbers, but it will be much better than before. We are still working to do it ”.
