Future of the Theme Review process & making it smoother and faster
In this session we continued the Theme Review specific discussions from the 13th of June.
The main focus was an improved review process with a new backend.
We discussed the possible changes and differences between the current and a new system:
- The current Themes Trac would be replaced by WordPress.
- Reviewers will no longer be assigned to themes. This means that anyone can do a followup review. The reviewer would not need to wait for the reply and would be free to review another theme meanwhile.
- There will no longer be a 7 day deadline for an author to reply. The theme will wait until an update is submitted.
- The reviewer would not need to close a theme as “not approved” because of minor coding errors, only if the themes are copies, spam etc.
- The term queue will no longer be used. Reviewers will be able to decide if they want to review the oldest theme or any other theme.
- Communication between authors and reviewers would be through e-mail.
- Theme authors should have access to SVN to update their theme once it is live.
- Parsing of the readme file: multiple authors, multiple screenshots etc.
- There should also be standard texts for common errors, that could be added to a review.
We discussed how we can make the reviews available for anyone to read, and also where the accessibility review would fit in the new workflow.
Parts of the system is already in use by plugins, but it needs to be improved and completed. Once it is fully working for plugins, it will be copied over to themes. The first deadline is set to December 2017.
Another question concerned what information or feedback the theme author needs after submitting a theme. The suggestion is that instead of showing a theme’s queue position, we should show the current status (pending -awaiting review, pending -awaiting update).
Most of the points above requires the backend to be complete, but we also discussed what the Theme Review Team could do short term.
The team would need to inform theme authors about the specific format for the readme file so that it can be parsed correctly when the backend is ready. The suggested format is the current plugin readme.
We briefly discussed Theme Review requirements and the .org theme previews. It was suggested that the reviewers should allow more minor content creation and also focus more on design.
Code Automation for Themes/Plugins
- PHPCS’s problem requires certain functions and it can’t handle that.
- Theme Check doesn’t have unit tests, so it rewrote it.
- Dream: Checkboxes show green, yellow or red. Showing a theme/plugin health. Some required, some not.
- Rank according to severity of issues.
- Some people are overwhelmed with the requirements to be a TRT volunteer.
- Humans would still be used to help with the quality of themes, but just from more people, not just contributors.
- Three contexts: when developing, on upload, and continuous.
- False positives are an issue at times.
- Could be one backend, multiple front ends.
- Tag errors so they’re suppressed if it’s not really an issue.
- Local tool would help with false positives.
- People could check in code, and when it passes, it could be made public to other people.
- Huge opportunity for educating on best practices.
- WordPress coding standards could be a check, but not required.
- Plugins/Themes could include their own checks. Would need to be limited.
- Checks could give compatibility for PHP7, etc.
- Same check for WordPress versions. Knowing whether code generates warnings or errors would be helpful.
- Searching for 500 errors.
- Ticker for themes being activated WITH PHP errors.
- This could improve directory interfaces for plugins and themes.
- Can plugins be easily transferred. Need to know the expectations and norms. This way, plugins and themes could be more cared for.
- Up to versions run now to tell people that’s not up to date, and it helps with updates.
- Could be checks for mobile, etc. to push better themes to the top.
- Probably need to start small. Up to checks and go from there.
- Locally first, plugin, then server everywhere.
- Reduce amount of manual checks for accessibility checks.
- TRT is working on a lot of tools to start solving some of this.
- Visual regression testing in themes has helped some developers.
- Starter theme content could help with regression testing. What would be done with the differences? Users might want to know this.
- Do the easy stuff first.