Automation analysis

[simcha-vos-from-tu-delft]

I'm with the Software Engineering Research Group at the TU Delft, and I am investigating automations. I have looked at your repository and which GitHub workflows and Maven plugins you are using. I hope to provide you with some interesting insights about your repository! I'll give you a quick summary of what I found, and the automation tasks I recommend you to focus on next (:

Level of maturity	Basic	Intermediate	Advanced
Collaboration	‎ ✅ Completed this level!
	✔️ Prepare or create documentation artifacts	✔️ Generate documentation from source code ❌ Bot commits	❌ Publish documentation ❌ Commit validation ❌ Issues or PRs management
Code quality	‎ ⚠️ Still working on this level!
	✔️ Run tests ❌ Static code style analysis	✔️ Test coverage and validity ✔️ Generate test reports ❌ Automatic code formatting ❌ Static code quality analysis	✔️ Sign artifacts ❌ Verify packaging correctness ❌ Vulnerability scans ❌ License checks
Development		‎ ✅ Completed this level!
	✔️ Build files configuration	✔️ Build environment configuration	❌ Optimization
Artifacts		‎ ✅ Completed this level!
	✔️ Code compilation ✔️ Dependency management of artifact	✔️ Build tasks, resources and configuration ✔️ Packaging ✔️ Release tagging ✔️ Publish artifacts to a registry	✔️ Generate source and metadata artifacts ❌ Generate release notes ❌ Source control management ❌ Containerization

And now? Next steps!

It is not always clear which automation tasks should be prioritized. It is however important to balance your automation efforts, as a uniform level of maturity is most productive. I'm here to help! Below is a list of tasks that you can work on to help level up your maturity across the automation domains:

• Implement Static code style analysis; implemented by 9% of GitHub repositories
• Implement Automatic code formatting; implemented by 10% of GitHub repositories

What do you think?

Do you think my analysis is correct, do these recommendations help you? Or have I missed something?

For my master's thesis I'm doing research into the use of automations in GitHub repositories. With my findings, I want to see if I can help out developers with what automations they could focus on. Do you want to help me out? Leaving a response is much appreciated!

If you have any other Java/Maven projects you'd like analyzed, you can send me the repositories and I can analyze them for you!

[iSnow]

Hi, thanks for your work, I will have to dig into this, but it looks very useful.

In general, I would say the assessment is correct. There are possible false negatives/positives:

• a somewhat false negative is "Generate release notes". We craft releases manually, but use the "generate release notes" button in Github. So, we do not use a plugin here, but at least the diffs are semi-automatically added.
• false positive is "sign artifacts". As we never got around to publish to mvncentral, the GPG plugin for signing are there but unused. Maybe your scanner could be adapted to actually check for signed artifacts.

Maybe your scanner could also discriminate between libraries (like this project) and user-facing software. For us, containerization is not applicable, so it would be more useful if it didn't appear in the report.

Thanks for your work, it can only do good for the ecosystem! Actually, GitHub should implement this as a report.

[simcha-vos-from-tu-delft]

Hi @iSnow,
Thanks for your comments! Indeed you raise a valid point that it is much harder to check whether the repository is actually employing some of these automations. It's comparable to creating a workflow that never is actually triggered; the implication though is that if someone set it up, they probably either use it or are about to use it.

I agree, eventually it would be best to have different models for different kind of projects!