About a week and a half ago, I mentioned that I'd been working on making SReview, my AGPLv3 video review and transcode system work from inside a Kubernetes cluster. I noted at the time that while I'd made it work inside minikube, it couldn't actually be run from within a real Kubernetes cluster yet, mostly because I misunderstood how Kubernetes works, and assumed you could just mount the same Kubernetes volume from multiple pods, and share data that way (answer: no you can't).

The way to fix that is to share the data not through volumes, but through something else. That would require that the individual job containers download and upload files somehow.

I had a look at how the Net::Amazon::S3 perl module works (answer: it's very simple really) and whether it would be doable to add a transparent file access layer to SReview which would access files either on the local file system, or an S3 service (answer: yes).

So, as of yesterday or so, SReview supports interfacing with an S3 service (only tested with MinIO for now) rather than "just" files on the local file system. As part of that, I also updated the code so it would not re-scan all files every time the sreview-detect job for detecting new files runs, but only when the "last changed" time (or mtime for local file system access) has changed -- otherwise it would download far too many files every time.

This turned out to be a lot easier than I anticipated, and I have now successfully managed, using MinIO, to run a full run of a review cycle inside Kubernetes, without using any volumes except for the "ReadWriteOnce" ones backing the database and MinIO containers.

Additionally, my kubernetes configuration files are now split up a bit (so you can apply the things that make sense for your configuration somewhat more easily), and are (somewhat) tested.

If you want to try out SReview and you've already got Kubernetes up and running, this may be for you! Please give it a try and don't forget to send some feedback my way.