OK, on my weblog I have a small captchaed comment mechanism, that is used to prevent spam. The weblog is "Google free", so something self hosted needs to be used.
Not sure, if on the age of spam, it is still useful. But there we go. The setup is somewhat like this:
On my weblog there is no captcha! #
Turns out, when I look into the log of php-fpm, that a 500 is returned. There is no error message in the php-log, which is set to notice.
What could have happened ? #
I have no idea. The docker setup is specifically made like this, so things do not break internally, when there is an upgrade on the frontend.
The backend is maintained by "docker-compose", so the low brow strategy is to rebuild the containers, in case they got owned.
Debian / php breakage #
The containers are built on debian:jessie and this is out of date
apparently. I can't use debian:bullseye yet though, so I need to change my
Dockerfile:
1RUN echo "deb http://archive.debian.org/debian/ jessie main" > /etc/apt/sources.list && \
2 echo "deb-src http://archive.debian.org/debian/ jessie main" >> /etc/apt/sources.list && \
3 echo "deb http://archive.debian.org/debian-security jessie/updates main" >> /etc/apt/sources.list && \
4 echo "deb-src http://archive.debian.org/debian-security jessie/updates main" >> /etc/apt/sources.list && \
5 echo "Acquire::Check-Valid-Until false;" > /etc/apt/apt.conf.d/10-nocheckvalid
Now I rebuild all my containers manually in the proper order as to not disturb other containers on the system. The result: 500.
Time to do some cgi-bin debugging, which I wrote yesterday about.
The gist of this was:
And there you go suddenly an error out of nowhere:
PHP message: PHP Fatal error: Call to undefined function json_decode() in ...
This time the problem was that for unknown reasons the json extension was no longer loaded. I had to place it into the php.ini file. ../fpm/php.ini and then restart the php-fpm container.
Apparently within one debian version, a php extension changed from "loaded by default" to "optional", which broke my setup.
Fixed this by adding extension=json.so to the php.ini file. Restarted the
docker container. The 500 was gone. So I can now use the captcha.
Still no submissions arrived though.
ssh breakage #
My backend script loads a small yaml file, checks it for spam, censors
unwanted HTML and then commits it into a git repository. It does to over
ssh into a gitolite repository. That failed. Of course.
Basically the ssh client and the ssh server deal out which keys they will accept. The ssh server was still happy with my client key, but now the debian client (apparently) was not happy with the host key or something. I don't know, the same key worked from another machine. I'd label this again as "debian breakage". I created a new key. Problem solved.
jekyll / ruby breakage #
Jekyll is also a constant breakage point because of the number of dependencies.
Quite frankly, I don't really care all that much about jekyll new features,
so I would really like to dockerize it. Unfortunately on "www" we are running
FreeBSD, so I can't do that. Because this system updates often, jekyll also
breaks often. So I have resigned and just do not run jekyll there anymore.
Still I gotta run jekyll on my PC and there was this ruby breakage:
1 yaml_data = YAML.load( $1)
2 yaml_data[ 'content'] = $POSTMATCH
this doesn't work anymore, you need to write:
1 yaml_data = YAML.safe_load($1, permitted_classes: [String, Integer, Float, Time, Date, Hash])
you kind of wonder, if YAML.load breaks anyway, why there isn't a
"safe" default implementation.
Then there were about 70 retarded sass deprecation warnings.
I fixed 65 of them. In the end there were some deprecation warnings
about @import I could not fix. I downgraded to
gem "sass-embedded", "~> 1.78.0". I should have done this earlier and saved
myself a lot of time.
_config.yml sass deprecation flag not working #
1sass: 2 quiet_deps: truedid not work for me.
Conclusion #
And then there were unix owner and permission problems, and a git repository that was outright missing, due to a recent server migration. 😩
A weekend day ruined by admin work.