Redis CE 8.0 Milestone 1 release for Debian

[adamiBs]

Related links:

1. redis/docker-library-redis@8f1a23f
2. https://github.com/redis/redis/releases/tag/8.0-m01

[tianon]

Because our build system needs to handle a complex dependency tree between images, we unfortunately cannot support variables inside dependency statements (FROM lines, in our case). Additionally, ADD with a remote file has a lot of poor behaviors such that we cannot support it (again, especially not with a variable, but in general).

I also don't know that we're falling within the constraints of https://github.com/docker-library/faq#multi-stage-builds

(there are also some other failures in the CI, such as the branching / metadata that tells our system what to build and where to get it)

[adamiBs]

I've removed the variables (ARG) and the retrieval of the remote source code tarball using ADD.
Please let me know if this setup complies with the requirements from the official build system's setup.
Thanks
@tianon

[yosifkit]

I haven't looked at the full content of the Dockerfile, but here are a couple changes to fix the build failures:

From the "Diff Comment" test (and also affecting the build test):

cp: cannot stat 'tar/redis_8.0-M01-bookworm/debian/docker-entrypoint.sh': No such file or directory

Change the COPY from https://github.com/redis/docker-library-redis/blob/6c299a6986fa7aabaf9683002dc6c528a6969dc3/debian/Dockerfile#L163C1-L164C1 to COPY docker-entrypoint.sh /usr/local/bin/ so that it is relative to the Directory in the relevant section of the library/redis file.

From the "Naughty" test:

   - commit 6c299a6986fa7aabaf9683002dc6c528a6969dc3:
     - is not in the specified ref GitFetch: refs/heads/master

This one will need a GitFetch: refs/heads/release/8.0 added the 8.0 section of the library/redis file to tell our tools which branch the commit comes from.

[adamiBs]

@yosifkit I see that the diff CI passes now. Thanks for the explanations and directing me towards a setup that would work in Docker's CI system!

[yosifkit]

The entrypoint changes are causing the image to fail on the override-cmd test. This test helps ensure our "Consistency" requirement, which basically means that docker run -it --rm redis bash works for bash or any binary in the image. Because of this, I'd recommend wrapping your new functionality to only run if redis-server is the command being run. I'd also recommend using the argv array instead of relying on eval (like the set -- line earlier in the script) and checking that the .so is likely to work (like existence, size, or that it isn't a directory). So, roughly something like this?

if [ "$1" = 'redis-server' ]; then
	modules_dir='/usr/local/lib/redis/modules'

	for module in "$modules_dir"/*.so; do
		# TODO warn/error if the `.so` doesn't have a size, `-s`?
		# or is a directory, `-d`?
		# or isn't readable by the running user, `-r`? (since we'll be the `redis` user at this point)
		if [ ! -d "$module" ] && [ -s "$module" ]; then
			set -- "$@" --loadmodule "$module"
		fi
	done
fi

exec "$@"

[github-actions]

Diff for 2619c66:

diff --git a/_bashbrew-cat b/_bashbrew-cat
index ca00a36..aa426a6 100644
--- a/_bashbrew-cat
+++ b/_bashbrew-cat
@@ -1,4 +1,4 @@
-Maintainers: David Maier <[email protected]> (@dmaier-redislabs), Yossi Gottlieb <[email protected]> (@yossigo)
+Maintainers: Adam Ben Shmuel <[email protected]> (@adamiBs), Yossi Gottlieb <[email protected]> (@yossigo)
 GitRepo: https://github.com/redis/docker-library-redis.git
 
 Tags: 6.2.14, 6.2, 6, 6.2.14-bookworm, 6.2-bookworm, 6-bookworm
@@ -40,3 +40,9 @@ Tags: 7.4.0-alpine, 7.4-alpine, 7-alpine, alpine, 7.4.0-alpine3.20, 7.4-alpine3.
 Architectures: amd64, arm32v6, arm32v7, arm64v8, i386, ppc64le, riscv64, s390x
 GitCommit: 0ae559ebb19b1cafc36720e8de97839d5d402883
 Directory: 7.4/alpine
+
+Tags: 8.0-M01, 8.0-M01-bookworm
+Architectures: amd64, arm32v5, arm32v7, arm64v8, i386, mips64le, ppc64le, s390x
+GitFetch: refs/heads/release/8.0
+GitCommit: af8fe134a94d9d3ac4c696a5d8fd0096e7df6794
+Directory: debian
diff --git a/_bashbrew-list b/_bashbrew-list
index 3c986fc..e1c196f 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -38,6 +38,8 @@ redis:7.4.0
 redis:7.4.0-alpine
 redis:7.4.0-alpine3.20
 redis:7.4.0-bookworm
+redis:8.0-M01
+redis:8.0-M01-bookworm
 redis:alpine
 redis:alpine3.20
 redis:bookworm
diff --git a/_bashbrew-list-build-order b/_bashbrew-list-build-order
index c06d5a8..2ec6767 100644
--- a/_bashbrew-list-build-order
+++ b/_bashbrew-list-build-order
@@ -4,5 +4,6 @@ redis:7.0-alpine3.20
 redis:7.0-bookworm
 redis:7.2-alpine3.20
 redis:7.2-bookworm
+redis:8.0-M01-bookworm
 redis:alpine3.20
 redis:bookworm
diff --git a/redis_7.2-bookworm/Dockerfile b/redis_8.0-M01-bookworm/Dockerfile
similarity index 87%
copy from redis_7.2-bookworm/Dockerfile
copy to redis_8.0-M01-bookworm/Dockerfile
index 3e2fbd1..05644b0 100644
--- a/redis_7.2-bookworm/Dockerfile
+++ b/redis_8.0-M01-bookworm/Dockerfile
@@ -1,9 +1,3 @@
-#
-# NOTE: THIS DOCKERFILE IS GENERATED VIA "apply-templates.sh"
-#
-# PLEASE DO NOT EDIT IT DIRECTLY.
-#
-
 FROM debian:bookworm-slim
 
 # add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
@@ -22,7 +16,7 @@ RUN set -eux; \
 
 # grab gosu for easy step-down from root
 # https://github.com/tianon/gosu/releases
-ENV GOSU_VERSION 1.17
+ENV GOSU_VERSION=1.17
 RUN set -eux; \
 	savedAptMark="$(apt-mark showmanual)"; \
 	apt-get update; \
@@ -56,9 +50,9 @@ RUN set -eux; \
 	gosu --version; \
 	gosu nobody true
 
-ENV REDIS_VERSION 7.2.5
-ENV REDIS_DOWNLOAD_URL http://download.redis.io/releases/redis-7.2.5.tar.gz
-ENV REDIS_DOWNLOAD_SHA 5981179706f8391f03be91d951acafaeda91af7fac56beffb2701963103e423d
+ENV REDIS_VERSION=8.0-m01
+ENV REDIS_DOWNLOAD_URL=https://github.com/redis/redis/archive/refs/tags/8.0-m01.tar.gz
+ENV REDIS_DOWNLOAD_SHA=4c77c2218747505c50c43a45d12a067a3631a26d9397929da180e183b03e862c
 
 RUN set -eux; \
 	\
@@ -67,13 +61,36 @@ RUN set -eux; \
 	apt-get install -y --no-install-recommends \
 		ca-certificates \
 		wget \
-		\
 		dpkg-dev \
 		gcc \
 		libc6-dev \
 		libssl-dev \
 		make \
 	; \
+	\
+	arch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
+	case "$arch" in \
+		'amd64') export BUILD_WITH_MODULES=yes; export INSTALL_RUST_TOOLCHAIN=yes; export DISABLE_WERRORS=yes ;; \
+		'arm64') export BUILD_WITH_MODULES=yes; export INSTALL_RUST_TOOLCHAIN=yes; export DISABLE_WERRORS=yes ;; \
+		*) echo >&2 "Modules are NOT supported! unsupported architecture: '$arch'"; export BUILD_WITH_MODULES=no ;; \
+	esac; \
+	if [ "$BUILD_WITH_MODULES" = "yes" ]; then \
+		apt-get install -y --no-install-recommends \
+			git \
+			cmake \
+			python3 \
+			python3-pip \
+			python3-venv \
+			python3-dev \
+			unzip \
+			rsync \
+			clang \
+			automake \
+			autoconf \
+			libtool \
+			g++; \
+	fi; \
+	\
 	rm -rf /var/lib/apt/lists/*; \
 	\
 	wget -O redis.tar.gz "$REDIS_DOWNLOAD_URL"; \
@@ -97,8 +114,7 @@ RUN set -eux; \
 	gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
 	extraJemallocConfigureFlags="--build=$gnuArch"; \
 # https://salsa.debian.org/debian/jemalloc/-/blob/c0a88c37a551be7d12e4863435365c9a6a51525f/debian/rules#L8-23
-	dpkgArch="$(dpkg --print-architecture)"; \
-	case "${dpkgArch##*-}" in \
+	case "${arch##*-}" in \
 		amd64 | i386 | x32) extraJemallocConfigureFlags="$extraJemallocConfigureFlags --with-lg-page=12" ;; \
 		*) extraJemallocConfigureFlags="$extraJemallocConfigureFlags --with-lg-page=16" ;; \
 	esac; \
@@ -122,6 +138,7 @@ RUN set -eux; \
 		-exec ln -svfT 'redis-server' '{}' ';' \
 	; \
 	\
+	make -C /usr/src/redis distclean; \
 	rm -r /usr/src/redis; \
 	\
 	apt-mark auto '.*' > /dev/null; \
@@ -135,6 +152,7 @@ RUN set -eux; \
 		| xargs -r apt-mark manual \
 	; \
 	apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
+	rm -rf /var/cache/debconf/*; \
 	\
 	redis-cli --version; \
 	redis-server --version
diff --git a/redis_7.0-alpine3.20/docker-entrypoint.sh b/redis_8.0-M01-bookworm/docker-entrypoint.sh
similarity index 43%
copy from redis_7.0-alpine3.20/docker-entrypoint.sh
copy to redis_8.0-M01-bookworm/docker-entrypoint.sh
index 30406a5..114e094 100755
--- a/redis_7.0-alpine3.20/docker-entrypoint.sh
+++ b/redis_8.0-M01-bookworm/docker-entrypoint.sh
@@ -21,4 +21,39 @@ if [ "$um" = '0022' ]; then
 	umask 0077
 fi
 
-exec "$@"
+command="exec \"\$@\""
+if [ "$1" = 'redis-server' ]; then
+	echo "Starting Redis Server"
+	modules_dir="/usr/local/lib/redis/modules/"
+	
+	if [ ! -d "$modules_dir" ]; then
+		echo "Warning: Default Redis modules directory $modules_dir does not exist."
+	elif [ -n "$(ls -A $modules_dir 2>/dev/null)" ]; then
+		for module in "$modules_dir"/*.so; 
+		do
+			if [ ! -s "$module" ]; then
+				echo "Skipping module $module: file has no size."
+				continue
+			fi
+			
+			if [ -d "$module" ]; then
+				echo "Skipping module $module: is a directory."
+				continue
+			fi
+			
+			if [ ! -r "$module" ]; then
+				echo "Skipping module $module: file is not readable."
+				continue
+			fi
+
+			if [ ! -x "$module" ]; then
+				echo "Warning: Module $module is not executable."
+			fi
+			
+			command="$command --loadmodule $module"
+		done
+	fi
+fi
+
+
+eval "$command"
\ No newline at end of file

Relevant Maintainers:

• redis: @dmaier-redislabs @yossigo

[adamiBs]

@tianon / @yosifkit Could you please take a look so that I could know if we should modify anything?
Thanks 🙏

[tianon]

I still agree with @yosifkit that you ought to switch from building up a string and then evaling it to using set to add things to "$@" directly (#17549 (comment)), but I don't think that's a blocker.

[tianon]

It looks like 7.0 is EOL (unless I'm reading the dates wrong); if so, it should be removed here.

To be clear / as a refresher, removing tags here will remove them from the "Supported" section on the Hub readme (and will prevent us from spending cycles rebuilding them on the official build servers), but the tags will still be available to users who want them. (See https://github.com/docker-library/official-images#library-definition-files for more detail on this.)

[adamiBs]

Thank you for approving @tianon

I will look into both of the comments that you left and will address them accordingly in a future PR.