Docker

I have a very basic understanding of what docker is. My knowledge is based mainly on a couple of YT videos and a few articles. I never had an opportunity to use it. So today I decided to at least try to make my hands dirty.

Steps

I plan to do a few things in following order:

• develop an app: develop a very basic Spring Boot application (REST web sevice) that talks to a database (postgresql)
• install docker on my system
• use external image: try to somehow use an image of postgresql so that when it is running, my app can use it
• create an image with my app
• make them communicate: I expect my app in a container would communicate with a container containing database server
• use my app from localhost when both containers are started

Develop

This one is easy. My REST App has following endpoints:

Did I forget about DELETE /thought/? Yes, I did.

Install docker

I thought I would just

1

sudo apt install docker

but on official Docker page there are some more complex install instrucitons for Ubuntu. I was astonished how well things went with installing using apt repository and I could run sample image:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

[karma@tpd|~] sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete 
Digest: sha256:10d7d58d5ebd2a652f4d93fdd86da8f265f5318c6a73cc5b6a9798ff6d2b2e67
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.
(...)

I only had some minor doubts when I saw the size of all required packages:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

The following additional packages will be installed:
  docker-ce-rootless-extras docker-scan-plugin libslirp0 pigz slirp4netns
Sugerowane pakiety:
  aufs-tools cgroupfs-mount | cgroup-lite
Zostaną zainstalowane następujące NOWE pakiety:
  containerd.io docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin
  docker-scan-plugin libslirp0 pigz slirp4netns
0 aktualizowanych, 9 nowo instalowanych, 0 usuwanych i 32 nieaktualizowanych.
Konieczne pobranie 103 MB archiwów.
Po tej operacji zostanie dodatkowo użyte 432 MB miejsca na dysku.
Kontynuować? [T/n]

432MB ?

So I am more than happy that docker webpage also has clear instructions on how to uninstall docker-related packages and clean up the system…

Use external image

When I read this basic Baeldung tutorial on Spring Boot with Docker I start to undrestand that I just need to use docker compose and it would start up two containers: my application container and postgresql container. I won’t even have to download any image (at least not explicitely).

Directory structure

Here is my directory structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

▾ stoic/
  ▸ .mvn/
  ▸ .vscode/
  ▾ src/
    ▾ main/
      ▾ docker/
          .dockerignore
          docker-compose.yml
          Dockerfile
      ▸ java/
      ▸ resources/
    ▸ test/
  ▾ target/
    ▸ classes/
    ▾ generated-sources/annotations/
    ▾ generated-test-sources/test-annotations/
    ▸ maven-archiver/
    ▸ maven-status/maven-compiler-plugin/
    ▸ surefire-reports/
    ▸ test-classes/com/kamilachyla/stoic/
      stoic-0.0.1-SNAPSHOT.jar
      stoic-0.0.1-SNAPSHOT.jar.original

Here’s my Dockerfile:

1
2
3
4

FROM openjdk:17-alpine
ARG JAR_FILE=../../target/*.jar
COPY ${JAR_FILE} application.jar
ENTRYPOINT ["java", "-jar", "application.jar"]

Here’s my docker-compose.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

version: '2'

services:
  app:
    image: 'docker-stoic:latest'

    build:
      context: .
    container_name: boot-stoic-docker-app
    depends_on:
      - db
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/stoic_db
      - SPRING_DATASOURCE_USERNAME=stoic
      - SPRING_DATASOURCE_PASSWORD=stoic

  db:
    image: 'postgres:13.1-alpine'
    container_name: db
    environment:
      - POSTGRES_USER=stoic
      - POSTGRES_PASSWORD=stoic
      - POSTGRES_DB=stioc_db

Now let’s docker-compose up!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

[karma@tpd|~/d/j/s/s/s/m/docker]  (master) λ sudo docker compose up
[+] Running 9/10
 ⠿ app Error                                                                                       2.7s
 ⠿ db Pulled                                                                                      42.2s
   ⠿ 4c0d98bf9879 Pull complete                                                                    2.8s
   ⠿ 7ff5918c11c3 Pull complete                                                                    2.9s
   ⠿ c393806625cd Pull complete                                                                    3.0s
   ⠿ 9307f3bcca3a Pull complete                                                                   38.1s
   ⠿ 5eee78b95230 Pull complete                                                                   38.1s
   ⠿ c0f2174cad0e Pull complete                                                                   38.2s
   ⠿ dd6b4e21c993 Pull complete                                                                   38.2s
   ⠿ 1011823211fa Pull complete                                                                   38.3s
Sending build context to Docker daemon     546B
Step 1/4 : FROM openjdk:17-alpine
17-alpine: Pulling from library/openjdk
5843afab3874: Pull complete 
53c9466125e4: Pull complete 
d8d715783b80: Pull complete 
Digest: sha256:4b6abae565492dbe9e7a894137c966a7485154238902f2f25e9dbd9784383d81
Status: Downloaded newer image for openjdk:17-alpine
 ---> 264c9bdce361
Step 2/4 : ARG JAR_FILE=../../*.jar
 ---> Running in 98c8eb6f8ace
Removing intermediate container 98c8eb6f8ace
 ---> 9a1a007ed818
Step 3/4 : COPY ${JAR_FILE} application.jar
1 error occurred:
	* Status: COPY failed: no source files were specified, Code: 1

Oh, ARG JAR_FILE points to wrong path. So I manually copy the jar to docker directory.

By default postgres will use user name as the name of newly created database, so in order to change the default I use POSTGRES_DB env variable and use stoic_db as name (so that it is the same name as it is in the definition of SPRING_DATASOURCE_URL variable).

Both images start now as I do

1

docker compose up

Unfortunately, my Spring app complains that the database does not exist. Shouldn’t the database be created automatically as promised in postgresql image documentation ?

Before I check this problem, there is one thing I need to do: I need to add myself (i.e. me as linux user) to docker group so that I don’t have to sudo each docker command.

Add my user to docker group

This is actually explained in linux postinstall steps which I completely missed, so let’s fix this:

1
2

sudo groupadd docker
sudo usermod -aG docker $USER

Logout/login sequence is not enough for the change to actually be applied, but this one command activates the changes nicely:

1

newgrp docker

which I check by running docker run hello-world and it works - no sudo needed.

Analyze problems with my app

The problem is that the app could not connect to stoic_db database. It also complains about not having hibernate.dialect set. So I set the property in application.properties and enable creation of tables:

1
2

spring.jpa.database-platform = org.hibernate.dialect.PostgreSQL94Dialect
spring.jpa.hibernate.ddl-auto = create

but the app doesn’t start properly anyway. I look around carefully… at docker-compose.yml - and the enlightment comes instantly. Can you spot the typo I’ve made?

Yes, I misspelled database name… After fixing it everything seems fine.

How can I access my app from my host system?

I think it has something to do with port mapping. I haven’t used any networking options neither in Dockerfile nor in docker-compose.yml. I have, however, started reading networking tutorial and found out that I can:

• list networks with docker network ls:

  1 2 3 4 5 6

  [karma@tpd|~] docker network ls NETWORK ID NAME DRIVER SCOPE a38f71bf264b bridge bridge local 4e98bd840f8e docker_default bridge local 0e828702870c host host local 91ef5856ce09 none null local

• inspect networks and find out what networks are running containers connected to:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

  [karma@tpd|~] docker network inspect docker_default [ { "Name": "docker_default", "Id": "4e98bd840f8e4da8299a4fe0a21b2ad3ba76726760637f3f5571ffcb48887de6", "Created": "2022-04-28T19:32:05.703472022+02:00", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": null, "Config": [ { "Subnet": "172.18.0.0/16", "Gateway": "172.18.0.1" } ] }, "Internal": false, "Attachable": false, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": { "34bb88fe363df182fdc6c99722827aec619eabaebf708c9c23558771b65fcff5": { "Name": "boot-stoic-docker-app", "EndpointID": "8bff4b759ec2d45513f9534828c5f1200627fc559ca3e915adc2e37aab15e8d2", "MacAddress": "02:42:ac:12:00:03", "IPv4Address": "172.18.0.3/16", "IPv6Address": "" }, "fe6120c2c4bb7efd3604a02660ed84743c6b06f5dc1d159c0c57906ff1476409": { "Name": "db", "EndpointID": "03b2775aecf0996e7e1f685e9b0a3d0e7467f727f8250305004033dc9ff63d74", "MacAddress": "02:42:ac:12:00:02", "IPv4Address": "172.18.0.2/16", "IPv6Address": "" } }, "Options": {}, "Labels": { "com.docker.compose.network": "default", "com.docker.compose.project": "docker", "com.docker.compose.version": "2.3.3" } } ]

So I can see that both containers: db and boot-stoic-docker-app are using docker_network network.

Now I understand that:

• this network (docker_network) is a default network created by docker compose (as described in this compose networking tutorial
• it is named after the directory in which I run docker compose which is docker (if I were in app directory, docker compose would create a network named app_network)
• such user-created networks allow containers inside it not only use IP addresses (db container has IP address 172.18.0.2 and boot-stoic-docker-app has IP address 172.18.0.3), but also can connect between each-other using container name
  • this explains why the value of SPRING_DATASOURCE_URL in docker-compose.yml uses db - it directly refers to the name of the container

So, two containers talk to each other using docker_network. But this network is internal - I cannot use it in my host system. I need the boot-stoic-docker-app to “expose” its 8080 port (which is default port on which Spring Boot listens to http connections - to - let’s say - 8081 port on my local host (it could be the same port, but I want to make sure I understand the order of ports in “colon port mapping notation”).

PORTS convention is: local_port:container_port

To do that I need to modify my compose file and define port mapping.

Defining ports

The change to my service definition in docker-compose is simple, here’s the relevant fragment (I added two last lines):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

app:
    image: 'docker-stoic:latest'

    build:
      context: .
    container_name: boot-stoic-docker-app
    depends_on:
      - db
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://db:5432/stoic_db
      - SPRING_DATASOURCE_USERNAME=stoic
      - SPRING_DATASOURCE_PASSWORD=stoic
    ports:
      - "8081:8080"

Starting both services (app and db) is successful. I can now connect to web app from local host, however, it seems that the tables are not creeated: here is GET error:

1
2
3

[karma@tpd|~] curl localhost:8081/quote/

{"comment":"Exception handler","message":"could not extract ResultSet; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet","date":"2022-04-29T07:36:49.508429935"}⏎

So, almost done. The app is responding to queries.

Solve db issue

Core exception from the spring boot logs: Caused by: org.postgresql.util.PSQLException: ERROR: relation "quote" does not exist and I have just no idea why my spring.jpa.hibernate.ddl-auto:create is not working.

I try a few things:

• rebuild the application (perhaps my jar is old?)
• use the property of ddl-auto in docker-compose.yml, not only in application.properties (perhaps properties file is not taken into account?)
• remove conditional creation of CommandlineRunner and add some entites on my boot’s service @PostConstruct (perhaps conditions are not being evaluated correctly)?

Nothing helps.

At some point I start to modify Dockerfile and instead of FROM opejdk:17-alpine I use FROM openjdk:latest and things are getting weird: I start getting:

1

java.lang.NoClassDefFoundError: java/util/random/RandomGenerator

in the logs and the app just shuts down.

Containers cleanup

The number of containers and images grows. I’m not sure what is necessary and what is not. I also start realize (but not understand) that perhaps those layers, intermediate images or containers, container-image references and caching mechanisms make the state of the system invalid: I probably use od image or try to start (instead of rebuild) the container pointing to invaid image.

I see no other way to do some cleanup:

• get rid of all containers
• get rid of all images (except for postgres:13.1-alpine which I’m using in my docker-compose)
• I change the Dockerfile to use FROM openjdk:latest

Then I just docker compose up and… everything just works. 😄

And it even works if I start and stop the container with docker container stop boot-stoic-docker-app and docker container start boot-stoic-docker-app.

Update: 2022-05-06

I have learned two things today:

• docker compose up documentation states two interesting options: –build and –force-recreate. In order for the docker to pick-up changes in my Dockerfile it would be sufficient to just –force-recreate. However, changes in Java app require re-building of the image so I could have just run docker compose up --build.
• maven-resource-plugin is very handy as it can copy the jar automatically to the location specified by my dockerDicrectory maven property (here is the relevant stack overflow question with answers that suggested this solution):

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

  diff --git a/stoic/pom.xml b/stoic/pom.xml index 450ff88..f6ab5dd 100644 --- a/stoic/pom.xml +++ b/stoic/pom.xml @@ -15,6 +15,7 @@ <description>Web service for stoic meditation practice</description> <properties> <java.version>17</java.version> + <dockerDirectory>${project.basedir}/src/main/docker</dockerDirectory> </properties> <dependencies> <dependency> @@ -44,8 +45,37 @@ <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> + <!-- copy resulting jar file to /src/main/docker --> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-resources-plugin</artifactId> + <version>3.2.0</version> + <executions> + <execution> + <id>copy-resources</id> + <!-- here the phase you need --> + <phase>validate</phase> + <goals> + <goal>copy-resources</goal> + </goals> + <configuration> + <outputDirectory>${dockerDirectory}</outputDirectory> + <resources> + <resource> + <directory>${project.build.directory}</directory> + <includes> + <include>*.jar</include> + </includes> + </resource> + </resources> + </configuration> + </execution> + + </executions> + </plugin> </plugins> </build> + <repositories> <repository> <id>spring-milestones</id>

Conclusion

So I did it :) And I had some fun!

This excercise was just an apptizer 🍰. Now it is time to read and understand what I actually did. So my path forward would be:

• skim-read a couple of introductory articles (“docker for beginners”)
• read in-depth concepts on docker website
• read about Linux containers (or containers in general) and how they are used by software like Docker
• how Docker compares to systems like Snap (which I use on ubuntu) or flatpak (which I use on debian)
• what it takes to implement Docker yourself?

Resources

Docker related

• Docker commandline reference: https://docs.docker.com/engine/reference/commandline/docker
• post body with curl (and other curl-related commands):https://reqbin.com/req/c-d2nzjn3z/curl-post-body
• first tutorial (seem too strange) https://www.baeldung.com/spring-boot-postgresql-docker
• another one, seems simple: https://www.baeldung.com/dockerizing-spring-boot-application
• tutorial on docker site: https://docs.docker.com/language/java/build-images/

Unrelated, but helpful

• https://www.baeldung.com/maven-copy-files
• preparing data in Spring for integration tests https://www.baeldung.com/spring-boot-data-sql-and-schema-sql

This text is part of the series docker...

• 2022-21-09 - Dockerize Spring Boot App with Frontend in React
• 2022-15-09 - Docker Volumes
• 2022-28-04 - Dockerize Spring Boot REST Service with Postgresql on Docker