JIYIK CN >

Logs are information recorded by the application when a specific event or state occurs. They help us monitor the application and take necessary actions.

For example, when we deploy an application to a production environment, logs can help us understand whether the application status is up or down. When an error occurs, we can use logs to determine the type of error that occurred.

With this information, we can make the necessary changes to fix the error. Managing logs can be challenging in large applications used by multiple people, but when using docker, we can manage logs easily as it provides features that help in manipulating logs.

Docker provides commands such as --details , --since , --tail , --timestamps , and --until , which we can use to manipulate logs. By default, Docker saves logs in JSON format in the folder /var/lib/docker/containers/, but please note that we must build the image and run its container as the root user to create a file containing the log.

If this is done without root privileges, we will receive the error File or directory not found. In this tutorial, we will learn the different methods that we can utilize to clear */*-json.logthe logs in the file.

Create a new project

Open WebStorm IDEA and select File > New > Project. In the window that opens, select Node.js and change the project name from untitled to docker-logs-clear or any preferred name.

Finally, press the button labeled Create to generate the project.

Create a new index.js file in the docker-logs-clear folder and copy and paste the following code into the file.

console.log("This is an express application")
console.error("An error occurred")
throw new Error("An exception occurred")

This file creates one STDOUT message and two STDERR messages that will be added to the log file when we run the application. We can change this code to a web application, but note that only the STDOUT and STDERR messages will be added to the log file.

Defining an image

Create a file Dockerfile in the docker-logs-clear folder and copy and paste the following instructions into the file.

FROM node:18-alpine
WORKDIR /com/app
ADD package*.json ./
RUN npm install
ADD . .
CMD node index.js

Here, FROM sets the base image on which subsequent instructions are used to create our custom image. In this case, we use alpine to use a lightweight image of Node as the base image.

The Docker documentation provides a detailed Dockerfile reference where we can learn more about other instructions.

Building an Image

Note that this stage must be performed as the root user. To do this, open a new terminal window using the keyboard shortcut on your computer and run the following command to change the current user to the root user.

$ sudo su

Output:

root$

Make sure to change directory cd to the location of the project as shown above to be able to build the image. Once we have gained access as the root user, we can execute the following command to build an image with the tag docker-logs:latest.

root$ docker build --tag docker-logs:latest .

This command executes our Dockerfile, and we can see the instructions being run in the terminal window as shown below.

Step 1/6 : FROM node:18-alpine
18-alpine: Pulling from library/node
213ec9aee27d: Already exists
f379b689aea3: Pull complete
fe299d5780c0: Pull complete
c34a027bbf26: Pull complete
Digest: sha256:f829c27f4f7059609e650023586726a126db25aded0c401e836cb81ab63475ff
Status: Downloaded newer image for node:18-alpine
 ---> 867dce98a500
Step 2/6 : WORKDIR /com/app
 ---> Running in 3b215b9ad992
Removing intermediate container 3b215b9ad992
 ---> aba9cfa2472b
Step 3/6 : ADD package*.json ./
 ---> 6243ccacf178
Step 4/6 : RUN npm install
 ---> Running in 9b90745b171e

added 57 packages, and audited 58 packages in 9s

7 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Removing intermediate container 9b90745b171e
 ---> e73c696d9743
Step 5/6 : ADD . .
 ---> e5235f761af0
Step 6/6 : CMD node index.js
 ---> Running in 7a857eea0a06
Removing intermediate container 7a857eea0a06
 ---> 29a367a3be2d
Successfully built 29a367a3be2d
Successfully tagged docker-logs:latest

Running the container

To run a container using the docker-logs image, execute the following command which runs a container named docker-logs-prod. Note that since it is not a web application, no ports are exposed.

root$ docker run -d --name docker-logs-prod docker-logs:latest

Check container logs

Running the container executes the code in our application. Execute the following command to view the logs generated by the container due to STDOUT and STDERR messages.

root$ docker logs -f docker-logs-prod

Output:

This is an express application
An error occurred
/com/app/index.js:3
throw new Error("An exception occurred")
^

Error: An exception occurred
    at Object.<anonymous> (/com/app/index.js:3:7)
    at Module._compile (node:internal/modules/cjs/loader:1149:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1203:10)
    at Module.load (node:internal/modules/cjs/loader:1027:32)
    at Module._load (node:internal/modules/cjs/loader:868:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:23:47

Node.js v18.10.0

View container logs in log files

To verify that the previous logs are added to our log file, use the following command to check the location of the log file where the container saves the generated logs.

root$ docker inspect --format='{{.LogPath}}' docker-logs-prod

Output:

/var/lib/docker/containers/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da-json.log

Now that we have the location where the log file is created, execute the following command to print the JSON content of the log file.

root$ cat /var/lib/docker/containers/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da-json.log

Output:

{"log":"This is an express application\n","stream":"stdout","time":"2022-10-07T10:47:16.594937015Z"}
{"log":"An error occurred\n","stream":"stderr","time":"2022-10-07T10:47:16.596273395Z"}
{"log":"/com/app/index.js:3\n","stream":"stderr","time":"2022-10-07T10:47:16.617728515Z"}
{"log":"throw new Error("An exception occurred")\n","stream":"stderr","time":"2022-10-07T10:47:16.61780931Z"}
{"log":"^\n","stream":"stderr","time":"2022-10-07T10:47:16.617822419Z"}
{"log":"\n","stream":"stderr","time":"2022-10-07T10:47:16.617832094Z"}
{"log":"Error: An exception occurred\n","stream":"stderr","time":"2022-10-07T10:47:16.617846368Z"}
{"log":"    at Object.\u003canonymous\u003e (/com/app/index.js:3:7)\n","stream":"stderr","time":"2022-10-07T10:47:16.617855581Z"}
{"log":"    at Module._compile (node:internal/modules/cjs/loader:1149:14)\n","stream":"stderr","time":"2022-10-07T10:47:16.617864838Z"}
{"log":"    at Module._extensions..js (node:internal/modules/cjs/loader:1203:10)\n","stream":"stderr","time":"2022-10-07T10:47:16.617882182Z"}
{"log":"    at Module.load (node:internal/modules/cjs/loader:1027:32)\n","stream":"stderr","time":"2022-10-07T10:47:16.617890043Z"}
{"log":"    at Module._load (node:internal/modules/cjs/loader:868:12)\n","stream":"stderr","time":"2022-10-07T10:47:16.617898124Z"}
{"log":"    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n","stream":"stderr","time":"2022-10-07T10:47:16.617906808Z"}
{"log":"    at node:internal/main/run_main_module:23:47\n","stream":"stderr","time":"2022-10-07T10:47:16.617914665Z"}
{"log":"\n","stream":"stderr","time":"2022-10-07T10:47:16.61792284Z"}
{"log":"Node.js v18.10.0\n","stream":"stderr","time":"2022-10-07T10:47:16.617930182Z"}

From the JSON content returned by the file, we can see all the logs generated by the container, including the flow and the time when the event occurred. In the next section, we will learn how to clear the logs from this file.

Clear Docker container logs using echo command

To clear the JSON content from the log file, execute the following command, which uses the echo command to overwrite the file with an empty string.

root$ echo "" > $(docker inspect --format='{{.LogPath}}' docker-logs-prod)

To verify that our log file has been cleared, execute docker logs -f docker-logs-prodthe command and notice that it does not return any logs from that file.

Use the truncate command to clear Docker container logs

Rerun the previous container to regenerate the logs and execute the following command to clear the logs added to the log file. This uses the truncate command to shrink the file to 0.

root$ truncate -s 0 $(docker inspect --format='{{.LogPath}}' docker-logs-prod)

To verify that our log file has been cleared, execute docker logs -f docker-logs-prodthe command and notice that it does not return any logs from that file.

Clear Docker container logs using log rotation

In the first two examples, we used the echo and truncate commands to manually clear the contents of the log files. These methods are not recommended because they may interfere with the Docker logging system and cause unexpected behavior.

The recommended way to clear logs from the container is to use log rotation. To use the default logging driver json file for log rotation, we can set the maximum size and maximum files of the log files in the file named daemon.json located under /etc/docker/ folder as shown below.

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

This should be done with root user privileges. For the changes to take effect, we must reload docker using the command systemctl reload docker.

If we want to specify log rotation for a single container, we can specify the --log-driver json-file , **--log-opt max-size=10m** , and --log-opt max-file=3 options when running the container.

Summarize

In this article, we have learned different ways to clear docker container logs by clearing the JSON content added to the log file. The methods we have covered in this tutorial include using echothe command, truncatecommand, and log-rotation.