Docker includes multiple logging mechanisms to help you get information from running containers and services. These mechanisms are called logging drivers. These mechanisms are called logging drivers. Each Docker daemon has a default logging driver, which each container uses unless you configure it to use a different logging driver, or 'log.
- We would like to show you a description here but the site won't allow us.
- The fastest and easiest way to get started with Docker on Windows.
Estimated reading time: 8 minutes
Docker includes multiple logging mechanisms to help youget information from running containers and services.These mechanisms are called logging drivers. Each Docker daemon has a defaultlogging driver, which each container uses unless you configure it to use adifferent logging driver, or 'log-driver' for short.
As a default, Docker uses the json-file logging driver, whichcaches container logs as JSON internally. In addition to using the logging driversincluded with Docker, you can also implement and use logging driver plugins.
Tip: use the 'local' logging driver to prevent disk-exhaustion
By default, no log-rotation is performed. As a result, log-files stored by thedefault json-file logging driver logging driver can cause a significant amount of disk space to be used for containers that generate muchoutput, which can lead to disk space exhaustion.
Docker keeps the json-file logging driver (without log-rotation) as a defaultto remain backward compatibility with older versions of Docker, and for situationswhere Docker is used as runtime for Kubernetes.
For other situations, the 'local' logging driver is recommended as it performslog-rotation by default, and uses a more efficient file format. Refer to theConfigure the default logging driversection below to learn how to configure the 'local' logging driver as a default,and the local file logging driver page for more details about the'local' logging driver.
Configure the default logging driver
To configure the Docker daemon to default to a specific logging driver, set thevalue of log-driver to the name of the logging driver in the daemon.jsonconfiguration file. Refer to the 'daemon configuration file' section in thedockerd reference manualfor details.
The default logging driver is json-file. The following example sets the defaultlogging driver to the local log driver:
If the logging driver has configurable options, you can set them in thedaemon.json file as a JSON object with the key log-opts. The followingexample sets two configurable options on the json-file logging driver:
Restart Docker for the changes to take effect for newly created containers.Existing containers do not use the new logging configuration.
Note
log-opts configuration options in the daemon.json configuration file mustbe provided as strings. Boolean and numeric values (such as the value formax-file in the example above) must therefore be enclosed in quotes (').
If you do not specify a logging driver, the default is json-file.To find the current default logging driver for the Docker daemon, rundocker info and search for Logging Driver. You can use the followingcommand on Linux, macOS, or PowerShell on Windows:
Note
Changing the default logging driver or logging driver options in the daemonconfiguration only affects containers that are created after the configurationis changed. Existing containers retain the logging driver options that wereused when they were created. To update the logging driver for a container, thecontainer has to be re-created with the desired options.Refer to the configure the logging driver for a containersection below to learn how to find the logging-driver configuration of acontainer.
Configure the logging driver for a container
When you start a container, you can configure it to use a different loggingdriver than the Docker daemon's default, using the --log-driver flag. If thelogging driver has configurable options, you can set them using one or moreinstances of the --log-opt = flag. Even if the container uses thedefault logging driver, it can use different configurable options.
The following example starts an Alpine container with the none logging driver.
To find the current logging driver for a running container, if the daemonis using the json-file logging driver, run the following docker inspectcommand, substituting the container name or ID for :
Configure the delivery mode of log messages from container to log driver
Docker provides two modes for delivering messages from the container to the logdriver:
- (default) direct, blocking delivery from container to driver
- non-blocking delivery that stores log messages in an intermediate per-containerring buffer for consumption by driver
The non-blocking message delivery mode prevents applications from blocking dueto logging back pressure. Applications are likely to fail in unexpected ways whenSTDERR or STDOUT streams block.
Warning
When the buffer is full and a new message is enqueued, the oldest message inmemory is dropped. Dropping messages is often preferred to blocking thelog-writing process of an application.
The mode log option controls whether to use the blocking (default) ornon-blocking message delivery.
The max-buffer-size log option controls the size of the ring buffer used forintermediate message storage when mode is set to non-blocking. max-buffer-sizedefaults to 1 megabyte.
The following example starts an Alpine container with log output in non-blockingmode and a 4 megabyte buffer:
Use environment variables or labels with logging drivers
Some logging drivers add the value of a container's --env|-e or --labelflags to the container's logs. This example starts a container using the Dockerdaemon's default logging driver (let's assume json-file) but sets theenvironment variable os=ubuntu.
If the logging driver supports it, this adds additional fields to the loggingoutput. The following output is generated by the json-file logging driver:
Supported logging drivers
The following logging drivers are supported. See the link to each driver'sdocumentation for its configurable options, if applicable. If you are usinglogging driver plugins, you maysee more options.
| Driver | Description |
|---|---|
none | No logs are available for the container and docker logs does not return any output. |
local | Logs are stored in a custom format designed for minimal overhead. |
json-file | The logs are formatted as JSON. The default logging driver for Docker. |
syslog | Writes logging messages to the syslog facility. The syslog daemon must be running on the host machine. |
journald | Writes log messages to journald. The journald daemon must be running on the host machine. |
gelf | Writes log messages to a Graylog Extended Log Format (GELF) endpoint such as Graylog or Logstash. |
fluentd | Writes log messages to fluentd (forward input). The fluentd daemon must be running on the host machine. |
awslogs | Writes log messages to Amazon CloudWatch Logs. |
splunk | Writes log messages to splunk using the HTTP Event Collector. |
etwlogs | Writes log messages as Event Tracing for Windows (ETW) events. Only available on Windows platforms. |
gcplogs | Writes log messages to Google Cloud Platform (GCP) Logging. |
logentries | Writes log messages to Rapid7 Logentries. |
Note
When using Docker Engine 19.03 or older, the docker logs commandis only functional for the local, json-file and journald logging drivers.Docker 20.10 and up introduces 'dual logging', which uses a local buffer thatallows you to use the docker logs command for any logging driver. Refer toreading logs when using remote logging drivers for details.
Limitations of logging drivers
- Reading log information requires decompressing rotated log files, which causesa temporary increase in disk usage (until the log entries from the rotatedfiles are read) and an increased CPU usage while decompressing.
- The capacity of the host storage where the Docker data directory residesdetermines the maximum size of the log file information.
Estimated reading time: 8 minutes
Docker Engine volume plugins enable Engine deployments to be integrated withexternal storage systems such as Amazon EBS, and enable data volumes to persistbeyond the lifetime of a single Docker host. See theplugin documentation for more information.
Changelog
1.13.0
- If used as part of the v2 plugin architecture, mountpoints that are part ofpaths returned by the plugin must be mounted under the directory specified by
PropagatedMountin the plugin configuration(#26398)
1.12.0
- Add
Statusfield toVolumeDriver.Getresponse(#21006) - Add
VolumeDriver.Capabilitiesto get capabilities of the volume driver(#22077)
1.10.0
- Add
VolumeDriver.Getwhich gets the details about the volume(#16534) - Add
VolumeDriver.Listwhich lists all volumes owned by the driver(#16534)
1.8.0
- Initial support for volume driver plugins(#14659)
Command-line changes
To give a container access to a volume, use the --volume and --volume-driverflags on the docker container run command. The --volume (or -v) flagaccepts a volume name and path on the host, and the --volume-driver flagaccepts a driver type.
--volume
The --volume (or -v) flag takes a value that is in the format:. The two parts of the value areseparated by a colon (:) character.
- The volume name is a human-readable name for the volume, and cannot begin witha
/character. It is referred to asvolume_namein the rest of this topic. - The
Mountpointis the path on the host (v1) or in the plugin (v2) where thevolume has been made available.
volumedriver
Specifying a volumedriver in conjunction with a volumename allows you touse plugins such as Flocker to managevolumes external to a single host, such as those on EBS.
Create a VolumeDriver
The container creation endpoint (/containers/create) accepts a VolumeDriverfield of type string allowing to specify the name of the driver. If notspecified, it defaults to 'local' (the default driver for local volumes).
Volume plugin protocol
If a plugin registers itself as a VolumeDriver when activated, it mustprovide the Docker Daemon with writeable paths on the host filesystem. The Dockerdaemon provides these paths to containers to consume. The Docker daemon makesthe volumes available by bind-mounting the provided paths into the containers.
Note
Volume plugins should not write data to the /var/lib/docker/ directory,including /var/lib/docker/volumes. The /var/lib/docker/ directory isreserved for Docker.
/VolumeDriver.Create
Request:
Docker Get Bash Shell
Instruct the plugin that the user wants to create a volume, given a userspecified volume name. The plugin does not need to actually manifest thevolume on the filesystem yet (until Mount is called).Opts is a map of driver specific options passed through from the user request.
Response:
Respond with a string error if an error occurred.
/VolumeDriver.Remove
Request:
log-opts configuration options in the daemon.json configuration file mustbe provided as strings. Boolean and numeric values (such as the value formax-file in the example above) must therefore be enclosed in quotes (').
If you do not specify a logging driver, the default is json-file.To find the current default logging driver for the Docker daemon, rundocker info and search for Logging Driver. You can use the followingcommand on Linux, macOS, or PowerShell on Windows:
Note
Changing the default logging driver or logging driver options in the daemonconfiguration only affects containers that are created after the configurationis changed. Existing containers retain the logging driver options that wereused when they were created. To update the logging driver for a container, thecontainer has to be re-created with the desired options.Refer to the configure the logging driver for a containersection below to learn how to find the logging-driver configuration of acontainer.
Configure the logging driver for a container
When you start a container, you can configure it to use a different loggingdriver than the Docker daemon's default, using the --log-driver flag. If thelogging driver has configurable options, you can set them using one or moreinstances of the --log-opt = flag. Even if the container uses thedefault logging driver, it can use different configurable options.
The following example starts an Alpine container with the none logging driver.
To find the current logging driver for a running container, if the daemonis using the json-file logging driver, run the following docker inspectcommand, substituting the container name or ID for :
Configure the delivery mode of log messages from container to log driver
Docker provides two modes for delivering messages from the container to the logdriver:
- (default) direct, blocking delivery from container to driver
- non-blocking delivery that stores log messages in an intermediate per-containerring buffer for consumption by driver
The non-blocking message delivery mode prevents applications from blocking dueto logging back pressure. Applications are likely to fail in unexpected ways whenSTDERR or STDOUT streams block.
Warning
When the buffer is full and a new message is enqueued, the oldest message inmemory is dropped. Dropping messages is often preferred to blocking thelog-writing process of an application.
The mode log option controls whether to use the blocking (default) ornon-blocking message delivery.
The max-buffer-size log option controls the size of the ring buffer used forintermediate message storage when mode is set to non-blocking. max-buffer-sizedefaults to 1 megabyte.
The following example starts an Alpine container with log output in non-blockingmode and a 4 megabyte buffer:
Use environment variables or labels with logging drivers
Some logging drivers add the value of a container's --env|-e or --labelflags to the container's logs. This example starts a container using the Dockerdaemon's default logging driver (let's assume json-file) but sets theenvironment variable os=ubuntu.
If the logging driver supports it, this adds additional fields to the loggingoutput. The following output is generated by the json-file logging driver:
Supported logging drivers
The following logging drivers are supported. See the link to each driver'sdocumentation for its configurable options, if applicable. If you are usinglogging driver plugins, you maysee more options.
| Driver | Description |
|---|---|
none | No logs are available for the container and docker logs does not return any output. |
local | Logs are stored in a custom format designed for minimal overhead. |
json-file | The logs are formatted as JSON. The default logging driver for Docker. |
syslog | Writes logging messages to the syslog facility. The syslog daemon must be running on the host machine. |
journald | Writes log messages to journald. The journald daemon must be running on the host machine. |
gelf | Writes log messages to a Graylog Extended Log Format (GELF) endpoint such as Graylog or Logstash. |
fluentd | Writes log messages to fluentd (forward input). The fluentd daemon must be running on the host machine. |
awslogs | Writes log messages to Amazon CloudWatch Logs. |
splunk | Writes log messages to splunk using the HTTP Event Collector. |
etwlogs | Writes log messages as Event Tracing for Windows (ETW) events. Only available on Windows platforms. |
gcplogs | Writes log messages to Google Cloud Platform (GCP) Logging. |
logentries | Writes log messages to Rapid7 Logentries. |
Note
When using Docker Engine 19.03 or older, the docker logs commandis only functional for the local, json-file and journald logging drivers.Docker 20.10 and up introduces 'dual logging', which uses a local buffer thatallows you to use the docker logs command for any logging driver. Refer toreading logs when using remote logging drivers for details.
Limitations of logging drivers
- Reading log information requires decompressing rotated log files, which causesa temporary increase in disk usage (until the log entries from the rotatedfiles are read) and an increased CPU usage while decompressing.
- The capacity of the host storage where the Docker data directory residesdetermines the maximum size of the log file information.
Estimated reading time: 8 minutes
Docker Engine volume plugins enable Engine deployments to be integrated withexternal storage systems such as Amazon EBS, and enable data volumes to persistbeyond the lifetime of a single Docker host. See theplugin documentation for more information.
Changelog
1.13.0
- If used as part of the v2 plugin architecture, mountpoints that are part ofpaths returned by the plugin must be mounted under the directory specified by
PropagatedMountin the plugin configuration(#26398)
1.12.0
- Add
Statusfield toVolumeDriver.Getresponse(#21006) - Add
VolumeDriver.Capabilitiesto get capabilities of the volume driver(#22077)
1.10.0
- Add
VolumeDriver.Getwhich gets the details about the volume(#16534) - Add
VolumeDriver.Listwhich lists all volumes owned by the driver(#16534)
1.8.0
- Initial support for volume driver plugins(#14659)
Command-line changes
To give a container access to a volume, use the --volume and --volume-driverflags on the docker container run command. The --volume (or -v) flagaccepts a volume name and path on the host, and the --volume-driver flagaccepts a driver type.
--volume
The --volume (or -v) flag takes a value that is in the format:. The two parts of the value areseparated by a colon (:) character.
- The volume name is a human-readable name for the volume, and cannot begin witha
/character. It is referred to asvolume_namein the rest of this topic. - The
Mountpointis the path on the host (v1) or in the plugin (v2) where thevolume has been made available.
volumedriver
Specifying a volumedriver in conjunction with a volumename allows you touse plugins such as Flocker to managevolumes external to a single host, such as those on EBS.
Create a VolumeDriver
The container creation endpoint (/containers/create) accepts a VolumeDriverfield of type string allowing to specify the name of the driver. If notspecified, it defaults to 'local' (the default driver for local volumes).
Volume plugin protocol
If a plugin registers itself as a VolumeDriver when activated, it mustprovide the Docker Daemon with writeable paths on the host filesystem. The Dockerdaemon provides these paths to containers to consume. The Docker daemon makesthe volumes available by bind-mounting the provided paths into the containers.
Note
Volume plugins should not write data to the /var/lib/docker/ directory,including /var/lib/docker/volumes. The /var/lib/docker/ directory isreserved for Docker.
/VolumeDriver.Create
Request:
Docker Get Bash Shell
Instruct the plugin that the user wants to create a volume, given a userspecified volume name. The plugin does not need to actually manifest thevolume on the filesystem yet (until Mount is called).Opts is a map of driver specific options passed through from the user request.
Response:
Respond with a string error if an error occurred.
/VolumeDriver.Remove
Request:
Delete the specified volume from disk. This request is issued when a userinvokes docker rm -v to remove volumes associated with a container.
Response:
Respond with a string error if an error occurred.
/VolumeDriver.Mount
Request:
Docker requires the plugin to provide a volume, given a user specified volumename. Mount is called once per container start. If the same volume_name is requestedmore than once, the plugin may need to keep track of each new mount request and provisionat the first mount request and deprovision at the last corresponding unmount request.
ID is a unique ID for the caller that is requesting the mount.
Response:
v1:
v2:
Mountpoint is the path on the host (v1) or in the plugin (v2) where the volumehas been made available.
Err is either empty or contains an error string.
/VolumeDriver.Path
Request:
Request the path to the volume with the given volume_name.
Response:
v1:
v2:
Respond with the path on the host (v1) or inside the plugin (v2) where thevolume has been made available, and/or a string error if an error occurred.
Mountpoint is optional. However, the plugin may be queried again later if oneis not provided.
/VolumeDriver.Unmount
Request:
Docker is no longer using the named volume. Unmount is called once percontainer stop. Plugin may deduce that it is safe to deprovision the volume atthis point.
ID is a unique ID for the caller that is requesting the mount.
Response:
Respond with a string error if an error occurred.
/VolumeDriver.Get
Request:
Get info about volume_name.
Response:
v1:
v2:
Respond with a string error if an error occurred. Mountpoint and Status areoptional.
/VolumeDriver.List
Get Docker Shell
Request:
Get the list of volumes registered with the plugin.
Response:
v1:
v2:
Get Docker Shell Script
Respond with a string error if an error occurred. Mountpoint is optional.
/VolumeDriver.Capabilities
Get Docker Sh
Request:
Get the list of capabilities the driver supports.
The driver is not required to implement Capabilities. If it is notimplemented, the default values are used.
Response:
Supported scopes are global and local. Any other value in Scope will beignored, and local is used. Scope allows cluster managers to handle thevolume in different ways. For instance, a scope of global, signals to thecluster manager that it only needs to create the volume once instead of on eachDocker host. More capabilities may be added in the future.
Get Docker Sha
Examples, Usage, volume, docker, data, volumes, plugin, api