Troubleshooting§
Logging§
Unit maintains a single general-purpose log for diagnostics and troubleshooting (not to be confused with the access log). To find out its default location in your installation:
$ unitd -h
unit options:
...
--log FILE set log filename
default: "/path/to/unit.log"
The --log option overrides the default value; if Unit is already running, check whether this option is set:
$ ps ax | grep unitd
...
unit: main v1.33.0 [/path/to/unitd ... --log /path/to/unit.log ...]
If Unit isn’t running, see its system startup scripts or configuration files to check if --log is set, and how.
Available log levels:
- [alert]: Non-fatal errors such as app exceptions or misconfigurations.
- [error]: Serious errors such as invalid ports or addresses.
- [warn]: Recoverable issues such as umount2(2) failures.
- [notice]: Self-diagnostic and router events.
- [info]: General-purpose reporting.
- [debug]: Debug events.
Note
Mind that our Docker images forward their log output to the Docker log collector instead of a file.
Router events§
The log_route option in Unit’s settings allows recording routing choices in the general-purpose log:
| Event | Log Level | Description |
|---|---|---|
| HTTP request line | [notice] | Incoming request line. |
| URI rewritten | [notice] | The request URI is updated. |
| Route step selected | [notice] | The route step is selected to serve the request. |
| Fallback taken | [notice] | A fallback action is taken after the step is selected. |
Sample router logging output may look like this:
[notice] 8308#8339 *16 http request line "GET / HTTP/1.1"
[info] 8308#8339 *16 "routes/0" discarded
[info] 8308#8339 *16 "routes/1" discarded
[notice] 8308#8339 *16 "routes/2" selected
[notice] 8308#8339 *16 URI rewritten to "/backend/"
[notice] 8308#8339 *16 "fallback" taken
It lists specific steps and actions (such as routes/2) that can be queried via the control API for details:
# curl --unix-socket /path/to/control.unit.sock http://localhost/config/routes/2
Debug Events§
Unit’s log can be set to record [debug]-level events; the steps to enable this mode vary by install method.
Warning
Debug log is meant for developers; it grows rapidly, so enable it only for detailed reports and inspection.
Our repositories provide a debug version of unitd called unitd-debug within the unit package:
# unitd-debug <command line options>
To enable debug-level logging when using our Docker images:
$ docker run -d unit:1.33.0-minimal unitd-debug --no-daemon \
--control unix:/var/run/control.unit.sock
Another option is adding a new layer in a Dockerfile:
FROM unit:1.33.0-minimal
CMD ["unitd-debug","--no-daemon","--control","unix:/var/run/control.unit.sock"]
The CMD instruction above replaces the default unitd executable with its debug version.
To enable debug-level logging when installing from source, use the --debug option:
$ ./configure --debug <other options>
Then recompile and reinstall Unit and your language modules of choice.
Core Dumps§
Core dumps help us investigate crashes; attach them when reporting an issue. For builds from our repositories, we maintain debug symbols in special packages; they have the original packages’ names with the -dbg suffix appended, such as unit-dbg.
Note
This section assumes you’re running Unit as root (recommended).
To enable saving core dumps while running Unit as a systemd service (for example, with packaged installations), adjust the service settings in /lib/systemd/system/unit.service:
[Service]
...
LimitCORE=infinity
LimitNOFILE=65535
Alternatively, update the global settings in /etc/systemd/system.conf:
[Manager]
...
DefaultLimitCORE=infinity
DefaultLimitNOFILE=65535
Next, reload the service configuration and restart Unit to reproduce the crash condition:
# systemctl daemon-reload
# systemctl restart unit.service
After a crash, locate the core dump file:
# coredumpctl -1 # optional
TIME PID UID GID SIG COREFILE EXE
Mon 2020-07-27 11:05:40 GMT 1157 0 0 11 present /usr/sbin/unitd
# ls -al /var/lib/systemd/coredump/ # default, see also /etc/systemd/coredump.conf and /etc/systemd/coredump.conf.d/*.conf
...
-rw-r----- 1 root root 177662 Jul 27 11:05 core.unitd.0.6135489c850b4fb4a74795ebbc1e382a.1157.1590577472000000.lz4
Check the core dump settings in /etc/security/limits.conf, adjusting them if necessary:
root soft core 0 # disables core dumps by default
root hard core unlimited # enables raising the size limit
Next, raise the core dump size limit with ulimit, then restart Unit to reproduce the crash condition:
# ulimit -c unlimited
# cd /path/to/unit/
# sbin/unitd # or sbin/unitd-debug
After a crash, locate the core dump file:
# ls -al /path/to/unit/working/directory/ # default location, see /proc/sys/kernel/core_pattern
...
-rw-r----- 1 root root 177662 Jul 27 11:05 core.1157
Check the core dump settings in /etc/sysctl.conf, adjusting them if necessary:
kern.coredump=1
# must be set to 1
kern.corefile=/path/to/core/files/%N.core
# must provide a valid pathname
Alternatively, update the settings in runtime:
# sysctl kern.coredump=1
# sysctl kern.corefile=/path/to/core/files/%N.core
Next, restart Unit to reproduce the crash condition. If Unit is installed as a service:
# service unitd restart
If it’s installed manually:
# cd /path/to/unit/
# sbin/unitd
After a crash, locate the core dump file:
# ls -al /path/to/core/files/
...
-rw------- 1 root root 9912320 Jul 27 11:05 unitd.core
Getting Support§
| Support Channel | Details |
|---|---|
| GitHub | Visit our repo to submit issues, suggest features, ask questions, or see the roadmap. |
| Mailing lists | To post questions to unit@nginx.org and get notifications, including release news, email unit-subscribe@nginx.org or sign up here. To receive all OSS release announcements from NGINX, join the general mailing list here. |
| Security alerts | Please report security issues to security-alert@nginx.org, specifically mentioning NGINX Unit in the subject and following the CVSS v3.1 specification. |
In addition, we offer commercial support.