Join the NGINX Unit Community Call on June 7, 2023

NGINX Unit
v. 1.30.0

Troubleshooting§

Logging§

Unit maintains a single general-purpose log for diagnostics and troubleshooting (not to be confused with the access log). To find out its default location in your installation:

$ unitd -h

    unit options:
    ...
    --log FILE           set log filename
                         default: "/path/to/unit.log"

The --log option overrides the default value; if Unit is already running, check whether this option is set:

$ ps ax | grep unitd
    ...
    unit: main v1.30.0 [/path/to/unitd ... --log /path/to/unit.log ...]

If Unit isn’t running, see its system startup scripts or configuration files to check if --log is set, and how.

Available log levels:

  • [alert]: Non-fatal errors such as app exceptions or misconfigurations.
  • [error]: Serious errors such as invalid ports or addresses.
  • [warn]: Recoverable issues such as umount2(2) failures.
  • [notice]: Self-diagnostic and router events.
  • [info]: General-purpose reporting.
  • [debug]: Debug events.

Note

Mind that our Docker images forward their log output to the Docker log collector instead of a file.

Router Events§

The log_route option in Unit’s settings allows recording routing choices in the general-purpose log:

Event Log Level Description
HTTP request line [notice] Incoming request line.
URI rewritten [notice] The request URI is updated.
Route step selected [notice] The route step is selected to serve the request.
Fallback taken [notice] A fallback action is taken after the step is selected.

Sample router logging output may look like this:

[notice] 8308#8339 *16 http request line "GET / HTTP/1.1"
[info] 8308#8339 *16 "routes/0" discarded
[info] 8308#8339 *16 "routes/1" discarded
[notice] 8308#8339 *16 "routes/2" selected
[notice] 8308#8339 *16 URI rewritten to "/backend/"
[notice] 8308#8339 *16 "fallback" taken

It lists specific steps and actions (such as routes/2) that can be queried via the control API for details:

# curl --unix-socket /path/to/control.unit.sock http://localhost/config/routes/2

Debug Events§

Unit’s log can be set to record [debug]-level events; the steps to enable this mode vary by install method.

Warning

Debug log is meant for developers; it grows rapidly, so enable it only for detailed reports and inspection.

Our repositories provide a debug version of unitd called unitd-debug within the unit package:

# unitd-debug <command line options>

To enable debug-level logging when using our Docker images:

$ docker run -d unit:1.30.0-minimal unitd-debug --no-daemon  \
      --control unix:/var/run/control.unit.sock

Another option is adding a new layer in a Dockerfile:

FROM unit:1.30.0-minimal

CMD ["unitd-debug","--no-daemon","--control","unix:/var/run/control.unit.sock"]

The CMD instruction above replaces the default unitd executable with its debug version.

To enable debug-level logging when installing from source, use the --debug option:

$ ./configure --debug <other options>

Then recompile and reinstall Unit and your language modules of choice.

Core Dumps§

Core dumps help us investigate crashes; attach them when reporting an issue. For builds from our repositories, we maintain debug symbols in special packages; they have the original packages’ names with the -dbg suffix appended, such as unit-dbg.

Note

This section assumes you’re running Unit as root (recommended).

To enable saving core dumps while running Unit as a systemd service (for example, with packaged installations), adjust the service settings in /lib/systemd/system/unit.service:

[Service]
...
LimitCORE=infinity
LimitNOFILE=65535

Alternatively, update the global settings in /etc/systemd/system.conf:

[Manager]
...
DefaultLimitCORE=infinity
DefaultLimitNOFILE=65535

Next, reload the service configuration and restart Unit to reproduce the crash condition:

# systemctl daemon-reload
# systemctl restart unit.service

After a crash, locate the core dump file:

# coredumpctl -1                     # optional

      TIME                            PID   UID   GID SIG COREFILE  EXE
      Mon 2020-07-27 11:05:40 GMT    1157     0     0  11 present   /usr/sbin/unitd

# ls -al /var/lib/systemd/coredump/  # default, see also /etc/systemd/coredump.conf and /etc/systemd/coredump.conf.d/*.conf

      ...
      -rw-r----- 1 root root 177662 Jul 27 11:05 core.unitd.0.6135489c850b4fb4a74795ebbc1e382a.1157.1590577472000000.lz4

Check the core dump settings in /etc/security/limits.conf, adjusting them if necessary:

root           soft    core       0          # disables core dumps by default
root           hard    core       unlimited  # enables raising the size limit

Next, raise the core dump size limit with ulimit, then restart Unit to reproduce the crash condition:

# ulimit -c unlimited
# cd /path/to/unit/
# sbin/unitd           # or sbin/unitd-debug

After a crash, locate the core dump file:

# ls -al /path/to/unit/working/directory/  # default location, see /proc/sys/kernel/core_pattern

      ...
      -rw-r----- 1 root root 177662 Jul 27 11:05 core.1157

Check the core dump settings in /etc/sysctl.conf, adjusting them if necessary:

kern.coredump=1
# must be set to 1
kern.corefile=/path/to/core/files/%N.core
# must provide a valid pathname

Alternatively, update the settings in runtime:

# sysctl kern.coredump=1
# sysctl kern.corefile=/path/to/core/files/%N.core

Next, restart Unit to reproduce the crash condition. If Unit is installed as a service:

# service unitd restart

If it’s installed manually:

# cd /path/to/unit/
# sbin/unitd

After a crash, locate the core dump file:

# ls -al /path/to/core/files/

      ...
      -rw-------  1 root     root  9912320 Jul 27 11:05 unitd.core

Getting Support§

Support Channel Details
GitHub Visit our repo to submit issues, suggest features, ask questions, or see the roadmap.
Mailing lists To post questions to unit@nginx.org and get notifications, including release news, email unit-subscribe@nginx.org or sign up here. To receive all OSS release announcements from NGINX, join the general mailing list here.
Security alerts Please report security issues to security-alert@nginx.org, specifically mentioning NGINX Unit in the subject and following the CVSS v3.1 specification.

In addition, we offer commercial support.

Improve this page