Nexus Thread Dump and Heap Dump Guide

Nexus Thread Dump and Heap Dump Guide


REST thread dump - Nexus 2.5+
curl -u admin:admin123 http://localhost:8081/nexus/internal/threads > nexus.tdump

Print the Nexus PID
ps -e -o pid,command | /bin/egrep -o '^ *[0-9]{1,5} *java.*wrapper' | /bin/egrep -o '^ *[0-9]{1,5}'
This works if there is only one Nexus instance on your host.

Quick thread dump
kill -QUIT <nexus_pid> or kill -3 <nexus_pid>
This does not stop(kill) the Nexus process - merely tells the JVM to take a thread dump. Dump is in wrapper.log - simply send the zipped wrapper logs, sending multiple log files if the log files have rolled over during the thread dump.

Thread dump with locking details
jstack -l <nexus_pid>
The "-l" switch will provide detailed information about the locks that are held. Use same java version and user of that running nexus ( thread dump in wrapper.log )


Introduction

When Nexus is not performing well, the most common step to troubleshoot the issue besides examining DEBUG logs, is to gather thread dumps during the performance issue.

Preparation

The Nexus bundle uses the Java Service Wrapper (JSW) to control the Nexus application process. In practice, some default settings in Nexus prior to 2.5 made getting a thread dump from the command line in some circumstances more difficult. To make it easier to get a thread dump from Nexus, we sugggest the following changes to the JSW configuration in versions prior to Nexus 2.5 .

Disable wrapper restart after ping timeout to allow time for heap/thread dump

Set wrapper.ping.timeout=0 in "<nexus_root>/bin/jsw/conf/wrapper.conf"

Note: This is the default starting in Nexus 2.5 - see NEXUS-5678

Disable wrapper startup timeout to allow time for Nexus to start up, jetty port accessible

Set wrapper.startup.timeout=0 in "<nexus_root>/bin/jsw/conf/wrapper.conf"

Note: This is the default starting in Nexus 2.5 - see NEXUS-5678

Getting a Thread Dump in Nexus 2.5+

Nexus now exposes a simple REST call to get a minimal thread dump. Send a request to the following URL as a Nexus Administrator or a user assigned the "Metrics Endpoints" role or permission.

http://localhost:8081/nexus/internal/threads

Note: The downside of this method is that it does not print heap information as a regular thread dump or jstack -l would.

Getting a Complete Thread Dump in All Nexus Versions

Unix/Linux

Finding the Nexus Application Process ID (PID)

In order to trigger a thread dump from a terminal, you need to know the Nexus application PID. When the standard bundle is launched, two processes are started - initially the JSW wrapper process - then the actual Nexus Application process.

The Nexus control script has a 'status' cmd. The ./bin/nexus status command will print a process id. For example:

Nexus Professional is running (6893).

However, this PID will be the Java Service Wrapper process, not the Nexus application process. You cannot get a thread dump from the wrapper PID. Instead, follow steps 1 and 2 below.


Trigger a Thread Dump From the Command Line

Note: Please ensure you are running these commands as the user running the Nexus process.

Single Command

ps -e -o pid,command | /bin/egrep -o '^ *[0-9]{1,5} *java.*wrapper' | /bin/egrep -o '^ *[0-9]{1,5}' | xargs kill -3

Alternative

1) ./bin/nexus status

If Nexus is running, a PID of the JSW  wrapper process is in brackets. Note this number and proceed to step 2.

2) ps -e -o pid,ppid,command | grep wrapper.pid=<wrapper_pid> | grep nexus

 Replace <wrapper_pid> with the PID from step 1. The number that starts the resulting output should be the Nexus Application PID. Note this number and proceed to step 3

3) kill -QUIT <nexus_pid> or kill -3 <nexus_pid>

The complete thread dump should get printed in the <nexus_app>/logs/wrapper.log file.

Note: In the case of a large number of threads, the wrapper.log may roll over and be spread across more than one log file. You can adjust the size of the wrapper log file to prevent this if getting a complete thread dump in this manner is proving difficult.

Windows

Getting a thread dump from a Java process in Windows is often more difficult that under *nix based systems. Detailed steps will follow here in the near future.

Getting a Heap Dump

In very rare cases we may suggest triggering a heap dump from your Nexus instance. This is rare because %99 of the time, a thread dump + detailed logs will provide enough context for us to troubleshoot your issue. Triggering a heap dump can itself cause instability problems in a JVM if resources are being strained. It also can consume a lot of disk space.

However, in the case of your Nexus has thrown an OutOfMemoryException, a heap dump may prove extremely useful to determine what was consuming all of the memory.

Manual Heap Dump
jmap -dump:format=b,file=heapdump.bin <nexus_pid>

Automatic Heap Dump on Oracle JVM OOM
Edit $NEXUS_HOME/bin/jsw/conf/wrapper.conf. Add a line wrapper.java.additional.4=-XX:+HeapDumpOnOutOfMemoryError, where 4 is the next available unused number for java additional properties.

Have more questions? Submit a request

0 Comments

Article is closed for comments.
Powered by Zendesk