linux kernel – What do the fields in rpcdebug -c’s dmesg output mean?

I’m trying to track down a stall that may or may not be on the client host, but on the server side. Unfortunately this is all kernel RPC level, with some of if not controlled by my code…

I’m something of a neophyte when it comes to rpc debugging, so any references would be helpful.

I have this output in dmesg when rpcdebug -c -m all is run:

[2109401.599881] -pid- flgs status -client- --rqstp- -timeout ---ops--
[2109401.600055] 51580 0880      0 ffff9af4c4da9800 ffff9af4c4416600    15000 ffffffffc0b26680 nfsv3 GETATTR a:call_status [sunrpc] q:xprt_pending
[2109401.600300] 51581 0880      0 ffff9af4c4da9800 ffff9af42465f800    15000 ffffffffc0b26680 nfsv3 GETATTR a:call_status [sunrpc] q:xprt_pending

I get the PID, flags, and timeout, but:

  1. What are the “client” and “rqstp” fields supposed to mean? If I see either duplicated in subsequent outputs, does that mean the RPC is stalled? And which way?
  2. Is “xprt_pending” a “waiting to send the RPC” queue? If that queue was “delayq,” I would know what that means in the context of the problem we’re trying to diagnose. But this state doesn’t seem to be explained anywhere I find in Google. (And my Google-Fu is usually better than this…)
  3. What is the “ffffffffc0b26680” supposed to be? It repeats all over the output, for EVERY RPC listed.

I’m trying to avoid running with rpcdebug set, because I’m dealing with an intermittent stall, and I’d rather not slow EVERYTHING down in the hopes of catching the stall.

Read more here: Source link