We have certain Ubuntu (18.04) servers operating .NET Core apps on Kestrel. Recently the apps and OSes have been hanging, requiring entire VM restart.
What we found was the app had way too many file descriptors for network sockets. i.e. /proc/
Correspondingly, a query with lsof would reveal an ever-growing pool of "protocol:TCP" of type sock the dotnet process leaks.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
dotnet 6905 www-data 675u sock 0,9 0t0 96816 protocol: TCP
dotnet 6905 www-data 676u sock 0,9 0t0 96863 protocol: TCP
dotnet 6905 www-data 677u sock 0,9 0t0 96910 protocol: TCP
dotnet 6905 www-data 678u sock 0,9 0t0 96959 protocol: TCP
dotnet 6905 www-data 679u sock 0,9 0t0 97006 protocol: TCP
dotnet 6905 www-data 680u sock 0,9 0t0 97053 protocol: TCP
dotnet 6905 www-data 681u sock 0,9 0t0 97101 protocol: TCP
From another thread with similar reported problem, https://serverfault.com/questions/153983/sockets-found-by-lsof-but-not-by-netstat we understand that is due to some code (likely some library) in the .NET app that is opening sockets without binding IP address or port to it.
However, what we particularly like to know in this case is, what are DEVICE 0 and 9 in this context? I have seen other reports list 0,5 or 0,7 but nobody explains what those numbers represent which type of device.
