CPU 100% with kswapd0 process, although no swap is needed

Question

I have this small virtual server (1GB RAM, 1 vcore), which served me well untill it suddenly started reporting 100% CPU usage because of kswapd0. Memory consumption is between 50% and 60%. I've set swappiness to 0 just to check if it releases CPU resources, but it did not. I've had this issue on 16.04, and now on 18.04 after upgrade. How to get rid of this kswapd0 CPU hog?

Answer 1

In case someone faces the same issues - the reason was malware: Multios.Coinminer.Miner.

kswapd0 was a binary file located in /root/.configrc/a/kswapd0. What you need to do is:

1. Clear crontab jobs referring to /root/.configrc
2. Clear ssh keys
3. Delete /root/.configrc

Answer 2

This is also relevant for 20.04. I caught from a Steam user adding content to the KillingFloor2 Dedicated server. Here's what it looks like.

Results of crontab -e:

Results of htop:

How to solve this problem:

1. I commented lines in the cronjob
2. Rebooted the server (CPU calmed down)
3. Deleted the .configrc folder
4. Checked again the paths of the crown and cleared it.

Answer 3

I changed swappiness from 60 to 0, but it didn't work. According to the answer from Paul, I found VIRT/RES/SHR non-zero for those CPU-hungry kswapd0 tasks, suggesting the existence of malwares.

Recent updates on similar questions also show the existence of malwares that hide as kswapd0:

1. kswapd0 is taking a lot of cpu
2. kswapd0 in ubuntu slows down the system
3. 记录被kswapd0这个挖矿病毒折磨的一天

Below is my solution. (Please use sudo, killall, rm -rf carefully)

1. Find where the fake kswapd0 is.

$ sudo find /home -name kswapd0
/home/user/.configrc5/a/kswapd0

2. Since the new server has few accounts, I double-checked that nobody is using this account, and the .configrc5 folder is relatively new (created in about one week).

$ sudo ls /home/user -al
total 224
drwxr-xr-x 18 user user   4096  四  10 23:56 .
drwxr-xr-x  8 root root   4096  四  10 20:44 ..
-rw-r--r--  1 user user    220  一  13 09:10 .bash_logout
-rw-r--r--  1 user user   3771  一  13 09:10 .bashrc
drwxrwxr-x 14 user user   4096  二  10 17:40 .cache
drwx------ 12 user user   4096  二  24 15:09 .config
drwxrwxr-x  4 user user   4096  四   4 00:48 .configrc5
...
-rw-r--r--  1 root root 131953  二  26 23:34 plm
-rw-r--r--  1 root root   6300  二  26 23:34 plm2
-rw-r--r--  1 user user    807  一  13 09:10 .profile
drwxr-xr-x  2 user user   4096  一  13 09:34 Public
drwx------  3 user user   4096  二  10 16:09 snap
drwx------  2 user user   4096  四  10 23:56 .ssh
-rw-r--r--  1 user user      0  一  13 09:51 .sudo_as_admin_successful
drwxr-xr-x  2 user user   4096  一  13 09:34 Templates
drwxr-xr-x  2 user user   4096  一  13 09:34 Videos

3. (Can skip) I thought I could kill the user account once and for all, and check that it had sudo privilege, but sudo userdel user is rejected.
4. Kill all the processes run by user

$ sudo killall -u user

5. Remove all files under .configrc5

$ sudo rm -rf /home/user/.configrc5

6. Check htop again and no strange CPU activities anymore.

Answer 4

Malware running as guest

If you have ever enabled Ubuntu's guest account and later enabled SSH, you may have malware running using your guest account.

sudo find /home -f kswapd0

It can be found in any user's home directory, such as /home/guest/.configrc/

Joining any network (conference, cafe, city) that has a compromised machine, or if you use a service like ngrok, all while having SSH open on your system leaves your computer exposed to this simple guest vulnerability.

Because privileges were never elevated (guest can't write files outside /home/guest, nor sudo without password), I was able to delete the guest account and everything is running normally.