0

I have 2 SSDs set up in RAID 0 configuration with a single volume on them that I've been using without problems for 3 months straight to run a PostgreSQL database. Suddenly, today, the DB got really slow. I used to run multiple queries on huge tables and see 100% CPU usage for each postgres task (I have 12 cores). Now, one query by itself uses 15%-30% CPU, as if there's a disk bottleneck, and the same queries with the same data are taking much longer to run.

I don't know if this was happening before, but if I run iotop, I see kworker/u24:1 using 99.99% IO and 0 disk read and write. I don't know if that's normal, but it looks suspicious.

I ran benchmarks with dd and hdparm. The SSD read/write time looks fast enough; no problem there. RAM usage is fine. I see almost no swap used. I have very little free because all the free memory goes to disk caching, but that's supposed to be OK. I don't actually have any processes using up all my RAM.

What's the deal with kworker? I know it's a kernel task. Is it a problem that it's using so many IOs? Anything I should check?

Update: It stopped doing it. Not sure when or why.

4
  • 1
    I have similar behaviour due to software RAID re-synchronization. If you have 99% IO load on kworker again, could you check : cat /proc/mdstat (assuming you have software raid) or mdadm --detail ? I have RAID5 and it re-syncs every array once per month. During this, it only reads all the disks and in case of RAID0 it should only compare both copies, which is fast, so it causes bottleneck on disk read/write but keeps processor on low load. Maybe you have not noticed the two previous months (it may take only few hours depending on array size).
    – ludvik02
    May 12, 2016 at 11:28
  • @ludvik02 Thanks. Maybe that's what happened because I don't have the problem anymore. What should I be looking for when I run those commands? From mdadm, I see Update Time : Sun Dec 20 16:54:20 2015 (which is when I built the array).
    – sudo
    May 15, 2016 at 1:14
  • 1
    Unfortunaly mdadm --detail does not show when re-check happened in the past, but if it happens now, you will see the progress indicator in cat /proc/mdstat : like this [======>..............] check = 22.5% (123456/234567) . Also, you can force immediate re-check this way : echo check > /sys/block/mdX/md/sync_action . This may be useful to force it if you have known low-load periods for your server.
    – ludvik02
    May 15, 2016 at 11:38
  • @ludvik02 I didn't see a checking progress indicator this time. Alright, I'll check it next time it's behaving strangely.
    – sudo
    May 16, 2016 at 3:57

1 Answer 1

0

kworker using 100% IO doesn't mean it's using all my IOs. It means it's waiting on IO. I think the Postgres issue was separate. Never figured that one out, but it hasn't come back.

See: https://serverfault.com/questions/659164/kworker-consuming-90-io-and-zero-disk-write/785415#785415

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .