These days, newer devices have started coming with onboard AI chips. The correct technical term for the ‘AI chip’ is NPU, which stands for neural processing units.
If you have got an NPU on-board, you may wonder whether your system is utilizing it.
In this tutorial, I’ll share how I monitor NPU usage on my Rockchip processor.
You can check if your system has npu with this command:
dmesg | grep -i npu
Analyze the output to figure out if there is an NPU or not.
Checking Rockchip NPU utilization
Thankfully, Rockchip utilizes the debugfs feature. Debugfs is a special type of virtual filesystem that provides debug information in real time, similar to what you get with proc.
The file that will give you the NPU usage at any given time is:
/sys/kernel/debug/rknpu/load
Here’s what it shows when I ran it on my ArmSoM device. It shows the usage for all three cores of the NPU:
abhishek@armsom:~$ sudo cat /sys/kernel/debug/rknpu/load
NPU load: Core0: 0%, Core1: 0%, Core2: 0%,
abhishek@armsom:~$
As you can see, it just gives the NPU usage at the moment and ends it. Not very useful if you want to monitor the NPU usage.
An alternative can be to combine it with the watch command which will run the same command as above but every two seconds (by default):
watch sudo cat /sys/kernel/debug/rknpu/load
Here’s a screenshot I was running a LLM locally that utilized the NPU:
To stop the running watch command, press Ctrl+C.
There is another way of monitoring NPU usage. I found it when I was experimenting with the ezrknpu project to run LLMs with Rockchip NPU.
The project has a ntop.sh script that shows the NPU usage in a top command like manner. Which is not entirely true because the top command doesn’t pollute the screen. The ntop.sh script, on the other hand, floods the screen with all those lines and they don’t go away after you press Ctrl+C.
The content of the script is:
#!/bin/bash
# Title: ntop.sh
# Author: Pelochus
# Brief: A very basic ‘top’ style program that shows the status of the NPU in Rockchip’s SoCs
# Variables
CLEAR=””
# Parameters check
if [[ $1 = ‘-h’ ]]
then
echo
echo “ntop Help”
echo
echo “-c: Clears output every refresh”
echo “-h: Shows this help screen”
echo
echo “For more information visit https://github.com/Pelochus/ezrknpu”
echo
exit
elif [[ $1 = ‘-c’ ]]
then
CLEAR=”clear”
fi
while true; do
eval $CLEAR # If empty, will not clear
cat /sys/kernel/debug/rknpu/load
sleep 0.5
done
The important part is the while loop at the end. Here’s an example of NPU usage check with this script.
Conclusion
As you can see, there is no standard top like command that shows the NPU usage for various manufacturers. At present, I only have Rockchip so my exploration was limited and thus this tutorial was limited to Rockchip NPU. If I get access to more such hardware, I’ll write about them too.
Source: Read More
