How does GoLang know how many CPUs to use?

English 简体中文 繁体中文 ภาษาไทย Tiếng Việt
Summary

Go's `runtime.NumCPU()` and `GOMAXPROCS` report the number of CPUs allocated to the current process, not necessarily the total physical CPUs on the system. This value, `ncpu`, is determined during process startup via `runtime/os_linux.go`'s `osinit()` function, which calls `getproccount()`. Internally, `getproccount()` leverages `sched_getaffinity()` (an assembly-implemented native function) to read the process's CPU affinity mask. While typically matching the machine's total CPUs by default, this count reflects the OS-allocated resources. Developers can explicitly limit the number of CPUs a Go process utilizes by configuring cgroups, specifically using `cpuset` to restrict the process's CPU affinity.

When running lscpu command on Linux, it will list the CPU info on the machine. Take one example where there is one CPU with 2 cores and each core has two threads which indicates there are 4 cores available.

Now let's see how many cores GoLang program would identify.

From output, NumCPU and GOMAXPROCS both output 4 which is expected. How does go runtime get this info, does it get it through similar command like lscpu or /proc/cpuinfo? Let's dig more in GoLang's source code.

In runtime/debug.go, it tells that this value is from the global variable ncpu which is set when process starts and will not change once set. 

Where is ncpu set?  Search this keyword with grep in runtime directory, in runtime/os_linux.go. there is a function osinit() which calls getproccount() to get the ncpu value.

Inside getproccount(), it calls sche_getaffinity() which will have an output buf where each bit whose value is 1 will be considered as one core.

This function sche_getaffinity() is a native function defined with go assembly. its implementation can be found in runtime/sys_linux_amd64.s.

From above, the value of ncpu is not the same as the output from lscpu, it is the number of CPUs the OS allocates to current process. In most cases, one would see the ncpu is the same as the number of CPUs available on the OS, that's because OS will allow one process to use all available CPUs by default. If wanna limit number of CPUs to be used by one process, can use cpuset from cgroup to make the change.

For example, let's create a cgroup named gocpu.

cgcreate -g cpuset:/gocpu

If wanna allow only CPU 0-1 to b e used by cgroup gocpu,  can run

#echo '0' > cpuset.mems
#echo '0-1' > cpuset.cpus
#echo $$ > tasks

Then let bash shell run in this new cgroup, and launch the go program again, it will now have only 2 cores to use.

Reference: https://www.toutiao.com/i6872354717305930254/

CPU GOLANG NCPU

  RELATED

  COMMENTS

0

No comment for this article.