I have a question about how GPU communicates with each other.
For example, I have one CPU running ps as server, and have two GPUs (in one machine) running as worker.
Does these two GPUs communicates with each other directly, i.e. without going through CPU?
If so, what method do they use to communicate? What function call or what level function calls do they use to communicate? Is that possible for me to identify the function calls? like the send() in the socket communications?
The same questions in the case that these two GPUs are mounted in two different machine?