To be more concrete about the question, suppose I have two treads running, thread 1 and thread 2. Thread 1 calls with autograd.record()
which happens first. Then, thread 2 calls with autograd.pause()
while thread one is still executing code in its record block.
Is that safe, or will the code being exectued in the record block now be detached because thread 2 called a pause before it was finished?
Additionally, if it’s not safe, is there a pattern recommended for managing multiple threads with autograd use? Or are there plans to make it thread safe in the future?