How do I implement a new compression method instead of the default over here
Thanks!
What are you compressing? Gradients or the model itself?
I’m trying to compress the gradients from every worker in a distributed setup.
To do it properly, you’d have to modify the C++ source code. Best would be to follow implementation of two-bit compression (I’d start by searching for kTwoBit in the source code and follow its path). Once that’s there, you can build from source and follow instructions in the link you provided to select your new compression algorithm.
Also if you decide to go down this path, I highly recommend reviewing the steps in this guide to make sure your development environment is setup for easy debugging.