Hi @MrGGLS,
Thanks for opensourcing your codebase.
I am wondering how I can save the pruned model checkpoint so I can load it in a different codebase. I am looking for something like the below image so it can be easily used by from_pretrained() of the other codebase.
The ppls/xxxx.json has the list of modules that can be deleted I presume. So following that, is it possible to provide a script in the repo where one can load this json and actual model, remove the respective blocks and save the final model.
Follow-up question: As it's structured pruning, I believe the memory footprint and runtime should be better compared to base model. Let me know if my understanding is right?
Looking forward
Hi @MrGGLS,
Thanks for opensourcing your codebase.
I am wondering how I can save the pruned model checkpoint so I can load it in a different codebase. I am looking for something like the below image so it can be easily used by
from_pretrained()of the other codebase.The
ppls/xxxx.jsonhas the list of modules that can be deleted I presume. So following that, is it possible to provide a script in the repo where one can load thisjsonand actual model, remove the respective blocks and save the final model.Follow-up question: As it's structured pruning, I believe the memory footprint and runtime should be better compared to base model. Let me know if my understanding is right?
Looking forward