paddle版本ctrl
-
CTRL: A Conditional Transformer Language Model for Controllable Generation
paddle2.x 实现 CTRL
地址
https://github.com/JunnYu/paddle_ctrl
准备工作：
下载预训练权重（pytorch版本的，https://huggingface.co/ctrl/ 和 https://huggingface.co/sshleifer/tiny-ctrl）,放入hg/文件夹对应目录。
【可选/推荐】手动转换：python convert.py
【不推荐】百度云下载：链接：https://pan.baidu.com/s/1dnRwCRClqsXvG8475v2m9g 提取码：ogn8CTRLLMHeadModel前向对齐
python compare_lm.py ############ sshleifer-tiny-ctrl # compare loss: # mean difference: tensor(0.) # max difference: tensor(0.) # compare logits: # mean difference: tensor(1.2408e-08) # max difference: tensor(5.6252e-07) # compare hidden_states: # mean difference: tensor(4.2710e-08) # max difference: tensor(3.2783e-06) # mean difference: tensor(4.3044e-08) # max difference: tensor(3.2783e-06) # mean difference: tensor(1.2185e-07) # max difference: tensor(4.5300e-06) ############ 加载ctrl权重误差非常大,进入文件夹【探究ctrl原版权重误差大的原因】查看原因 误差大原因同reformer。CTRLForSequenceClassification前向对齐
python compare_cls.py ############ sshleifer-tiny-ctrl # compare loss # mean difference: tensor(2.3842e-07) # max difference: tensor(2.3842e-07) # compare logits # mean difference: tensor(6.0303e-09) # max difference: tensor(1.4901e-08) ############ 加载ctrl权重误差非常大,进入文件夹【探究ctrl原版权重误差大的原因】查看原因 误差大原因同reformer。
