主页 / coding / 问题 / 77879635
Accepted
Asked: 2024-01-25 19:34:07 +0800 CST

如何从 AutoModelForSequenceClassification 重置参数？

  • 772

目前，要重新初始化 的模型AutoModelForSequenceClassification，我们可以这样做：

from transformers import AutoModel, AutoConfig, AutoModelForSequenceClassification

m = "moussaKam/frugalscore_tiny_bert-base_bert-score"
config = AutoConfig.from_pretrained(m)
model_from_scratch = AutoModel(config)

model_from_scratch.save_pretrained("frugalscore_tiny_bert-from_scratch")

model = AutoModelForSequenceClassification(
  "frugalscore_tiny_bert-from_scratch", local_files_only=True
)

有没有某种方法可以重新初始化模型权重而不保存用 初始化的新预训练模型AutoConfig？

model = AutoModelForSequenceClassification(
  "moussaKam/frugalscore_tiny_bert-base_bert-score", 
  local_files_only=True
  reinitialize_weights=True
)

或类似的东西：

model = AutoModelForSequenceClassification(
  "moussaKam/frugalscore_tiny_bert-base_bert-score", 
  local_files_only=True
)

model.reinitialize_parameters()

1 个回答

  1. Best Answer

    这就是from_config的目的（即创建模型但不加载相应的权重）：

    from transformers import AutoModel, AutoConfig, AutoModelForSequenceClassification

m = "moussaKam/frugalscore_tiny_bert-base_bert-score"
config = AutoConfig.from_pretrained(m)

no_weights_model = AutoModelForSequenceClassification.from_config(config)
weights_model = AutoModelForSequenceClassification.from_pretrained(m)

import torch

print(torch.allclose(no_weights_model.bert.embeddings.word_embeddings.weight,weights_model.bert.embeddings.word_embeddings.weight))

    输出：

    False
    • 1

相关问题