Skip to content

only load lm_head.weight and embed_tokens.weight parameters #19

@qwjaskzxl

Description

@qwjaskzxl

I can only load these parameters,
○ lm_head.weight : 131076096
○ model.embed_tokens.weight : 131076096
and the other parameter is None.

Number of parameters:  262152192
model.embed_tokens.weight : 131076096
model.layers.0.self_attn.q_proj.weight : 0
model.layers.0.self_attn.k_proj.weight : 0
model.layers.0.self_attn.v_proj.weight : 0
model.layers.0.self_attn.o_proj.weight : 0
model.layers.0.mlp.gate_proj.weight : 0
model.layers.0.mlp.down_proj.weight : 0
model.layers.0.mlp.up_proj.weight : 0
model.layers.0.input_layernorm.weight : 0
model.layers.0.post_attention_layernorm.weight : 0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions