Skip to content

init from scratch #243

@karpathy

Description

@karpathy

Follow the GPT-2 reference .py file and initialize the weights in C from scratch in the exact same way.
Allow init from scratch instead of init from checkpoint when building the GPT-2.
Add argparse flag to configure which way to go.
Ok to only change the mainline development file train_gpt2.cu.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions