Skip to content

gpjt/llm-from-scratch

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

92 Commits
.gitignore
.gitignore
 
 
README.md
README.md
 
 
all-layers-accuracy-plot.png
all-layers-accuracy-plot.png
 
 
all-layers-loss-plot.png
all-layers-loss-plot.png
 
 
big-train-model-conf.json
big-train-model-conf.json
 
 
big-train-prepare-datasets.py
big-train-prepare-datasets.py
 
 
big-training-run-chart.html
big-training-run-chart.html
 
 
big_train.py
big_train.py
 
 
chart_surprise_options.py
chart_surprise_options.py
 
 
checkpointing.py
checkpointing.py
 
 
classification-data-wrangling.py
classification-data-wrangling.py
 
 
classification_train.py
classification_train.py
 
 
convert_openai_weights_to_safetensors.py
convert_openai_weights_to_safetensors.py
 
 
dataloader.py
dataloader.py
 
 
download-fineweb-10b.py
download-fineweb-10b.py
 
 
download_and_use_gpt2.py
download_and_use_gpt2.py
 
 
evaluate-per-book.py
evaluate-per-book.py
 
 
evaluate-with-openai.py
evaluate-with-openai.py
 
 
first_generation_test.py
first_generation_test.py
 
 
generate_text.py
generate_text.py
 
 
generate_text_simple.py
generate_text_simple.py
 
 
gpt.py
gpt.py
 
 
graph_fiddling.py
graph_fiddling.py
 
 
ift-from-book.py
ift-from-book.py
 
 
ift-our-base-model.py
ift-our-base-model.py
 
 
initial_test.py
initial_test.py
 
 
last-layers-accuracy-plot.png
last-layers-accuracy-plot.png
 
 
last-layers-loss-plot.png
last-layers-loss-plot.png
 
 
measure-tokens-per-second.py
measure-tokens-per-second.py
 
 
measure-validation-timing.py
measure-validation-timing.py
 
 
model_config.py
model_config.py
 
 
naive_surprise.png
naive_surprise.png
 
 
openai-weights-gpt-medium.json
openai-weights-gpt-medium.json
 
 
openai-weights-gpt-small.json
openai-weights-gpt-small.json
 
 
qwen-transcript-hack.py
qwen-transcript-hack.py
 
 
real_surprise.png
real_surprise.png
 
 
requirements.txt
requirements.txt
 
 
second_generation_test.py
second_generation_test.py
 
 
test-checkpointing.py
test-checkpointing.py
 
 
test_openai_weights_against_our_val_dataset.py
test_openai_weights_against_our_val_dataset.py
 
 
test_our_weights_against_our_dataset.py
test_our_weights_against_our_dataset.py
 
 
test_own_base.py
test_own_base.py
 
 
the-verdict.txt
the-verdict.txt
 
 
train_per_book.py
train_per_book.py
 
 
transcript-hack.py
transcript-hack.py
 
 

Repository files navigation

Scratchpad for working through Sebastian Raschka's "Build a Large Language Model (From Scratch)".

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

12 stars

Watchers

1 watching

Forks

2 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages