Overriding config with config/train_shakespeare_char.py:
# train a miniature character-level shakespeare model
# good for debugging and playing on macbooks and such

out_dir = 'out-shakespeare-char'
eval_interval = 250 # keep frequent because we'll overfit
eval_iters = 200
log_interval = 10 # don't print too too often

# we expect to overfit on this small dataset, so only save when val improves
always_save_checkpoint = False

wandb_log = False # override via command line if you like
wandb_project = 'shakespeare-char'
wandb_run_name = 'mini-gpt'

dataset = 'shakespeare_char'
gradient_accumulation_steps = 1
batch_size = 64
block_size = 256 # context of up to 256 previous characters

# baby GPT model :)
n_layer = 6
n_head = 6
n_embd = 384
dropout = 0.2

learning_rate = 1e-3 # with baby networks can afford to go a bit higher
max_iters = 5000
lr_decay_iters = 5000 # make equal to max_iters usually
min_lr = 1e-4 # learning_rate / 10 usually
beta2 = 0.99 # make a bit bigger because number of tokens per iter is small

warmup_iters = 100 # not super necessary potentially

# on macbook also add
# device = 'cpu'  # run on cpu only
# compile = False # do not torch compile the model

Overriding: device = cpu
Overriding: compile = False
Overriding: eval_iters = 20
Overriding: log_interval = 1
Overriding: block_size = 64
Overriding: batch_size = 12
Overriding: n_layer = 4
Overriding: n_head = 4
Overriding: n_embd = 128
Overriding: max_iters = 2000
Overriding: lr_decay_iters = 2000
Overriding: dropout = 0.0
tokens per iteration will be: 768
found vocab_size = 65 (inside data/shakespeare_char/meta.pkl)
Initializing a new model from scratch
number of parameters: 0.80M
num decayed parameter tensors: 18, with 802,944 parameters
num non-decayed parameter tensors: 9, with 1,152 parameters
using fused AdamW: False
step 0: train loss 4.1676, val loss 4.1649
iter 0: loss 4.1828, time 1607.18ms, mfu -100.00%
iter 1: loss 4.1373, time 152.69ms, mfu -100.00%
iter 2: loss 4.1347, time 177.86ms, mfu -100.00%
iter 3: loss 4.0995, time 91.66ms, mfu -100.00%
iter 4: loss 4.0387, time 60.61ms, mfu -100.00%
iter 5: loss 3.9758, time 59.60ms, mfu 0.02%
iter 6: loss 3.9126, time 74.90ms, mfu 0.02%
iter 7: loss 3.8463, time 58.60ms, mfu 0.02%
iter 8: loss 3.8169, time 66.13ms, mfu 0.02%
iter 9: loss 3.7311, time 70.54ms, mfu 0.02%
iter 10: loss 3.7366, time 58.24ms, mfu 0.02%
iter 11: loss 3.7234, time 60.43ms, mfu 0.02%
iter 12: loss 3.7059, time 64.06ms, mfu 0.02%
iter 13: loss 3.6791, time 40.41ms, mfu 0.02%
iter 14: loss 3.6453, time 44.15ms, mfu 0.02%
iter 15: loss 3.6266, time 85.32ms, mfu 0.02%
iter 16: loss 3.6050, time 74.82ms, mfu 0.02%
iter 17: loss 3.6008, time 72.94ms, mfu 0.02%
iter 18: loss 3.6244, time 61.44ms, mfu 0.02%
iter 19: loss 3.6423, time 53.60ms, mfu 0.02%
iter 20: loss 3.5233, time 34.52ms, mfu 0.02%
iter 21: loss 3.5387, time 71.57ms, mfu 0.02%
iter 22: loss 3.4933, time 50.67ms, mfu 0.02%
iter 23: loss 3.4671, time 53.60ms, mfu 0.02%
iter 24: loss 3.4993, time 39.56ms, mfu 0.02%
iter 25: loss 3.3414, time 38.94ms, mfu 0.02%
iter 26: loss 3.3209, time 36.95ms, mfu 0.03%
iter 27: loss 3.3452, time 59.54ms, mfu 0.03%
iter 28: loss 3.2668, time 64.21ms, mfu 0.02%
iter 29: loss 3.2942, time 152.96ms, mfu 0.02%
iter 30: loss 3.3294, time 48.86ms, mfu 0.02%
iter 31: loss 3.2203, time 75.46ms, mfu 0.02%
iter 32: loss 3.1959, time 60.86ms, mfu 0.02%
iter 33: loss 3.2006, time 87.68ms, mfu 0.02%
iter 34: loss 3.1428, time 85.07ms, mfu 0.02%
iter 35: loss 3.2393, time 63.98ms, mfu 0.02%
iter 36: loss 3.1355, time 76.18ms, mfu 0.02%
iter 37: loss 3.1533, time 85.31ms, mfu 0.02%
iter 38: loss 3.0117, time 123.54ms, mfu 0.02%
iter 39: loss 3.0625, time 151.39ms, mfu 0.02%
iter 40: loss 3.0920, time 78.89ms, mfu 0.02%
iter 41: loss 3.0309, time 99.86ms, mfu 0.02%
iter 42: loss 2.9807, time 251.68ms, mfu 0.02%
iter 43: loss 2.9671, time 172.02ms, mfu 0.02%
iter 44: loss 3.0005, time 80.68ms, mfu 0.02%
iter 45: loss 2.9795, time 80.08ms, mfu 0.02%
iter 46: loss 2.9250, time 65.39ms, mfu 0.02%
iter 47: loss 3.0510, time 67.11ms, mfu 0.02%
iter 48: loss 2.9333, time 89.92ms, mfu 0.02%
iter 49: loss 2.9340, time 82.00ms, mfu 0.02%
iter 50: loss 2.9336, time 53.42ms, mfu 0.02%
iter 51: loss 2.9030, time 71.08ms, mfu 0.02%
iter 52: loss 2.9111, time 175.83ms, mfu 0.02%
iter 53: loss 2.8949, time 92.87ms, mfu 0.02%
iter 54: loss 2.8129, time 74.86ms, mfu 0.02%
iter 55: loss 2.8076, time 62.23ms, mfu 0.02%
iter 56: loss 2.7797, time 90.49ms, mfu 0.02%
iter 57: loss 2.8472, time 44.95ms, mfu 0.02%
iter 58: loss 2.8127, time 51.00ms, mfu 0.02%
iter 59: loss 2.8339, time 66.46ms, mfu 0.02%
iter 60: loss 2.7890, time 72.74ms, mfu 0.02%
iter 61: loss 2.8220, time 128.36ms, mfu 0.02%
iter 62: loss 2.7516, time 75.34ms, mfu 0.02%
iter 63: loss 2.7931, time 56.59ms, mfu 0.02%
iter 64: loss 2.7913, time 81.91ms, mfu 0.02%
iter 65: loss 2.7310, time 60.61ms, mfu 0.02%
iter 66: loss 2.7577, time 76.73ms, mfu 0.02%
iter 67: loss 2.7374, time 72.91ms, mfu 0.02%
iter 68: loss 2.7673, time 74.91ms, mfu 0.02%
iter 69: loss 2.7820, time 113.58ms, mfu 0.02%
iter 70: loss 2.7069, time 81.98ms, mfu 0.02%
iter 71: loss 2.7687, time 143.72ms, mfu 0.02%
iter 72: loss 2.6593, time 630.80ms, mfu 0.01%
iter 73: loss 2.7034, time 245.73ms, mfu 0.01%
iter 74: loss 2.6512, time 126.19ms, mfu 0.01%
iter 75: loss 2.7666, time 99.62ms, mfu 0.01%
iter 76: loss 2.6618, time 648.51ms, mfu 0.01%
iter 77: loss 2.7576, time 208.94ms, mfu 0.01%
iter 78: loss 2.6682, time 75.53ms, mfu 0.01%
iter 79: loss 2.6772, time 144.30ms, mfu 0.01%
iter 80: loss 2.6128, time 117.62ms, mfu 0.01%
iter 81: loss 2.6870, time 123.61ms, mfu 0.01%
iter 82: loss 2.6893, time 138.90ms, mfu 0.01%
iter 83: loss 2.6264, time 93.49ms, mfu 0.01%
iter 84: loss 2.7267, time 83.32ms, mfu 0.01%
iter 85: loss 2.6810, time 86.14ms, mfu 0.01%
iter 86: loss 2.5991, time 65.97ms, mfu 0.01%
iter 87: loss 2.6400, time 71.80ms, mfu 0.01%
iter 88: loss 2.6090, time 53.63ms, mfu 0.01%
iter 89: loss 2.7311, time 55.05ms, mfu 0.02%
iter 90: loss 2.6146, time 70.38ms, mfu 0.02%
iter 91: loss 2.6667, time 54.27ms, mfu 0.02%
iter 92: loss 2.6025, time 49.77ms, mfu 0.02%
iter 93: loss 2.5399, time 48.09ms, mfu 0.02%
iter 94: loss 2.7116, time 57.29ms, mfu 0.02%
iter 95: loss 2.6400, time 65.51ms, mfu 0.02%
iter 96: loss 2.6739, time 81.73ms, mfu 0.02%
iter 97: loss 2.5839, time 45.27ms, mfu 0.02%
iter 98: loss 2.5320, time 47.27ms, mfu 0.02%
iter 99: loss 2.6740, time 54.85ms, mfu 0.02%
iter 100: loss 2.6635, time 48.04ms, mfu 0.02%
iter 101: loss 2.6532, time 61.81ms, mfu 0.02%
iter 102: loss 2.5912, time 68.87ms, mfu 0.02%
iter 103: loss 2.6308, time 55.04ms, mfu 0.02%
iter 104: loss 2.5048, time 47.41ms, mfu 0.02%
iter 105: loss 2.5748, time 38.25ms, mfu 0.02%
iter 106: loss 2.5280, time 39.29ms, mfu 0.02%
iter 107: loss 2.5885, time 60.49ms, mfu 0.02%
iter 108: loss 2.5621, time 103.72ms, mfu 0.02%
iter 109: loss 2.5919, time 80.48ms, mfu 0.02%
iter 110: loss 2.5880, time 55.52ms, mfu 0.02%
iter 111: loss 2.5340, time 137.48ms, mfu 0.02%
iter 112: loss 2.5389, time 102.18ms, mfu 0.02%
iter 113: loss 2.5485, time 206.85ms, mfu 0.02%
iter 114: loss 2.6036, time 112.50ms, mfu 0.02%
iter 115: loss 2.5520, time 360.00ms, mfu 0.02%
iter 116: loss 2.5809, time 353.88ms, mfu 0.01%
iter 117: loss 2.5443, time 136.69ms, mfu 0.01%
iter 118: loss 2.5850, time 80.64ms, mfu 0.01%
iter 119: loss 2.6244, time 63.68ms, mfu 0.02%
iter 120: loss 2.6186, time 55.33ms, mfu 0.02%
iter 121: loss 2.7192, time 62.87ms, mfu 0.02%
iter 122: loss 2.6091, time 46.32ms, mfu 0.02%
iter 123: loss 2.5891, time 60.58ms, mfu 0.02%
iter 124: loss 2.6232, time 213.24ms, mfu 0.02%
iter 125: loss 2.5940, time 128.19ms, mfu 0.02%
iter 126: loss 2.6158, time 131.27ms, mfu 0.02%
iter 127: loss 2.6776, time 81.63ms, mfu 0.02%
iter 128: loss 2.5884, time 55.87ms, mfu 0.02%
iter 129: loss 2.5599, time 51.90ms, mfu 0.02%
iter 130: loss 2.5422, time 53.86ms, mfu 0.02%
iter 131: loss 2.6984, time 60.96ms, mfu 0.02%
iter 132: loss 2.5189, time 63.34ms, mfu 0.02%
iter 133: loss 2.5712, time 57.31ms, mfu 0.02%
iter 134: loss 2.4379, time 51.92ms, mfu 0.02%
iter 135: loss 2.5714, time 68.23ms, mfu 0.02%
iter 136: loss 2.5871, time 48.92ms, mfu 0.02%
iter 137: loss 2.5021, time 45.65ms, mfu 0.02%
iter 138: loss 2.5119, time 46.21ms, mfu 0.02%
iter 139: loss 2.5801, time 48.18ms, mfu 0.02%
iter 140: loss 2.4921, time 57.43ms, mfu 0.02%
iter 141: loss 2.5164, time 94.14ms, mfu 0.02%
iter 142: loss 2.5426, time 158.93ms, mfu 0.02%
iter 143: loss 2.5652, time 90.24ms, mfu 0.02%
iter 144: loss 2.5373, time 67.45ms, mfu 0.02%
iter 145: loss 2.5433, time 43.19ms, mfu 0.02%
iter 146: loss 2.5070, time 49.91ms, mfu 0.02%
iter 147: loss 2.5597, time 67.34ms, mfu 0.02%
iter 148: loss 2.4607, time 97.00ms, mfu 0.02%
iter 149: loss 2.5515, time 51.25ms, mfu 0.02%
iter 150: loss 2.5425, time 53.55ms, mfu 0.02%
iter 151: loss 2.5234, time 84.32ms, mfu 0.02%
iter 152: loss 2.5735, time 86.98ms, mfu 0.02%
iter 153: loss 2.4897, time 244.43ms, mfu 0.02%
iter 154: loss 2.4569, time 148.01ms, mfu 0.02%
iter 155: loss 2.4786, time 140.22ms, mfu 0.02%
iter 156: loss 2.5061, time 90.37ms, mfu 0.02%
iter 157: loss 2.5567, time 85.99ms, mfu 0.02%
iter 158: loss 2.5771, time 52.18ms, mfu 0.02%
iter 159: loss 2.5045, time 83.17ms, mfu 0.02%
iter 160: loss 2.5243, time 72.50ms, mfu 0.02%
iter 161: loss 2.4993, time 59.64ms, mfu 0.02%
iter 162: loss 2.4630, time 166.60ms, mfu 0.02%
iter 163: loss 2.4979, time 105.80ms, mfu 0.02%
iter 164: loss 2.5054, time 197.35ms, mfu 0.01%
iter 165: loss 2.6246, time 88.69ms, mfu 0.01%
iter 166: loss 2.5134, time 66.10ms, mfu 0.02%
iter 167: loss 2.4725, time 65.47ms, mfu 0.02%
iter 168: loss 2.5520, time 64.89ms, mfu 0.02%
iter 169: loss 2.4899, time 152.39ms, mfu 0.02%
iter 170: loss 2.4741, time 204.79ms, mfu 0.01%
iter 171: loss 2.3834, time 65.70ms, mfu 0.01%
iter 172: loss 2.5202, time 70.73ms, mfu 0.02%
iter 173: loss 2.5233, time 58.04ms, mfu 0.02%
iter 174: loss 2.4690, time 81.00ms, mfu 0.02%
iter 175: loss 2.4643, time 115.56ms, mfu 0.02%
iter 176: loss 2.4494, time 57.87ms, mfu 0.02%
iter 177: loss 2.4908, time 79.17ms, mfu 0.02%
iter 178: loss 2.4802, time 65.44ms, mfu 0.02%
iter 179: loss 2.6177, time 57.59ms, mfu 0.02%
iter 180: loss 2.4928, time 49.58ms, mfu 0.02%
iter 181: loss 2.4498, time 168.21ms, mfu 0.02%
iter 182: loss 2.4439, time 148.32ms, mfu 0.02%
iter 183: loss 2.5471, time 148.54ms, mfu 0.02%
iter 184: loss 2.4470, time 128.71ms, mfu 0.01%
iter 185: loss 2.5639, time 69.61ms, mfu 0.02%
iter 186: loss 2.5656, time 43.61ms, mfu 0.02%
iter 187: loss 2.5070, time 53.04ms, mfu 0.02%
iter 188: loss 2.4840, time 46.22ms, mfu 0.02%
iter 189: loss 2.5641, time 67.33ms, mfu 0.02%
iter 190: loss 2.5444, time 62.57ms, mfu 0.02%
iter 191: loss 2.5446, time 96.23ms, mfu 0.02%
iter 192: loss 2.4553, time 103.41ms, mfu 0.02%
iter 193: loss 2.5763, time 154.39ms, mfu 0.02%
iter 194: loss 2.4801, time 286.58ms, mfu 0.02%
iter 195: loss 2.4763, time 99.55ms, mfu 0.02%
iter 196: loss 2.4731, time 57.54ms, mfu 0.02%
iter 197: loss 2.4632, time 130.35ms, mfu 0.02%
iter 198: loss 2.4527, time 147.73ms, mfu 0.01%
iter 199: loss 2.4878, time 287.71ms, mfu 0.01%
iter 200: loss 2.5039, time 116.04ms, mfu 0.01%
iter 201: loss 2.4361, time 118.93ms, mfu 0.01%
iter 202: loss 2.4518, time 80.38ms, mfu 0.01%
iter 203: loss 2.5402, time 66.61ms, mfu 0.01%
iter 204: loss 2.4754, time 66.26ms, mfu 0.01%
iter 205: loss 2.4789, time 48.59ms, mfu 0.02%
iter 206: loss 2.4742, time 79.26ms, mfu 0.02%
iter 207: loss 2.5718, time 108.14ms, mfu 0.02%
iter 208: loss 2.3844, time 135.31ms, mfu 0.01%
iter 209: loss 2.5291, time 89.62ms, mfu 0.01%
iter 210: loss 2.4330, time 74.59ms, mfu 0.01%
iter 211: loss 2.5309, time 84.25ms, mfu 0.01%
iter 212: loss 2.4654, time 91.75ms, mfu 0.01%
iter 213: loss 2.4521, time 66.72ms, mfu 0.02%
iter 214: loss 2.5320, time 70.66ms, mfu 0.02%
iter 215: loss 2.5425, time 122.61ms, mfu 0.01%
iter 216: loss 2.5085, time 102.13ms, mfu 0.01%
iter 217: loss 2.4557, time 66.31ms, mfu 0.02%
iter 218: loss 2.5043, time 64.74ms, mfu 0.02%
iter 219: loss 2.4581, time 54.33ms, mfu 0.02%
iter 220: loss 2.4903, time 65.01ms, mfu 0.02%
iter 221: loss 2.5044, time 68.65ms, mfu 0.02%
iter 222: loss 2.4566, time 69.78ms, mfu 0.02%
iter 223: loss 2.5313, time 50.62ms, mfu 0.02%
iter 224: loss 2.3978, time 60.83ms, mfu 0.02%
iter 225: loss 2.5483, time 80.23ms, mfu 0.02%
iter 226: loss 2.4627, time 79.85ms, mfu 0.02%
iter 227: loss 2.4574, time 53.96ms, mfu 0.02%
iter 228: loss 2.4476, time 42.29ms, mfu 0.02%
iter 229: loss 2.5279, time 42.96ms, mfu 0.02%
iter 230: loss 2.4648, time 73.16ms, mfu 0.02%
iter 231: loss 2.4212, time 69.38ms, mfu 0.02%
iter 232: loss 2.4458, time 78.44ms, mfu 0.02%
iter 233: loss 2.4612, time 70.56ms, mfu 0.02%
iter 234: loss 2.4193, time 60.76ms, mfu 0.02%
iter 235: loss 2.4913, time 458.08ms, mfu 0.02%
iter 236: loss 2.4356, time 697.90ms, mfu 0.02%
iter 237: loss 2.4827, time 148.37ms, mfu 0.02%
iter 238: loss 2.5102, time 89.08ms, mfu 0.02%
iter 239: loss 2.4670, time 84.98ms, mfu 0.02%
iter 240: loss 2.5144, time 103.93ms, mfu 0.02%
iter 241: loss 2.4216, time 116.38ms, mfu 0.01%
iter 242: loss 2.4851, time 56.93ms, mfu 0.02%
iter 243: loss 2.4914, time 78.86ms, mfu 0.02%
iter 244: loss 2.4181, time 71.33ms, mfu 0.02%
iter 245: loss 2.5124, time 66.58ms, mfu 0.02%
iter 246: loss 2.4154, time 90.65ms, mfu 0.02%
iter 247: loss 2.4658, time 65.46ms, mfu 0.02%
iter 248: loss 2.4475, time 67.14ms, mfu 0.02%
iter 249: loss 2.4449, time 84.68ms, mfu 0.02%
step 250: train loss 2.4293, val loss 2.4447
saving checkpoint to out-shakespeare-char
iter 250: loss 2.4904, time 1183.66ms, mfu 0.01%
iter 251: loss 2.3638, time 82.92ms, mfu 0.01%
iter 252: loss 2.5004, time 63.30ms, mfu 0.02%
iter 253: loss 2.4087, time 46.07ms, mfu 0.02%
iter 254: loss 2.3968, time 131.35ms, mfu 0.02%
iter 255: loss 2.4558, time 124.22ms, mfu 0.02%
iter 256: loss 2.3588, time 175.96ms, mfu 0.01%
iter 257: loss 2.3921, time 200.10ms, mfu 0.01%
iter 258: loss 2.4123, time 170.19ms, mfu 0.01%
iter 259: loss 2.3825, time 80.48ms, mfu 0.01%
iter 260: loss 2.4461, time 81.80ms, mfu 0.01%
iter 261: loss 2.4801, time 121.50ms, mfu 0.01%
iter 262: loss 2.4433, time 80.95ms, mfu 0.01%
iter 263: loss 2.4063, time 121.58ms, mfu 0.01%
iter 264: loss 2.4662, time 63.27ms, mfu 0.01%
iter 265: loss 2.4957, time 356.44ms, mfu 0.01%
iter 266: loss 2.4313, time 416.69ms, mfu 0.01%
iter 267: loss 2.3545, time 248.50ms, mfu 0.01%
iter 268: loss 2.5298, time 220.32ms, mfu 0.01%
iter 269: loss 2.4722, time 115.82ms, mfu 0.01%
iter 270: loss 2.4044, time 102.61ms, mfu 0.01%
iter 271: loss 2.4732, time 94.49ms, mfu 0.01%
iter 272: loss 2.4242, time 47.30ms, mfu 0.01%
iter 273: loss 2.3979, time 72.45ms, mfu 0.01%
iter 274: loss 2.4059, time 58.56ms, mfu 0.01%
iter 275: loss 2.4296, time 60.84ms, mfu 0.01%
iter 276: loss 2.4052, time 51.24ms, mfu 0.02%
iter 277: loss 2.3876, time 69.67ms, mfu 0.02%
iter 278: loss 2.3873, time 66.20ms, mfu 0.02%
iter 279: loss 2.3665, time 65.60ms, mfu 0.02%
iter 280: loss 2.3788, time 78.01ms, mfu 0.02%
iter 281: loss 2.4736, time 65.78ms, mfu 0.02%
iter 282: loss 2.4060, time 92.61ms, mfu 0.02%
iter 283: loss 2.4246, time 73.39ms, mfu 0.02%
iter 284: loss 2.3952, time 105.80ms, mfu 0.02%
iter 285: loss 2.4339, time 120.00ms, mfu 0.02%
iter 286: loss 2.4566, time 99.92ms, mfu 0.02%
iter 287: loss 2.4284, time 82.97ms, mfu 0.02%
iter 288: loss 2.4340, time 123.42ms, mfu 0.01%
iter 289: loss 2.3849, time 51.54ms, mfu 0.02%
iter 290: loss 2.3959, time 60.82ms, mfu 0.02%
iter 291: loss 2.3171, time 81.55ms, mfu 0.02%
iter 292: loss 2.3974, time 93.22ms, mfu 0.02%
iter 293: loss 2.3744, time 72.51ms, mfu 0.02%
iter 294: loss 2.3162, time 93.59ms, mfu 0.02%
iter 295: loss 2.4403, time 106.54ms, mfu 0.02%
iter 296: loss 2.3260, time 77.63ms, mfu 0.02%
iter 297: loss 2.3889, time 97.93ms, mfu 0.02%
iter 298: loss 2.3428, time 67.45ms, mfu 0.02%
iter 299: loss 2.3893, time 53.93ms, mfu 0.02%
iter 300: loss 2.3350, time 114.19ms, mfu 0.02%
iter 301: loss 2.2811, time 65.85ms, mfu 0.02%
iter 302: loss 2.4488, time 44.34ms, mfu 0.02%
iter 303: loss 2.4061, time 54.61ms, mfu 0.02%
iter 304: loss 2.4852, time 47.91ms, mfu 0.02%
iter 305: loss 2.4113, time 46.79ms, mfu 0.02%
iter 306: loss 2.3722, time 43.89ms, mfu 0.02%
iter 307: loss 2.3499, time 44.41ms, mfu 0.02%
iter 308: loss 2.4330, time 39.87ms, mfu 0.02%
iter 309: loss 2.4252, time 56.42ms, mfu 0.02%
iter 310: loss 2.3489, time 65.89ms, mfu 0.02%
iter 311: loss 2.4107, time 49.30ms, mfu 0.02%
iter 312: loss 2.4517, time 59.77ms, mfu 0.02%
iter 313: loss 2.4372, time 73.27ms, mfu 0.02%
iter 314: loss 2.4837, time 38.55ms, mfu 0.02%
iter 315: loss 2.4368, time 54.52ms, mfu 0.02%
iter 316: loss 2.4451, time 37.06ms, mfu 0.02%
iter 317: loss 2.2934, time 48.88ms, mfu 0.02%
iter 318: loss 2.3787, time 32.03ms, mfu 0.03%
iter 319: loss 2.4132, time 37.10ms, mfu 0.03%
iter 320: loss 2.4062, time 47.18ms, mfu 0.03%
iter 321: loss 2.3898, time 64.92ms, mfu 0.03%
iter 322: loss 2.3933, time 48.73ms, mfu 0.03%
iter 323: loss 2.4027, time 75.64ms, mfu 0.03%
iter 324: loss 2.3942, time 147.82ms, mfu 0.02%
iter 325: loss 2.3657, time 41.88ms, mfu 0.02%
iter 326: loss 2.4684, time 65.37ms, mfu 0.02%
iter 327: loss 2.3654, time 61.97ms, mfu 0.02%
iter 328: loss 2.3823, time 55.93ms, mfu 0.02%
iter 329: loss 2.3280, time 62.51ms, mfu 0.02%
iter 330: loss 2.3390, time 100.12ms, mfu 0.02%
iter 331: loss 2.4196, time 48.10ms, mfu 0.02%
iter 332: loss 2.3537, time 62.98ms, mfu 0.02%
iter 333: loss 2.4219, time 67.94ms, mfu 0.02%
iter 334: loss 2.3949, time 72.55ms, mfu 0.02%
iter 335: loss 2.3995, time 347.58ms, mfu 0.02%
iter 336: loss 2.4226, time 277.09ms, mfu 0.02%
iter 337: loss 2.3236, time 288.81ms, mfu 0.02%
iter 338: loss 2.3643, time 221.45ms, mfu 0.02%
iter 339: loss 2.4672, time 156.49ms, mfu 0.01%
iter 340: loss 2.3313, time 198.51ms, mfu 0.01%
iter 341: loss 2.3150, time 184.46ms, mfu 0.01%
iter 342: loss 2.3031, time 202.08ms, mfu 0.01%
iter 343: loss 2.3194, time 105.21ms, mfu 0.01%
iter 344: loss 2.4064, time 61.90ms, mfu 0.01%
iter 345: loss 2.3602, time 69.20ms, mfu 0.01%
iter 346: loss 2.3595, time 58.11ms, mfu 0.01%
iter 347: loss 2.3857, time 97.87ms, mfu 0.01%
iter 348: loss 2.3178, time 121.12ms, mfu 0.01%
iter 349: loss 2.3682, time 105.66ms, mfu 0.01%
iter 350: loss 2.3375, time 130.47ms, mfu 0.01%
iter 351: loss 2.2778, time 89.67ms, mfu 0.01%
iter 352: loss 2.3887, time 73.62ms, mfu 0.01%
iter 353: loss 2.4588, time 70.30ms, mfu 0.01%
iter 354: loss 2.3892, time 75.94ms, mfu 0.01%
iter 355: loss 2.4674, time 107.00ms, mfu 0.01%
iter 356: loss 2.2908, time 88.74ms, mfu 0.01%
iter 357: loss 2.3011, time 83.54ms, mfu 0.01%
iter 358: loss 2.3417, time 139.40ms, mfu 0.01%
iter 359: loss 2.3650, time 74.01ms, mfu 0.01%
iter 360: loss 2.4137, time 64.36ms, mfu 0.01%
iter 361: loss 2.2958, time 105.97ms, mfu 0.01%
iter 362: loss 2.3831, time 99.85ms, mfu 0.01%
iter 363: loss 2.3644, time 149.44ms, mfu 0.01%
iter 364: loss 2.3618, time 85.53ms, mfu 0.01%
iter 365: loss 2.3879, time 56.65ms, mfu 0.01%
iter 366: loss 2.4220, time 90.01ms, mfu 0.01%
iter 367: loss 2.3350, time 69.50ms, mfu 0.02%
iter 368: loss 2.3737, time 51.05ms, mfu 0.02%
iter 369: loss 2.3915, time 612.51ms, mfu 0.01%
iter 370: loss 2.4959, time 203.62ms, mfu 0.01%
iter 371: loss 2.3696, time 111.35ms, mfu 0.01%
iter 372: loss 2.3158, time 71.00ms, mfu 0.01%
iter 373: loss 2.3926, time 104.81ms, mfu 0.01%
iter 374: loss 2.3979, time 427.93ms, mfu 0.01%
iter 375: loss 2.2866, time 326.87ms, mfu 0.01%
iter 376: loss 2.3689, time 365.43ms, mfu 0.01%
iter 377: loss 2.3612, time 363.61ms, mfu 0.01%
iter 378: loss 2.3669, time 179.48ms, mfu 0.01%
iter 379: loss 2.3135, time 264.03ms, mfu 0.01%
iter 380: loss 2.3445, time 441.25ms, mfu 0.01%
iter 381: loss 2.2895, time 237.42ms, mfu 0.01%
iter 382: loss 2.4056, time 136.30ms, mfu 0.01%
iter 383: loss 2.3747, time 91.87ms, mfu 0.01%
iter 384: loss 2.2996, time 81.16ms, mfu 0.01%
iter 385: loss 2.3016, time 156.81ms, mfu 0.01%
iter 386: loss 2.3440, time 76.30ms, mfu 0.01%
iter 387: loss 2.3218, time 119.56ms, mfu 0.01%
iter 388: loss 2.2424, time 94.51ms, mfu 0.01%
iter 389: loss 2.3601, time 79.43ms, mfu 0.01%
iter 390: loss 2.3626, time 127.19ms, mfu 0.01%
iter 391: loss 2.4102, time 78.61ms, mfu 0.01%
iter 392: loss 2.2965, time 63.77ms, mfu 0.01%
iter 393: loss 2.3174, time 59.47ms, mfu 0.01%
iter 394: loss 2.4587, time 75.57ms, mfu 0.01%
iter 395: loss 2.3097, time 69.29ms, mfu 0.01%
iter 396: loss 2.3778, time 202.12ms, mfu 0.01%
iter 397: loss 2.3060, time 188.90ms, mfu 0.01%
iter 398: loss 2.2219, time 64.01ms, mfu 0.01%
iter 399: loss 2.3750, time 91.19ms, mfu 0.01%
iter 400: loss 2.3405, time 82.17ms, mfu 0.01%
iter 401: loss 2.3534, time 69.69ms, mfu 0.01%
iter 402: loss 2.3740, time 84.76ms, mfu 0.01%
iter 403: loss 2.3630, time 75.95ms, mfu 0.01%
iter 404: loss 2.3433, time 67.48ms, mfu 0.01%
iter 405: loss 2.3948, time 67.47ms, mfu 0.02%
iter 406: loss 2.3297, time 87.34ms, mfu 0.02%
iter 407: loss 2.3814, time 61.85ms, mfu 0.02%
iter 408: loss 2.2795, time 58.28ms, mfu 0.02%
iter 409: loss 2.3073, time 73.54ms, mfu 0.02%
iter 410: loss 2.2802, time 57.74ms, mfu 0.02%
iter 411: loss 2.3615, time 84.24ms, mfu 0.02%
iter 412: loss 2.3462, time 78.98ms, mfu 0.02%
iter 413: loss 2.3275, time 49.50ms, mfu 0.02%
iter 414: loss 2.3518, time 65.62ms, mfu 0.02%
iter 415: loss 2.4136, time 54.28ms, mfu 0.02%
iter 416: loss 2.3288, time 53.79ms, mfu 0.02%
iter 417: loss 2.3052, time 59.61ms, mfu 0.02%
iter 418: loss 2.2778, time 62.79ms, mfu 0.02%
iter 419: loss 2.2965, time 68.05ms, mfu 0.02%
iter 420: loss 2.3759, time 101.35ms, mfu 0.02%
iter 421: loss 2.2363, time 311.01ms, mfu 0.02%
iter 422: loss 2.2847, time 385.57ms, mfu 0.02%
iter 423: loss 2.3250, time 209.87ms, mfu 0.01%
iter 424: loss 2.2748, time 226.21ms, mfu 0.01%
iter 425: loss 2.3195, time 249.54ms, mfu 0.01%
iter 426: loss 2.4147, time 223.01ms, mfu 0.01%
iter 427: loss 2.3659, time 153.85ms, mfu 0.01%
iter 428: loss 2.3075, time 127.65ms, mfu 0.01%
iter 429: loss 2.2580, time 158.47ms, mfu 0.01%
iter 430: loss 2.4103, time 198.49ms, mfu 0.01%
iter 431: loss 2.3262, time 89.76ms, mfu 0.01%
iter 432: loss 2.3371, time 132.09ms, mfu 0.01%
iter 433: loss 2.2726, time 120.04ms, mfu 0.01%
iter 434: loss 2.2847, time 126.55ms, mfu 0.01%
iter 435: loss 2.2916, time 75.85ms, mfu 0.01%
iter 436: loss 2.3977, time 103.32ms, mfu 0.01%
iter 437: loss 2.2551, time 91.08ms, mfu 0.01%
iter 438: loss 2.2854, time 235.50ms, mfu 0.01%
iter 439: loss 2.2792, time 96.52ms, mfu 0.01%
iter 440: loss 2.3480, time 74.89ms, mfu 0.01%
iter 441: loss 2.3338, time 87.54ms, mfu 0.01%
iter 442: loss 2.3563, time 132.21ms, mfu 0.01%
iter 443: loss 2.3662, time 124.28ms, mfu 0.01%
iter 444: loss 2.3575, time 124.73ms, mfu 0.01%
iter 445: loss 2.2895, time 65.37ms, mfu 0.01%
iter 446: loss 2.2954, time 47.86ms, mfu 0.01%
iter 447: loss 2.3782, time 73.35ms, mfu 0.01%
iter 448: loss 2.3840, time 69.79ms, mfu 0.01%
iter 449: loss 2.3348, time 79.87ms, mfu 0.01%
iter 450: loss 2.3362, time 277.46ms, mfu 0.01%
iter 451: loss 2.2999, time 323.73ms, mfu 0.01%
iter 452: loss 2.3507, time 288.21ms, mfu 0.01%
iter 453: loss 2.3094, time 898.21ms, mfu 0.01%
iter 454: loss 2.3534, time 146.21ms, mfu 0.01%
iter 455: loss 2.3407, time 100.58ms, mfu 0.01%
iter 456: loss 2.3085, time 76.01ms, mfu 0.01%
iter 457: loss 2.2953, time 131.70ms, mfu 0.01%
iter 458: loss 2.2863, time 110.95ms, mfu 0.01%
iter 459: loss 2.3017, time 241.36ms, mfu 0.01%
iter 460: loss 2.3215, time 119.62ms, mfu 0.01%
iter 461: loss 2.3446, time 125.67ms, mfu 0.01%
iter 462: loss 2.3883, time 149.48ms, mfu 0.01%
iter 463: loss 2.3071, time 96.92ms, mfu 0.01%
iter 464: loss 2.2828, time 138.53ms, mfu 0.01%
iter 465: loss 2.3161, time 145.84ms, mfu 0.01%
iter 466: loss 2.2996, time 87.71ms, mfu 0.01%
iter 467: loss 2.2252, time 96.11ms, mfu 0.01%
iter 468: loss 2.2891, time 74.11ms, mfu 0.01%
iter 469: loss 2.2152, time 68.74ms, mfu 0.01%
iter 470: loss 2.2820, time 75.81ms, mfu 0.01%
iter 471: loss 2.2975, time 193.01ms, mfu 0.01%
iter 472: loss 2.2505, time 174.87ms, mfu 0.01%
iter 473: loss 2.2848, time 104.92ms, mfu 0.01%
iter 474: loss 2.2092, time 80.53ms, mfu 0.01%
iter 475: loss 2.3322, time 73.24ms, mfu 0.01%
iter 476: loss 2.2793, time 42.24ms, mfu 0.01%
iter 477: loss 2.3126, time 43.46ms, mfu 0.02%
iter 478: loss 2.3488, time 88.13ms, mfu 0.02%
iter 479: loss 2.3008, time 77.42ms, mfu 0.02%
iter 480: loss 2.2210, time 85.80ms, mfu 0.02%
iter 481: loss 2.1983, time 68.60ms, mfu 0.02%
iter 482: loss 2.3162, time 46.36ms, mfu 0.02%
iter 483: loss 2.2768, time 50.18ms, mfu 0.02%
iter 484: loss 2.2773, time 59.66ms, mfu 0.02%
iter 485: loss 2.2800, time 62.66ms, mfu 0.02%
iter 486: loss 2.2805, time 69.26ms, mfu 0.02%
iter 487: loss 2.3032, time 52.82ms, mfu 0.02%
iter 488: loss 2.3084, time 41.35ms, mfu 0.02%
iter 489: loss 2.2776, time 89.29ms, mfu 0.02%
iter 490: loss 2.3192, time 46.07ms, mfu 0.02%
iter 491: loss 2.2949, time 42.66ms, mfu 0.02%
iter 492: loss 2.3934, time 33.05ms, mfu 0.02%
iter 493: loss 2.1888, time 45.04ms, mfu 0.02%
iter 494: loss 2.3493, time 44.50ms, mfu 0.02%
iter 495: loss 2.2627, time 40.36ms, mfu 0.02%
iter 496: loss 2.2630, time 56.53ms, mfu 0.02%
iter 497: loss 2.2483, time 50.92ms, mfu 0.02%
iter 498: loss 2.3021, time 39.97ms, mfu 0.03%
iter 499: loss 2.2272, time 65.30ms, mfu 0.02%
step 500: train loss 2.2732, val loss 2.3141
saving checkpoint to out-shakespeare-char
iter 500: loss 2.3827, time 3721.82ms, mfu 0.02%
iter 501: loss 2.2468, time 400.56ms, mfu 0.02%
iter 502: loss 2.2749, time 405.76ms, mfu 0.02%
iter 503: loss 2.2869, time 145.18ms, mfu 0.02%
iter 504: loss 2.2698, time 101.29ms, mfu 0.02%
iter 505: loss 2.4110, time 64.16ms, mfu 0.02%
iter 506: loss 2.2976, time 84.43ms, mfu 0.02%
iter 507: loss 2.2620, time 70.85ms, mfu 0.02%
iter 508: loss 2.3844, time 71.43ms, mfu 0.02%
iter 509: loss 2.2737, time 68.62ms, mfu 0.02%
iter 510: loss 2.2102, time 83.05ms, mfu 0.02%
iter 511: loss 2.2610, time 90.32ms, mfu 0.02%
iter 512: loss 2.3027, time 91.16ms, mfu 0.02%
iter 513: loss 2.2255, time 198.85ms, mfu 0.02%
iter 514: loss 2.2101, time 46.59ms, mfu 0.02%
iter 515: loss 2.2562, time 52.59ms, mfu 0.02%
iter 516: loss 2.1978, time 43.56ms, mfu 0.02%
iter 517: loss 2.3188, time 63.07ms, mfu 0.02%
iter 518: loss 2.3139, time 81.34ms, mfu 0.02%
iter 519: loss 2.3065, time 46.26ms, mfu 0.02%
iter 520: loss 2.3269, time 40.25ms, mfu 0.02%
iter 521: loss 2.3028, time 49.04ms, mfu 0.02%
iter 522: loss 2.2949, time 126.59ms, mfu 0.02%
iter 523: loss 2.2880, time 180.14ms, mfu 0.02%
iter 524: loss 2.2296, time 211.00ms, mfu 0.02%
iter 525: loss 2.2265, time 279.44ms, mfu 0.02%
iter 526: loss 2.2174, time 141.36ms, mfu 0.02%
iter 527: loss 2.3135, time 88.05ms, mfu 0.02%
iter 528: loss 2.2242, time 238.58ms, mfu 0.01%
iter 529: loss 2.2512, time 401.67ms, mfu 0.01%
iter 530: loss 2.2947, time 191.42ms, mfu 0.01%
iter 531: loss 2.2396, time 198.63ms, mfu 0.01%
iter 532: loss 2.3120, time 607.19ms, mfu 0.01%
iter 533: loss 2.2841, time 220.52ms, mfu 0.01%
iter 534: loss 2.2435, time 127.04ms, mfu 0.01%
iter 535: loss 2.2502, time 70.90ms, mfu 0.01%
iter 536: loss 2.3203, time 70.80ms, mfu 0.01%
iter 537: loss 2.2272, time 125.99ms, mfu 0.01%
iter 538: loss 2.1885, time 130.57ms, mfu 0.01%
iter 539: loss 2.1655, time 93.67ms, mfu 0.01%
iter 540: loss 2.2645, time 58.15ms, mfu 0.01%
iter 541: loss 2.2189, time 91.34ms, mfu 0.01%
iter 542: loss 2.2678, time 88.88ms, mfu 0.01%
iter 543: loss 2.4016, time 87.26ms, mfu 0.01%
iter 544: loss 2.2778, time 72.50ms, mfu 0.01%
iter 545: loss 2.2472, time 72.90ms, mfu 0.01%
iter 546: loss 2.3266, time 44.71ms, mfu 0.02%
iter 547: loss 2.2237, time 51.75ms, mfu 0.02%
iter 548: loss 2.1880, time 74.30ms, mfu 0.02%
iter 549: loss 2.2061, time 216.58ms, mfu 0.02%
iter 550: loss 2.2085, time 221.45ms, mfu 0.01%
iter 551: loss 2.2311, time 96.10ms, mfu 0.01%
iter 552: loss 2.2420, time 74.95ms, mfu 0.01%
iter 553: loss 2.3390, time 96.00ms, mfu 0.01%
iter 554: loss 2.2059, time 100.53ms, mfu 0.01%
iter 555: loss 2.2399, time 63.51ms, mfu 0.01%
iter 556: loss 2.2269, time 49.00ms, mfu 0.02%
iter 557: loss 2.2081, time 52.67ms, mfu 0.02%
iter 558: loss 2.2146, time 72.90ms, mfu 0.02%
iter 559: loss 2.2311, time 103.36ms, mfu 0.02%
iter 560: loss 2.2281, time 56.60ms, mfu 0.02%
iter 561: loss 2.2602, time 46.42ms, mfu 0.02%
iter 562: loss 2.1343, time 117.31ms, mfu 0.02%
iter 563: loss 2.2102, time 216.40ms, mfu 0.02%
iter 564: loss 2.1902, time 698.89ms, mfu 0.01%
iter 565: loss 2.1815, time 428.78ms, mfu 0.01%
iter 566: loss 2.1878, time 93.02ms, mfu 0.01%
iter 567: loss 2.2576, time 83.34ms, mfu 0.01%
iter 568: loss 2.2645, time 606.50ms, mfu 0.01%
iter 569: loss 2.2535, time 272.86ms, mfu 0.01%
iter 570: loss 2.1146, time 64.47ms, mfu 0.01%
iter 571: loss 2.1915, time 65.43ms, mfu 0.01%
iter 572: loss 2.2554, time 65.62ms, mfu 0.01%
iter 573: loss 2.2534, time 108.77ms, mfu 0.01%
iter 574: loss 2.3195, time 102.94ms, mfu 0.01%
iter 575: loss 2.2623, time 302.78ms, mfu 0.01%
iter 576: loss 2.2309, time 173.79ms, mfu 0.01%
iter 577: loss 2.2074, time 97.25ms, mfu 0.01%
iter 578: loss 2.2819, time 66.80ms, mfu 0.01%
iter 579: loss 2.2002, time 104.65ms, mfu 0.01%
iter 580: loss 2.2228, time 72.89ms, mfu 0.01%
iter 581: loss 2.3294, time 118.71ms, mfu 0.01%
iter 582: loss 2.2033, time 134.02ms, mfu 0.01%
iter 583: loss 2.2352, time 95.20ms, mfu 0.01%
iter 584: loss 2.2094, time 77.88ms, mfu 0.01%
iter 585: loss 2.2305, time 73.30ms, mfu 0.01%
iter 586: loss 2.2003, time 67.99ms, mfu 0.01%
iter 587: loss 2.2429, time 70.15ms, mfu 0.01%
iter 588: loss 2.2523, time 68.56ms, mfu 0.01%
iter 589: loss 2.1454, time 136.58ms, mfu 0.01%
iter 590: loss 2.2038, time 133.19ms, mfu 0.01%
iter 591: loss 2.1222, time 501.85ms, mfu 0.01%
iter 592: loss 2.1893, time 360.72ms, mfu 0.01%
iter 593: loss 2.2733, time 324.21ms, mfu 0.01%
iter 594: loss 2.1528, time 352.55ms, mfu 0.01%
iter 595: loss 2.1854, time 211.55ms, mfu 0.01%
iter 596: loss 2.2587, time 100.78ms, mfu 0.01%
iter 597: loss 2.2592, time 55.01ms, mfu 0.01%
iter 598: loss 2.2057, time 193.74ms, mfu 0.01%
iter 599: loss 2.1786, time 103.33ms, mfu 0.01%
iter 600: loss 2.1762, time 101.31ms, mfu 0.01%
iter 601: loss 2.2357, time 79.45ms, mfu 0.01%
iter 602: loss 2.2017, time 77.03ms, mfu 0.01%
iter 603: loss 2.1887, time 69.99ms, mfu 0.01%
iter 604: loss 2.1962, time 70.61ms, mfu 0.01%
iter 605: loss 2.2168, time 61.52ms, mfu 0.01%
iter 606: loss 2.2452, time 59.30ms, mfu 0.01%
iter 607: loss 2.2383, time 59.76ms, mfu 0.02%
iter 608: loss 2.2055, time 42.28ms, mfu 0.02%
iter 609: loss 2.1904, time 56.08ms, mfu 0.02%
iter 610: loss 2.2001, time 51.83ms, mfu 0.02%
iter 611: loss 2.1229, time 48.29ms, mfu 0.02%
iter 612: loss 2.1816, time 67.11ms, mfu 0.02%
iter 613: loss 2.1656, time 84.58ms, mfu 0.02%
iter 614: loss 2.1957, time 106.40ms, mfu 0.02%
iter 615: loss 2.2617, time 79.98ms, mfu 0.02%
iter 616: loss 2.1616, time 76.78ms, mfu 0.02%
iter 617: loss 2.2161, time 105.91ms, mfu 0.02%
iter 618: loss 2.2205, time 70.16ms, mfu 0.02%
iter 619: loss 2.2175, time 89.90ms, mfu 0.02%
iter 620: loss 2.1835, time 67.91ms, mfu 0.02%
iter 621: loss 2.1941, time 67.18ms, mfu 0.02%
iter 622: loss 2.2253, time 62.39ms, mfu 0.02%
iter 623: loss 2.2061, time 54.32ms, mfu 0.02%
iter 624: loss 2.1858, time 92.79ms, mfu 0.02%
iter 625: loss 2.2015, time 121.39ms, mfu 0.02%
iter 626: loss 2.1669, time 121.19ms, mfu 0.02%
iter 627: loss 2.1463, time 276.89ms, mfu 0.02%
iter 628: loss 2.1824, time 200.31ms, mfu 0.01%
iter 629: loss 2.2452, time 214.58ms, mfu 0.01%
iter 630: loss 2.1622, time 175.29ms, mfu 0.01%
iter 631: loss 2.1083, time 159.60ms, mfu 0.01%
iter 632: loss 2.2201, time 128.06ms, mfu 0.01%
iter 633: loss 2.1685, time 76.36ms, mfu 0.01%
iter 634: loss 2.2809, time 144.69ms, mfu 0.01%
iter 635: loss 2.2298, time 182.49ms, mfu 0.01%
iter 636: loss 2.2056, time 87.63ms, mfu 0.01%
iter 637: loss 2.1337, time 71.56ms, mfu 0.01%
iter 638: loss 2.1657, time 96.51ms, mfu 0.01%
iter 639: loss 2.2416, time 166.54ms, mfu 0.01%
iter 640: loss 2.1233, time 120.30ms, mfu 0.01%
iter 641: loss 2.2060, time 80.21ms, mfu 0.01%
iter 642: loss 2.2535, time 120.39ms, mfu 0.01%
iter 643: loss 2.1925, time 100.93ms, mfu 0.01%
iter 644: loss 2.2023, time 72.72ms, mfu 0.01%
iter 645: loss 2.1416, time 72.95ms, mfu 0.01%
iter 646: loss 2.2024, time 66.69ms, mfu 0.01%
iter 647: loss 2.1978, time 55.77ms, mfu 0.01%
iter 648: loss 2.1226, time 141.61ms, mfu 0.01%
iter 649: loss 2.1907, time 186.73ms, mfu 0.01%
iter 650: loss 2.1471, time 220.93ms, mfu 0.01%
iter 651: loss 2.1497, time 255.28ms, mfu 0.01%
iter 652: loss 2.2072, time 521.97ms, mfu 0.01%
iter 653: loss 2.2927, time 568.02ms, mfu 0.01%
iter 654: loss 2.1618, time 173.60ms, mfu 0.01%
iter 655: loss 2.1870, time 109.42ms, mfu 0.01%
iter 656: loss 2.2226, time 121.75ms, mfu 0.01%
iter 657: loss 2.2553, time 99.14ms, mfu 0.01%
iter 658: loss 2.3203, time 118.21ms, mfu 0.01%
iter 659: loss 2.0763, time 55.19ms, mfu 0.01%
iter 660: loss 2.2185, time 80.69ms, mfu 0.01%
iter 661: loss 2.1899, time 67.86ms, mfu 0.01%
iter 662: loss 2.1839, time 74.39ms, mfu 0.01%
iter 663: loss 2.1789, time 44.76ms, mfu 0.01%
iter 664: loss 2.1137, time 48.83ms, mfu 0.02%
iter 665: loss 2.1452, time 48.47ms, mfu 0.02%
iter 666: loss 2.1240, time 42.63ms, mfu 0.02%
iter 667: loss 2.2286, time 49.27ms, mfu 0.02%
iter 668: loss 2.2215, time 44.40ms, mfu 0.02%
iter 669: loss 2.1907, time 41.98ms, mfu 0.02%
iter 670: loss 2.1319, time 73.51ms, mfu 0.02%
iter 671: loss 2.1905, time 63.25ms, mfu 0.02%
iter 672: loss 2.2237, time 54.82ms, mfu 0.02%
iter 673: loss 2.1459, time 36.16ms, mfu 0.02%
iter 674: loss 2.1634, time 67.01ms, mfu 0.02%
iter 675: loss 2.2517, time 72.07ms, mfu 0.02%
iter 676: loss 2.2451, time 70.65ms, mfu 0.02%
iter 677: loss 2.2100, time 61.05ms, mfu 0.02%
iter 678: loss 2.1692, time 42.60ms, mfu 0.02%
iter 679: loss 2.1280, time 45.31ms, mfu 0.02%
iter 680: loss 2.1610, time 67.99ms, mfu 0.02%
iter 681: loss 2.2499, time 38.89ms, mfu 0.02%
iter 682: loss 2.2104, time 41.71ms, mfu 0.02%
iter 683: loss 2.1459, time 69.36ms, mfu 0.02%
iter 684: loss 2.1459, time 52.57ms, mfu 0.02%
iter 685: loss 2.1823, time 36.10ms, mfu 0.02%
iter 686: loss 2.1284, time 55.09ms, mfu 0.02%
iter 687: loss 2.0909, time 31.47ms, mfu 0.03%
iter 688: loss 2.2150, time 40.25ms, mfu 0.03%
iter 689: loss 2.1008, time 54.87ms, mfu 0.03%
iter 690: loss 2.0918, time 48.81ms, mfu 0.03%
iter 691: loss 2.0587, time 47.48ms, mfu 0.03%
iter 692: loss 2.1489, time 93.06ms, mfu 0.03%
iter 693: loss 2.1889, time 130.42ms, mfu 0.02%
iter 694: loss 2.2110, time 96.40ms, mfu 0.02%
iter 695: loss 2.0026, time 162.05ms, mfu 0.02%
iter 696: loss 2.1394, time 163.80ms, mfu 0.02%
iter 697: loss 2.2696, time 266.38ms, mfu 0.02%
iter 698: loss 2.1552, time 417.79ms, mfu 0.02%
iter 699: loss 2.1777, time 245.30ms, mfu 0.02%
iter 700: loss 2.0947, time 113.97ms, mfu 0.02%
iter 701: loss 2.0826, time 67.85ms, mfu 0.02%
iter 702: loss 2.2262, time 85.21ms, mfu 0.02%
iter 703: loss 2.1844, time 70.69ms, mfu 0.02%
iter 704: loss 2.0627, time 62.37ms, mfu 0.02%
iter 705: loss 2.1286, time 74.47ms, mfu 0.02%
iter 706: loss 2.2228, time 91.16ms, mfu 0.02%
iter 707: loss 2.0659, time 83.72ms, mfu 0.02%
iter 708: loss 2.0816, time 119.66ms, mfu 0.02%
iter 709: loss 2.1770, time 109.48ms, mfu 0.02%
iter 710: loss 2.1849, time 89.27ms, mfu 0.01%
iter 711: loss 2.1218, time 72.74ms, mfu 0.02%
iter 712: loss 2.2316, time 62.87ms, mfu 0.02%
iter 713: loss 2.0327, time 61.99ms, mfu 0.02%
iter 714: loss 2.0841, time 60.70ms, mfu 0.02%
iter 715: loss 2.0609, time 65.22ms, mfu 0.02%
iter 716: loss 2.1239, time 83.20ms, mfu 0.02%
iter 717: loss 2.1714, time 67.49ms, mfu 0.02%
iter 718: loss 2.1281, time 61.81ms, mfu 0.02%
iter 719: loss 2.1241, time 79.47ms, mfu 0.02%
iter 720: loss 2.1115, time 94.00ms, mfu 0.02%
iter 721: loss 2.2540, time 127.85ms, mfu 0.02%
iter 722: loss 2.0898, time 170.08ms, mfu 0.02%
iter 723: loss 2.0718, time 66.99ms, mfu 0.02%
iter 724: loss 2.1135, time 58.42ms, mfu 0.02%
iter 725: loss 2.2130, time 50.86ms, mfu 0.02%
iter 726: loss 2.1488, time 62.25ms, mfu 0.02%
iter 727: loss 2.1094, time 50.53ms, mfu 0.02%
iter 728: loss 2.1599, time 80.45ms, mfu 0.02%
iter 729: loss 2.1785, time 152.88ms, mfu 0.02%
iter 730: loss 2.1955, time 120.91ms, mfu 0.02%
iter 731: loss 2.1662, time 326.57ms, mfu 0.02%
iter 732: loss 2.1308, time 176.55ms, mfu 0.01%
iter 733: loss 2.1017, time 152.23ms, mfu 0.01%
iter 734: loss 2.0702, time 220.61ms, mfu 0.01%
iter 735: loss 2.1047, time 92.08ms, mfu 0.01%
iter 736: loss 2.1177, time 94.90ms, mfu 0.01%
iter 737: loss 2.2741, time 141.75ms, mfu 0.01%
iter 738: loss 2.0233, time 210.08ms, mfu 0.01%
iter 739: loss 2.1258, time 234.06ms, mfu 0.01%
iter 740: loss 2.1313, time 246.17ms, mfu 0.01%
iter 741: loss 2.0251, time 274.64ms, mfu 0.01%
iter 742: loss 2.1406, time 158.62ms, mfu 0.01%
iter 743: loss 2.1731, time 135.44ms, mfu 0.01%
iter 744: loss 2.1083, time 158.90ms, mfu 0.01%
iter 745: loss 2.0706, time 169.75ms, mfu 0.01%
iter 746: loss 2.0784, time 152.95ms, mfu 0.01%
iter 747: loss 2.0538, time 190.46ms, mfu 0.01%
iter 748: loss 2.0627, time 261.73ms, mfu 0.01%
iter 749: loss 2.0851, time 101.40ms, mfu 0.01%
step 750: train loss 2.1338, val loss 2.1905
saving checkpoint to out-shakespeare-char
iter 750: loss 2.0768, time 1883.43ms, mfu 0.01%
iter 751: loss 2.1407, time 73.48ms, mfu 0.01%
iter 752: loss 2.1244, time 86.67ms, mfu 0.01%
iter 753: loss 2.0533, time 78.63ms, mfu 0.01%
iter 754: loss 2.1386, time 88.72ms, mfu 0.01%
iter 755: loss 2.0479, time 113.52ms, mfu 0.01%
iter 756: loss 2.0506, time 81.69ms, mfu 0.01%
iter 757: loss 2.1101, time 50.86ms, mfu 0.01%
iter 758: loss 2.1412, time 132.09ms, mfu 0.01%
iter 759: loss 2.1293, time 151.60ms, mfu 0.01%
iter 760: loss 2.1775, time 64.64ms, mfu 0.01%
iter 761: loss 2.1686, time 54.92ms, mfu 0.01%
iter 762: loss 2.1632, time 47.80ms, mfu 0.02%
iter 763: loss 2.1387, time 42.83ms, mfu 0.02%
iter 764: loss 2.1546, time 38.81ms, mfu 0.02%
iter 765: loss 2.0865, time 464.27ms, mfu 0.02%
iter 766: loss 2.1429, time 353.84ms, mfu 0.02%
iter 767: loss 2.1232, time 85.78ms, mfu 0.02%
iter 768: loss 2.1197, time 60.18ms, mfu 0.02%
iter 769: loss 2.0845, time 56.52ms, mfu 0.02%
iter 770: loss 2.1071, time 44.75ms, mfu 0.02%
iter 771: loss 2.1252, time 70.06ms, mfu 0.02%
iter 772: loss 2.1457, time 89.64ms, mfu 0.02%
iter 773: loss 2.1527, time 150.23ms, mfu 0.02%
iter 774: loss 2.0532, time 83.34ms, mfu 0.02%
iter 775: loss 2.0334, time 48.30ms, mfu 0.02%
iter 776: loss 2.1300, time 48.27ms, mfu 0.02%
iter 777: loss 2.1024, time 39.73ms, mfu 0.02%
iter 778: loss 2.0927, time 67.31ms, mfu 0.02%
iter 779: loss 2.0962, time 75.80ms, mfu 0.02%
iter 780: loss 2.0800, time 62.20ms, mfu 0.02%
iter 781: loss 2.0187, time 79.66ms, mfu 0.02%
iter 782: loss 2.1151, time 49.73ms, mfu 0.02%
iter 783: loss 2.2014, time 58.22ms, mfu 0.02%
iter 784: loss 2.2639, time 55.55ms, mfu 0.02%
iter 785: loss 2.0340, time 37.21ms, mfu 0.02%
iter 786: loss 2.0403, time 44.18ms, mfu 0.02%
iter 787: loss 2.0703, time 40.75ms, mfu 0.02%
iter 788: loss 2.2301, time 29.98ms, mfu 0.03%
iter 789: loss 2.1560, time 27.98ms, mfu 0.03%
iter 790: loss 2.0264, time 56.96ms, mfu 0.03%
iter 791: loss 2.0941, time 42.14ms, mfu 0.03%
iter 792: loss 2.0005, time 36.96ms, mfu 0.03%
iter 793: loss 2.1123, time 36.84ms, mfu 0.03%
iter 794: loss 2.1383, time 32.82ms, mfu 0.03%
iter 795: loss 2.2083, time 44.79ms, mfu 0.03%
iter 796: loss 2.1705, time 54.87ms, mfu 0.03%
iter 797: loss 2.1470, time 65.83ms, mfu 0.03%
iter 798: loss 2.1218, time 43.73ms, mfu 0.03%
iter 799: loss 2.0639, time 60.84ms, mfu 0.03%
iter 800: loss 2.2092, time 74.15ms, mfu 0.03%
iter 801: loss 2.1228, time 81.14ms, mfu 0.03%
iter 802: loss 2.1245, time 280.69ms, mfu 0.02%
iter 803: loss 2.1444, time 227.57ms, mfu 0.02%
iter 804: loss 2.1201, time 276.74ms, mfu 0.02%
iter 805: loss 2.0980, time 459.50ms, mfu 0.02%
iter 806: loss 2.1103, time 307.76ms, mfu 0.02%
iter 807: loss 1.9970, time 155.52ms, mfu 0.02%
iter 808: loss 2.1820, time 255.90ms, mfu 0.01%
iter 809: loss 2.1460, time 95.64ms, mfu 0.01%
iter 810: loss 2.0301, time 105.82ms, mfu 0.01%
iter 811: loss 2.1747, time 355.08ms, mfu 0.01%
iter 812: loss 2.0890, time 229.24ms, mfu 0.01%
iter 813: loss 2.0966, time 231.47ms, mfu 0.01%
iter 814: loss 2.1267, time 139.51ms, mfu 0.01%
iter 815: loss 2.2304, time 69.35ms, mfu 0.01%
iter 816: loss 2.0944, time 63.44ms, mfu 0.01%
iter 817: loss 2.0696, time 71.25ms, mfu 0.01%
iter 818: loss 2.0443, time 153.96ms, mfu 0.01%
iter 819: loss 2.1775, time 111.87ms, mfu 0.01%
iter 820: loss 2.0808, time 96.33ms, mfu 0.01%
iter 821: loss 2.0726, time 137.23ms, mfu 0.01%
iter 822: loss 2.1358, time 87.63ms, mfu 0.01%
iter 823: loss 2.0497, time 77.94ms, mfu 0.01%
iter 824: loss 2.0255, time 80.83ms, mfu 0.01%
iter 825: loss 2.1156, time 67.55ms, mfu 0.01%
iter 826: loss 2.1116, time 64.49ms, mfu 0.01%
iter 827: loss 2.1186, time 59.00ms, mfu 0.02%
iter 828: loss 2.0776, time 58.40ms, mfu 0.02%
iter 829: loss 1.9653, time 73.14ms, mfu 0.02%
iter 830: loss 2.0330, time 81.03ms, mfu 0.02%
iter 831: loss 2.1503, time 53.58ms, mfu 0.02%
iter 832: loss 2.0106, time 53.06ms, mfu 0.02%
iter 833: loss 2.0833, time 71.27ms, mfu 0.02%
iter 834: loss 2.1326, time 113.64ms, mfu 0.02%
iter 835: loss 2.1199, time 115.76ms, mfu 0.02%
iter 836: loss 2.0372, time 134.20ms, mfu 0.02%
iter 837: loss 2.2104, time 126.63ms, mfu 0.02%
iter 838: loss 2.1157, time 74.27ms, mfu 0.02%
iter 839: loss 2.1274, time 64.22ms, mfu 0.02%
iter 840: loss 2.0744, time 103.01ms, mfu 0.02%
iter 841: loss 2.0000, time 152.10ms, mfu 0.01%
iter 842: loss 2.0769, time 71.34ms, mfu 0.01%
iter 843: loss 2.0576, time 46.43ms, mfu 0.02%
iter 844: loss 2.1378, time 77.71ms, mfu 0.02%
iter 845: loss 2.0132, time 87.44ms, mfu 0.02%
iter 846: loss 2.1043, time 219.69ms, mfu 0.02%
iter 847: loss 1.9704, time 103.45ms, mfu 0.01%
iter 848: loss 1.9988, time 130.10ms, mfu 0.01%
iter 849: loss 2.0418, time 70.96ms, mfu 0.01%
iter 850: loss 2.0917, time 63.85ms, mfu 0.02%
iter 851: loss 2.1259, time 60.06ms, mfu 0.02%
iter 852: loss 2.0850, time 60.06ms, mfu 0.02%
iter 853: loss 2.1108, time 42.35ms, mfu 0.02%
iter 854: loss 2.0620, time 92.29ms, mfu 0.02%
iter 855: loss 2.0700, time 104.88ms, mfu 0.02%
iter 856: loss 1.9849, time 105.04ms, mfu 0.02%
iter 857: loss 2.1726, time 76.96ms, mfu 0.02%
iter 858: loss 2.0617, time 67.26ms, mfu 0.02%
iter 859: loss 2.1456, time 56.24ms, mfu 0.02%
iter 860: loss 2.1284, time 68.14ms, mfu 0.02%
iter 861: loss 2.0030, time 59.89ms, mfu 0.02%
iter 862: loss 2.0335, time 150.85ms, mfu 0.02%
iter 863: loss 2.0240, time 83.10ms, mfu 0.02%
iter 864: loss 1.8698, time 66.98ms, mfu 0.02%
iter 865: loss 2.0460, time 68.90ms, mfu 0.02%
iter 866: loss 2.0577, time 56.55ms, mfu 0.02%
iter 867: loss 2.0163, time 59.80ms, mfu 0.02%
iter 868: loss 2.0983, time 51.43ms, mfu 0.02%
iter 869: loss 2.0579, time 70.54ms, mfu 0.02%
iter 870: loss 2.0902, time 87.14ms, mfu 0.02%
iter 871: loss 2.0539, time 82.78ms, mfu 0.02%
iter 872: loss 2.0660, time 102.80ms, mfu 0.02%
iter 873: loss 2.0734, time 60.44ms, mfu 0.02%
iter 874: loss 2.0514, time 89.83ms, mfu 0.02%
iter 875: loss 2.1123, time 41.86ms, mfu 0.02%
iter 876: loss 2.0686, time 37.48ms, mfu 0.02%
iter 877: loss 2.0769, time 43.94ms, mfu 0.02%
iter 878: loss 2.0774, time 57.56ms, mfu 0.02%
iter 879: loss 1.9879, time 86.16ms, mfu 0.02%
iter 880: loss 2.0552, time 67.56ms, mfu 0.02%
iter 881: loss 2.0070, time 97.40ms, mfu 0.02%
iter 882: loss 1.9897, time 114.98ms, mfu 0.02%
iter 883: loss 2.0791, time 300.72ms, mfu 0.02%
iter 884: loss 2.0410, time 256.68ms, mfu 0.02%
iter 885: loss 2.0493, time 148.68ms, mfu 0.02%
iter 886: loss 1.9413, time 186.68ms, mfu 0.01%
iter 887: loss 1.9713, time 101.79ms, mfu 0.01%
iter 888: loss 2.0845, time 134.55ms, mfu 0.01%
iter 889: loss 2.0774, time 100.12ms, mfu 0.01%
iter 890: loss 1.9706, time 96.42ms, mfu 0.01%
iter 891: loss 2.0223, time 121.33ms, mfu 0.01%
iter 892: loss 2.0640, time 70.33ms, mfu 0.01%
iter 893: loss 2.0120, time 62.62ms, mfu 0.01%
iter 894: loss 2.0396, time 64.55ms, mfu 0.01%
iter 895: loss 2.0066, time 62.11ms, mfu 0.02%
iter 896: loss 2.1424, time 56.38ms, mfu 0.02%
iter 897: loss 1.9973, time 85.38ms, mfu 0.02%
iter 898: loss 2.0522, time 58.07ms, mfu 0.02%
iter 899: loss 2.0506, time 45.47ms, mfu 0.02%
iter 900: loss 2.0011, time 50.21ms, mfu 0.02%
iter 901: loss 2.0311, time 68.97ms, mfu 0.02%
iter 902: loss 2.0733, time 126.41ms, mfu 0.02%
iter 903: loss 2.0824, time 104.87ms, mfu 0.02%
iter 904: loss 2.1021, time 81.04ms, mfu 0.02%
iter 905: loss 2.0285, time 73.73ms, mfu 0.02%
iter 906: loss 2.1380, time 45.83ms, mfu 0.02%
iter 907: loss 2.0708, time 56.59ms, mfu 0.02%
iter 908: loss 2.0270, time 58.19ms, mfu 0.02%
iter 909: loss 1.9628, time 60.67ms, mfu 0.02%
iter 910: loss 1.9365, time 74.81ms, mfu 0.02%
iter 911: loss 2.0287, time 85.16ms, mfu 0.02%
iter 912: loss 2.1096, time 87.05ms, mfu 0.02%
iter 913: loss 2.0701, time 168.20ms, mfu 0.02%
iter 914: loss 2.0542, time 94.13ms, mfu 0.02%
iter 915: loss 2.0517, time 84.37ms, mfu 0.02%
iter 916: loss 2.0870, time 42.79ms, mfu 0.02%
iter 917: loss 1.9541, time 60.53ms, mfu 0.02%
iter 918: loss 2.0061, time 58.53ms, mfu 0.02%
iter 919: loss 2.0926, time 201.53ms, mfu 0.02%
iter 920: loss 2.0709, time 84.78ms, mfu 0.02%
iter 921: loss 1.9773, time 128.51ms, mfu 0.02%
iter 922: loss 2.1213, time 188.42ms, mfu 0.02%
iter 923: loss 2.0736, time 228.49ms, mfu 0.01%
iter 924: loss 1.9797, time 205.48ms, mfu 0.01%
iter 925: loss 1.9758, time 139.49ms, mfu 0.01%
iter 926: loss 2.0349, time 89.35ms, mfu 0.01%
iter 927: loss 1.9805, time 65.98ms, mfu 0.01%
iter 928: loss 2.0132, time 154.41ms, mfu 0.01%
iter 929: loss 1.9769, time 132.47ms, mfu 0.01%
iter 930: loss 2.0989, time 101.37ms, mfu 0.01%
iter 931: loss 2.0028, time 129.83ms, mfu 0.01%
iter 932: loss 2.0502, time 74.81ms, mfu 0.01%
iter 933: loss 2.0049, time 129.88ms, mfu 0.01%
iter 934: loss 1.9724, time 83.25ms, mfu 0.01%
iter 935: loss 2.1588, time 71.76ms, mfu 0.01%
iter 936: loss 2.0550, time 72.23ms, mfu 0.01%
iter 937: loss 2.0525, time 55.04ms, mfu 0.01%
iter 938: loss 2.0305, time 59.78ms, mfu 0.02%
iter 939: loss 2.0991, time 91.57ms, mfu 0.02%
iter 940: loss 1.9304, time 50.23ms, mfu 0.02%
iter 941: loss 2.1198, time 73.78ms, mfu 0.02%
iter 942: loss 2.0252, time 53.19ms, mfu 0.02%
iter 943: loss 2.0708, time 78.36ms, mfu 0.02%
iter 944: loss 2.1878, time 82.40ms, mfu 0.02%
iter 945: loss 2.0171, time 64.82ms, mfu 0.02%
iter 946: loss 2.0426, time 32.19ms, mfu 0.02%
iter 947: loss 2.1607, time 59.77ms, mfu 0.02%
iter 948: loss 2.0450, time 36.42ms, mfu 0.02%
iter 949: loss 1.9960, time 33.93ms, mfu 0.02%
iter 950: loss 2.0163, time 46.81ms, mfu 0.02%
iter 951: loss 2.0025, time 51.42ms, mfu 0.02%
iter 952: loss 1.9233, time 75.60ms, mfu 0.02%
iter 953: loss 2.0273, time 84.29ms, mfu 0.02%
iter 954: loss 1.9408, time 75.94ms, mfu 0.02%
iter 955: loss 1.9250, time 60.59ms, mfu 0.02%
iter 956: loss 1.9735, time 81.01ms, mfu 0.02%
iter 957: loss 2.0590, time 74.26ms, mfu 0.02%
iter 958: loss 1.9693, time 78.31ms, mfu 0.02%
iter 959: loss 2.0408, time 85.84ms, mfu 0.02%
iter 960: loss 2.0722, time 81.93ms, mfu 0.02%
iter 961: loss 2.0412, time 51.47ms, mfu 0.02%
iter 962: loss 1.8536, time 46.62ms, mfu 0.02%
iter 963: loss 2.0340, time 41.99ms, mfu 0.02%
iter 964: loss 1.9469, time 32.37ms, mfu 0.02%
iter 965: loss 2.0986, time 32.14ms, mfu 0.02%
iter 966: loss 2.0314, time 38.96ms, mfu 0.03%
iter 967: loss 2.0290, time 31.83ms, mfu 0.03%
iter 968: loss 1.9671, time 34.62ms, mfu 0.03%
iter 969: loss 2.0317, time 63.87ms, mfu 0.03%
iter 970: loss 1.9988, time 46.47ms, mfu 0.03%
iter 971: loss 2.0281, time 32.85ms, mfu 0.03%
iter 972: loss 2.0254, time 37.58ms, mfu 0.03%
iter 973: loss 2.0472, time 36.32ms, mfu 0.03%
iter 974: loss 2.0509, time 73.63ms, mfu 0.03%
iter 975: loss 2.0205, time 76.33ms, mfu 0.03%
iter 976: loss 1.9363, time 55.97ms, mfu 0.03%
iter 977: loss 1.9784, time 85.34ms, mfu 0.03%
iter 978: loss 1.9620, time 130.69ms, mfu 0.02%
iter 979: loss 2.0388, time 969.64ms, mfu 0.02%
iter 980: loss 2.0707, time 685.44ms, mfu 0.02%
iter 981: loss 1.9428, time 103.02ms, mfu 0.02%
iter 982: loss 1.9679, time 75.65ms, mfu 0.02%
iter 983: loss 1.9931, time 66.04ms, mfu 0.02%
iter 984: loss 2.0907, time 89.52ms, mfu 0.02%
iter 985: loss 1.9108, time 115.76ms, mfu 0.02%
iter 986: loss 1.8807, time 148.95ms, mfu 0.02%
iter 987: loss 2.1196, time 60.35ms, mfu 0.02%
iter 988: loss 2.0061, time 83.08ms, mfu 0.02%
iter 989: loss 2.1192, time 84.54ms, mfu 0.02%
iter 990: loss 2.0501, time 119.57ms, mfu 0.02%
iter 991: loss 2.0294, time 86.68ms, mfu 0.02%
iter 992: loss 1.8942, time 94.25ms, mfu 0.02%
iter 993: loss 2.0586, time 113.11ms, mfu 0.02%
iter 994: loss 2.0846, time 187.02ms, mfu 0.01%
iter 995: loss 2.0892, time 83.13ms, mfu 0.01%
iter 996: loss 1.9608, time 64.43ms, mfu 0.02%
iter 997: loss 1.9493, time 56.17ms, mfu 0.02%
iter 998: loss 1.8290, time 72.66ms, mfu 0.02%
iter 999: loss 1.9259, time 50.09ms, mfu 0.02%
step 1000: train loss 1.9714, val loss 2.0528
saving checkpoint to out-shakespeare-char
iter 1000: loss 2.0455, time 3638.06ms, mfu 0.02%
iter 1001: loss 1.9187, time 140.31ms, mfu 0.01%
iter 1002: loss 2.0082, time 105.65ms, mfu 0.01%
iter 1003: loss 2.0436, time 118.64ms, mfu 0.01%
iter 1004: loss 1.9564, time 240.04ms, mfu 0.01%
iter 1005: loss 2.0181, time 48.93ms, mfu 0.01%
iter 1006: loss 1.9583, time 70.76ms, mfu 0.01%
iter 1007: loss 2.0487, time 69.62ms, mfu 0.02%
iter 1008: loss 1.9878, time 61.14ms, mfu 0.02%
iter 1009: loss 1.9712, time 60.31ms, mfu 0.02%
iter 1010: loss 2.0443, time 59.57ms, mfu 0.02%
iter 1011: loss 2.0011, time 65.95ms, mfu 0.02%
iter 1012: loss 1.9721, time 44.80ms, mfu 0.02%
iter 1013: loss 1.8608, time 50.93ms, mfu 0.02%
iter 1014: loss 1.9983, time 51.02ms, mfu 0.02%
iter 1015: loss 2.0427, time 34.15ms, mfu 0.02%
iter 1016: loss 2.1544, time 33.32ms, mfu 0.02%
iter 1017: loss 2.0251, time 55.85ms, mfu 0.02%
iter 1018: loss 2.0941, time 57.38ms, mfu 0.02%
iter 1019: loss 2.0758, time 76.67ms, mfu 0.02%
iter 1020: loss 1.9418, time 67.26ms, mfu 0.02%
iter 1021: loss 1.9768, time 61.66ms, mfu 0.02%
iter 1022: loss 2.0440, time 76.84ms, mfu 0.02%
iter 1023: loss 2.0341, time 54.93ms, mfu 0.02%
iter 1024: loss 1.9736, time 65.65ms, mfu 0.02%
iter 1025: loss 1.9829, time 69.14ms, mfu 0.02%
iter 1026: loss 1.9681, time 49.37ms, mfu 0.02%
iter 1027: loss 1.9797, time 43.50ms, mfu 0.02%
iter 1028: loss 2.1521, time 67.35ms, mfu 0.02%
iter 1029: loss 2.0399, time 77.29ms, mfu 0.02%
iter 1030: loss 2.0274, time 60.96ms, mfu 0.02%
iter 1031: loss 1.9911, time 50.74ms, mfu 0.02%
iter 1032: loss 2.1031, time 60.50ms, mfu 0.02%
iter 1033: loss 2.0002, time 164.41ms, mfu 0.02%
iter 1034: loss 1.9311, time 348.33ms, mfu 0.02%
iter 1035: loss 2.0402, time 326.34ms, mfu 0.02%
iter 1036: loss 1.9620, time 916.22ms, mfu 0.02%
iter 1037: loss 1.9719, time 153.55ms, mfu 0.01%
iter 1038: loss 1.9493, time 165.54ms, mfu 0.01%
iter 1039: loss 2.0411, time 90.56ms, mfu 0.01%
iter 1040: loss 1.9650, time 87.08ms, mfu 0.01%
iter 1041: loss 2.0880, time 104.94ms, mfu 0.01%
iter 1042: loss 1.9858, time 73.82ms, mfu 0.01%
iter 1043: loss 2.0099, time 46.26ms, mfu 0.02%
iter 1044: loss 2.0070, time 72.37ms, mfu 0.02%
iter 1045: loss 1.9664, time 61.20ms, mfu 0.02%
iter 1046: loss 1.9515, time 60.35ms, mfu 0.02%
iter 1047: loss 2.0003, time 83.28ms, mfu 0.02%
iter 1048: loss 1.9025, time 48.89ms, mfu 0.02%
iter 1049: loss 1.9591, time 67.97ms, mfu 0.02%
iter 1050: loss 1.9098, time 59.37ms, mfu 0.02%
iter 1051: loss 1.9447, time 39.19ms, mfu 0.02%
iter 1052: loss 2.0019, time 44.61ms, mfu 0.02%
iter 1053: loss 2.0268, time 40.24ms, mfu 0.02%
iter 1054: loss 2.0150, time 48.36ms, mfu 0.02%
iter 1055: loss 2.0053, time 48.89ms, mfu 0.02%
iter 1056: loss 1.9245, time 37.35ms, mfu 0.02%
iter 1057: loss 1.9400, time 43.29ms, mfu 0.02%
iter 1058: loss 2.0402, time 54.11ms, mfu 0.02%
iter 1059: loss 1.9493, time 38.51ms, mfu 0.02%
iter 1060: loss 1.9377, time 40.90ms, mfu 0.03%
iter 1061: loss 1.8814, time 47.28ms, mfu 0.03%
iter 1062: loss 1.9186, time 38.21ms, mfu 0.03%
iter 1063: loss 2.0572, time 46.13ms, mfu 0.03%
iter 1064: loss 1.9428, time 109.65ms, mfu 0.03%
iter 1065: loss 1.9515, time 100.41ms, mfu 0.02%
iter 1066: loss 2.0536, time 91.08ms, mfu 0.02%
iter 1067: loss 2.0140, time 62.60ms, mfu 0.02%
iter 1068: loss 2.0928, time 75.82ms, mfu 0.02%
iter 1069: loss 1.9994, time 60.11ms, mfu 0.02%
iter 1070: loss 1.9572, time 86.09ms, mfu 0.02%
iter 1071: loss 1.9742, time 99.91ms, mfu 0.02%
iter 1072: loss 2.0662, time 54.77ms, mfu 0.02%
iter 1073: loss 1.9530, time 64.59ms, mfu 0.02%
iter 1074: loss 2.0061, time 79.28ms, mfu 0.02%
iter 1075: loss 2.0648, time 76.06ms, mfu 0.02%
iter 1076: loss 1.9733, time 102.92ms, mfu 0.02%
iter 1077: loss 1.9448, time 127.66ms, mfu 0.02%
iter 1078: loss 1.9525, time 99.94ms, mfu 0.02%
iter 1079: loss 1.8720, time 90.79ms, mfu 0.02%
iter 1080: loss 2.1023, time 93.50ms, mfu 0.02%
iter 1081: loss 1.9793, time 91.52ms, mfu 0.02%
iter 1082: loss 1.9812, time 74.74ms, mfu 0.02%
iter 1083: loss 2.0209, time 75.49ms, mfu 0.02%
iter 1084: loss 2.0688, time 101.02ms, mfu 0.02%
iter 1085: loss 1.9859, time 54.25ms, mfu 0.02%
iter 1086: loss 1.9598, time 177.14ms, mfu 0.02%
iter 1087: loss 1.9040, time 205.36ms, mfu 0.02%
iter 1088: loss 1.8467, time 641.00ms, mfu 0.01%
iter 1089: loss 1.9743, time 270.67ms, mfu 0.01%
iter 1090: loss 1.9512, time 307.77ms, mfu 0.01%
iter 1091: loss 1.9888, time 103.50ms, mfu 0.01%
iter 1092: loss 1.9321, time 219.41ms, mfu 0.01%
iter 1093: loss 2.0330, time 113.48ms, mfu 0.01%
iter 1094: loss 2.0521, time 119.33ms, mfu 0.01%
iter 1095: loss 2.0652, time 85.13ms, mfu 0.01%
iter 1096: loss 2.0213, time 68.75ms, mfu 0.01%
iter 1097: loss 2.0423, time 87.77ms, mfu 0.01%
iter 1098: loss 1.9557, time 70.85ms, mfu 0.01%
iter 1099: loss 1.9890, time 41.63ms, mfu 0.01%
iter 1100: loss 1.7907, time 58.60ms, mfu 0.02%
iter 1101: loss 1.9019, time 45.72ms, mfu 0.02%
iter 1102: loss 1.9016, time 67.22ms, mfu 0.02%
iter 1103: loss 1.9404, time 83.71ms, mfu 0.02%
iter 1104: loss 1.9233, time 47.51ms, mfu 0.02%
iter 1105: loss 1.9644, time 78.51ms, mfu 0.02%
iter 1106: loss 2.0463, time 82.86ms, mfu 0.02%
iter 1107: loss 1.9365, time 70.82ms, mfu 0.02%
iter 1108: loss 1.9146, time 83.54ms, mfu 0.02%
iter 1109: loss 1.9921, time 144.11ms, mfu 0.02%
iter 1110: loss 1.9410, time 107.31ms, mfu 0.02%
iter 1111: loss 1.9718, time 139.88ms, mfu 0.02%
iter 1112: loss 1.8669, time 65.60ms, mfu 0.02%
iter 1113: loss 1.9348, time 66.41ms, mfu 0.02%
iter 1114: loss 1.8593, time 68.08ms, mfu 0.02%
iter 1115: loss 1.9424, time 92.99ms, mfu 0.02%
iter 1116: loss 1.8555, time 108.12ms, mfu 0.02%
iter 1117: loss 2.0105, time 79.36ms, mfu 0.02%
iter 1118: loss 2.0644, time 83.15ms, mfu 0.02%
iter 1119: loss 1.9743, time 73.73ms, mfu 0.02%
iter 1120: loss 1.9305, time 59.32ms, mfu 0.02%
iter 1121: loss 1.9816, time 62.96ms, mfu 0.02%
iter 1122: loss 1.8883, time 88.06ms, mfu 0.02%
iter 1123: loss 1.9025, time 62.42ms, mfu 0.02%
iter 1124: loss 1.9972, time 46.41ms, mfu 0.02%
iter 1125: loss 1.9137, time 102.35ms, mfu 0.02%
iter 1126: loss 1.9450, time 99.60ms, mfu 0.02%
iter 1127: loss 1.8206, time 152.24ms, mfu 0.02%
iter 1128: loss 1.9851, time 128.13ms, mfu 0.02%
iter 1129: loss 1.9800, time 88.07ms, mfu 0.02%
iter 1130: loss 1.8721, time 121.44ms, mfu 0.01%
iter 1131: loss 1.8721, time 328.92ms, mfu 0.01%
iter 1132: loss 2.0754, time 439.96ms, mfu 0.01%
iter 1133: loss 1.9428, time 109.02ms, mfu 0.01%
iter 1134: loss 1.8430, time 201.61ms, mfu 0.01%
iter 1135: loss 1.9807, time 163.88ms, mfu 0.01%
iter 1136: loss 1.9848, time 107.83ms, mfu 0.01%
iter 1137: loss 1.9448, time 56.23ms, mfu 0.01%
iter 1138: loss 2.0107, time 38.75ms, mfu 0.01%
iter 1139: loss 2.0077, time 39.42ms, mfu 0.02%
iter 1140: loss 1.9690, time 55.87ms, mfu 0.02%
iter 1141: loss 2.0303, time 62.25ms, mfu 0.02%
iter 1142: loss 1.8694, time 51.20ms, mfu 0.02%
iter 1143: loss 1.9680, time 56.14ms, mfu 0.02%
iter 1144: loss 1.8829, time 43.48ms, mfu 0.02%
iter 1145: loss 1.9843, time 59.71ms, mfu 0.02%
iter 1146: loss 1.8586, time 47.63ms, mfu 0.02%
iter 1147: loss 1.9180, time 34.28ms, mfu 0.02%
iter 1148: loss 1.9007, time 78.89ms, mfu 0.02%
iter 1149: loss 1.9126, time 64.08ms, mfu 0.02%
iter 1150: loss 1.9558, time 52.69ms, mfu 0.02%
iter 1151: loss 1.9550, time 68.14ms, mfu 0.02%
iter 1152: loss 2.0435, time 39.41ms, mfu 0.02%
iter 1153: loss 1.9602, time 47.41ms, mfu 0.02%
iter 1154: loss 1.9603, time 49.04ms, mfu 0.02%
iter 1155: loss 1.9797, time 42.85ms, mfu 0.02%
iter 1156: loss 1.9090, time 34.10ms, mfu 0.03%
iter 1157: loss 1.9850, time 47.36ms, mfu 0.03%
iter 1158: loss 2.0127, time 43.00ms, mfu 0.03%
iter 1159: loss 1.9794, time 58.38ms, mfu 0.03%
iter 1160: loss 2.0075, time 42.38ms, mfu 0.03%
iter 1161: loss 2.0370, time 52.28ms, mfu 0.03%
iter 1162: loss 1.9883, time 54.85ms, mfu 0.03%
iter 1163: loss 2.0064, time 41.28ms, mfu 0.03%
iter 1164: loss 2.0182, time 44.00ms, mfu 0.03%
iter 1165: loss 1.8655, time 49.68ms, mfu 0.03%
iter 1166: loss 1.8017, time 48.50ms, mfu 0.03%
iter 1167: loss 1.8910, time 44.99ms, mfu 0.03%
iter 1168: loss 1.7690, time 60.72ms, mfu 0.03%
iter 1169: loss 1.9658, time 45.37ms, mfu 0.03%
iter 1170: loss 1.9272, time 37.26ms, mfu 0.03%
iter 1171: loss 1.9648, time 58.01ms, mfu 0.03%
iter 1172: loss 1.8443, time 51.04ms, mfu 0.03%
iter 1173: loss 1.9639, time 63.46ms, mfu 0.03%
iter 1174: loss 1.9477, time 61.01ms, mfu 0.03%
iter 1175: loss 1.8315, time 51.01ms, mfu 0.03%
iter 1176: loss 1.9440, time 47.19ms, mfu 0.03%
iter 1177: loss 1.9990, time 68.53ms, mfu 0.02%
iter 1178: loss 1.9571, time 63.44ms, mfu 0.02%
iter 1179: loss 1.8974, time 49.77ms, mfu 0.02%
iter 1180: loss 2.0595, time 75.64ms, mfu 0.02%
iter 1181: loss 1.9522, time 75.47ms, mfu 0.02%
iter 1182: loss 1.9216, time 59.90ms, mfu 0.02%
iter 1183: loss 1.9478, time 73.35ms, mfu 0.02%
iter 1184: loss 1.9208, time 47.11ms, mfu 0.02%
iter 1185: loss 2.0696, time 65.22ms, mfu 0.02%
iter 1186: loss 1.9023, time 86.96ms, mfu 0.02%
iter 1187: loss 1.8995, time 80.05ms, mfu 0.02%
iter 1188: loss 1.9731, time 152.01ms, mfu 0.02%
iter 1189: loss 1.9414, time 123.56ms, mfu 0.02%
iter 1190: loss 1.9560, time 53.86ms, mfu 0.02%
iter 1191: loss 2.0242, time 52.99ms, mfu 0.02%
iter 1192: loss 1.9800, time 50.26ms, mfu 0.02%
iter 1193: loss 1.9320, time 39.37ms, mfu 0.02%
iter 1194: loss 1.9850, time 77.31ms, mfu 0.02%
iter 1195: loss 1.9885, time 48.12ms, mfu 0.02%
iter 1196: loss 1.9300, time 42.71ms, mfu 0.02%
iter 1197: loss 1.9234, time 67.95ms, mfu 0.02%
iter 1198: loss 1.8685, time 48.74ms, mfu 0.02%
iter 1199: loss 1.8680, time 47.22ms, mfu 0.02%
iter 1200: loss 1.9533, time 70.99ms, mfu 0.02%
iter 1201: loss 1.9166, time 81.11ms, mfu 0.02%
iter 1202: loss 1.9262, time 78.63ms, mfu 0.02%
iter 1203: loss 1.8414, time 85.36ms, mfu 0.02%
iter 1204: loss 1.9662, time 74.92ms, mfu 0.02%
iter 1205: loss 1.8065, time 207.51ms, mfu 0.02%
iter 1206: loss 1.8671, time 78.08ms, mfu 0.02%
iter 1207: loss 1.9174, time 96.92ms, mfu 0.02%
iter 1208: loss 1.8482, time 70.24ms, mfu 0.02%
iter 1209: loss 1.9081, time 39.10ms, mfu 0.02%
iter 1210: loss 1.9181, time 48.33ms, mfu 0.02%
iter 1211: loss 1.9738, time 75.94ms, mfu 0.02%
iter 1212: loss 1.9848, time 103.41ms, mfu 0.02%
iter 1213: loss 1.9242, time 261.17ms, mfu 0.02%
iter 1214: loss 1.8897, time 176.34ms, mfu 0.02%
iter 1215: loss 2.0222, time 322.55ms, mfu 0.02%
iter 1216: loss 1.9389, time 627.38ms, mfu 0.01%
iter 1217: loss 1.8715, time 244.52ms, mfu 0.01%
iter 1218: loss 1.8319, time 224.50ms, mfu 0.01%
iter 1219: loss 2.0098, time 232.66ms, mfu 0.01%
iter 1220: loss 1.8898, time 267.72ms, mfu 0.01%
iter 1221: loss 1.8818, time 131.46ms, mfu 0.01%
iter 1222: loss 1.7452, time 77.66ms, mfu 0.01%
iter 1223: loss 1.8916, time 82.67ms, mfu 0.01%
iter 1224: loss 1.9362, time 74.69ms, mfu 0.01%
iter 1225: loss 1.9277, time 64.14ms, mfu 0.01%
iter 1226: loss 1.8854, time 75.01ms, mfu 0.01%
iter 1227: loss 1.9490, time 77.38ms, mfu 0.01%
iter 1228: loss 1.8277, time 85.06ms, mfu 0.01%
iter 1229: loss 1.7937, time 113.85ms, mfu 0.01%
iter 1230: loss 1.9035, time 99.13ms, mfu 0.01%
iter 1231: loss 1.8742, time 78.46ms, mfu 0.01%
iter 1232: loss 1.9752, time 72.35ms, mfu 0.01%
iter 1233: loss 1.9890, time 66.47ms, mfu 0.01%
iter 1234: loss 1.9082, time 41.35ms, mfu 0.02%
iter 1235: loss 1.7999, time 45.69ms, mfu 0.02%
iter 1236: loss 1.9449, time 36.61ms, mfu 0.02%
iter 1237: loss 1.9138, time 62.01ms, mfu 0.02%
iter 1238: loss 1.8073, time 54.30ms, mfu 0.02%
iter 1239: loss 1.9190, time 38.14ms, mfu 0.02%
iter 1240: loss 1.8668, time 45.57ms, mfu 0.02%
iter 1241: loss 1.8846, time 90.21ms, mfu 0.02%
iter 1242: loss 1.8601, time 54.32ms, mfu 0.02%
iter 1243: loss 1.9524, time 59.95ms, mfu 0.02%
iter 1244: loss 1.8929, time 103.77ms, mfu 0.02%
iter 1245: loss 1.7851, time 118.80ms, mfu 0.02%
iter 1246: loss 1.8750, time 72.93ms, mfu 0.02%
iter 1247: loss 1.9050, time 237.26ms, mfu 0.02%
iter 1248: loss 1.9501, time 70.67ms, mfu 0.02%
iter 1249: loss 1.8222, time 76.67ms, mfu 0.02%
step 1250: train loss 1.8756, val loss 2.0089
saving checkpoint to out-shakespeare-char
iter 1250: loss 1.8472, time 1492.12ms, mfu 0.02%
iter 1251: loss 1.9505, time 198.42ms, mfu 0.02%
iter 1252: loss 2.0043, time 138.53ms, mfu 0.01%
iter 1253: loss 1.7978, time 121.24ms, mfu 0.01%
iter 1254: loss 1.9211, time 82.53ms, mfu 0.01%
iter 1255: loss 1.8505, time 170.28ms, mfu 0.01%
iter 1256: loss 1.9438, time 101.15ms, mfu 0.01%
iter 1257: loss 1.8914, time 373.46ms, mfu 0.01%
iter 1258: loss 1.9125, time 115.22ms, mfu 0.01%
iter 1259: loss 1.8568, time 106.94ms, mfu 0.01%
iter 1260: loss 1.8822, time 203.06ms, mfu 0.01%
iter 1261: loss 1.9794, time 160.29ms, mfu 0.01%
iter 1262: loss 1.8935, time 89.54ms, mfu 0.01%
iter 1263: loss 1.9714, time 115.82ms, mfu 0.01%
iter 1264: loss 1.8707, time 81.19ms, mfu 0.01%
iter 1265: loss 1.8929, time 106.75ms, mfu 0.01%
iter 1266: loss 1.8141, time 77.54ms, mfu 0.01%
iter 1267: loss 1.8904, time 91.48ms, mfu 0.01%
iter 1268: loss 1.9340, time 147.14ms, mfu 0.01%
iter 1269: loss 1.9194, time 145.98ms, mfu 0.01%
iter 1270: loss 1.9168, time 117.27ms, mfu 0.01%
iter 1271: loss 1.7886, time 85.59ms, mfu 0.01%
iter 1272: loss 1.8410, time 55.73ms, mfu 0.01%
iter 1273: loss 1.8824, time 139.17ms, mfu 0.01%
iter 1274: loss 1.8806, time 136.28ms, mfu 0.01%
iter 1275: loss 1.8993, time 302.27ms, mfu 0.01%
iter 1276: loss 1.6987, time 172.87ms, mfu 0.01%
iter 1277: loss 1.9277, time 237.66ms, mfu 0.01%
iter 1278: loss 1.8362, time 494.42ms, mfu 0.01%
iter 1279: loss 1.8692, time 403.75ms, mfu 0.01%
iter 1280: loss 1.7214, time 125.04ms, mfu 0.01%
iter 1281: loss 1.9492, time 191.76ms, mfu 0.01%
iter 1282: loss 1.9036, time 186.03ms, mfu 0.01%
iter 1283: loss 1.8903, time 96.99ms, mfu 0.01%
iter 1284: loss 1.8991, time 72.15ms, mfu 0.01%
iter 1285: loss 1.8804, time 84.72ms, mfu 0.01%
iter 1286: loss 1.9742, time 70.67ms, mfu 0.01%
iter 1287: loss 1.8957, time 80.93ms, mfu 0.01%
iter 1288: loss 1.7816, time 68.60ms, mfu 0.01%
iter 1289: loss 1.9166, time 81.61ms, mfu 0.01%
iter 1290: loss 1.9213, time 61.36ms, mfu 0.01%
iter 1291: loss 1.9487, time 94.19ms, mfu 0.01%
iter 1292: loss 1.8637, time 122.15ms, mfu 0.01%
iter 1293: loss 1.9852, time 193.60ms, mfu 0.01%
iter 1294: loss 1.8977, time 115.22ms, mfu 0.01%
iter 1295: loss 1.8581, time 98.18ms, mfu 0.01%
iter 1296: loss 1.8894, time 105.78ms, mfu 0.01%
iter 1297: loss 1.9383, time 90.20ms, mfu 0.01%
iter 1298: loss 1.8687, time 58.26ms, mfu 0.01%
iter 1299: loss 2.0321, time 127.68ms, mfu 0.01%
iter 1300: loss 1.9599, time 97.21ms, mfu 0.01%
iter 1301: loss 1.8696, time 79.46ms, mfu 0.01%
iter 1302: loss 1.8468, time 158.81ms, mfu 0.01%
iter 1303: loss 1.9068, time 67.58ms, mfu 0.01%
iter 1304: loss 1.8604, time 84.80ms, mfu 0.01%
iter 1305: loss 1.9658, time 81.60ms, mfu 0.01%
iter 1306: loss 1.9136, time 100.67ms, mfu 0.01%
iter 1307: loss 1.8064, time 103.77ms, mfu 0.01%
iter 1308: loss 1.9828, time 476.67ms, mfu 0.01%
iter 1309: loss 1.9513, time 411.68ms, mfu 0.01%
iter 1310: loss 1.8802, time 112.74ms, mfu 0.01%
iter 1311: loss 1.9208, time 59.13ms, mfu 0.01%
iter 1312: loss 1.8707, time 44.57ms, mfu 0.01%
iter 1313: loss 1.9803, time 49.51ms, mfu 0.02%
iter 1314: loss 1.8814, time 34.94ms, mfu 0.02%
iter 1315: loss 1.9117, time 62.67ms, mfu 0.02%
iter 1316: loss 1.8104, time 63.32ms, mfu 0.02%
iter 1317: loss 1.8911, time 73.40ms, mfu 0.02%
iter 1318: loss 1.8885, time 84.92ms, mfu 0.02%
iter 1319: loss 1.9571, time 79.15ms, mfu 0.02%
iter 1320: loss 1.9631, time 96.99ms, mfu 0.02%
iter 1321: loss 1.8718, time 65.49ms, mfu 0.02%
iter 1322: loss 2.0186, time 60.11ms, mfu 0.02%
iter 1323: loss 1.8636, time 42.70ms, mfu 0.02%
iter 1324: loss 1.8920, time 44.45ms, mfu 0.02%
iter 1325: loss 1.8935, time 39.82ms, mfu 0.02%
iter 1326: loss 1.9010, time 33.40ms, mfu 0.02%
iter 1327: loss 2.0102, time 65.91ms, mfu 0.02%
iter 1328: loss 1.7928, time 48.61ms, mfu 0.02%
iter 1329: loss 1.8508, time 79.56ms, mfu 0.02%
iter 1330: loss 1.9832, time 57.37ms, mfu 0.02%
iter 1331: loss 1.8661, time 78.33ms, mfu 0.02%
iter 1332: loss 1.8195, time 65.74ms, mfu 0.02%
iter 1333: loss 1.8314, time 68.42ms, mfu 0.02%
iter 1334: loss 1.9103, time 66.10ms, mfu 0.02%
iter 1335: loss 1.8740, time 54.50ms, mfu 0.02%
iter 1336: loss 1.9292, time 73.49ms, mfu 0.02%
iter 1337: loss 1.9404, time 49.37ms, mfu 0.02%
iter 1338: loss 1.7941, time 54.65ms, mfu 0.02%
iter 1339: loss 1.8813, time 40.11ms, mfu 0.02%
iter 1340: loss 1.9444, time 32.31ms, mfu 0.02%
iter 1341: loss 1.9021, time 48.14ms, mfu 0.02%
iter 1342: loss 1.9020, time 43.73ms, mfu 0.02%
iter 1343: loss 1.8683, time 42.63ms, mfu 0.03%
iter 1344: loss 1.8009, time 35.93ms, mfu 0.03%
iter 1345: loss 1.9014, time 47.21ms, mfu 0.03%
iter 1346: loss 1.8565, time 43.46ms, mfu 0.03%
iter 1347: loss 1.9952, time 40.52ms, mfu 0.03%
iter 1348: loss 1.8950, time 43.79ms, mfu 0.03%
iter 1349: loss 1.8574, time 76.33ms, mfu 0.03%
iter 1350: loss 1.8367, time 124.19ms, mfu 0.02%
iter 1351: loss 1.7387, time 100.38ms, mfu 0.02%
iter 1352: loss 1.8476, time 127.24ms, mfu 0.02%
iter 1353: loss 1.9139, time 253.28ms, mfu 0.02%
iter 1354: loss 1.9258, time 85.30ms, mfu 0.02%
iter 1355: loss 1.8739, time 67.85ms, mfu 0.02%
iter 1356: loss 1.8640, time 89.45ms, mfu 0.02%
iter 1357: loss 1.7721, time 128.05ms, mfu 0.02%
iter 1358: loss 1.8393, time 223.13ms, mfu 0.02%
iter 1359: loss 1.9871, time 460.89ms, mfu 0.02%
iter 1360: loss 1.8941, time 213.52ms, mfu 0.01%
iter 1361: loss 1.8890, time 135.37ms, mfu 0.01%
iter 1362: loss 1.8848, time 203.99ms, mfu 0.01%
iter 1363: loss 1.9245, time 677.54ms, mfu 0.01%
iter 1364: loss 1.8883, time 358.26ms, mfu 0.01%
iter 1365: loss 1.9321, time 377.07ms, mfu 0.01%
iter 1366: loss 1.9759, time 147.84ms, mfu 0.01%
iter 1367: loss 1.7825, time 96.94ms, mfu 0.01%
iter 1368: loss 1.8662, time 87.41ms, mfu 0.01%
iter 1369: loss 1.8926, time 64.94ms, mfu 0.01%
iter 1370: loss 1.8958, time 106.83ms, mfu 0.01%
iter 1371: loss 1.9527, time 76.46ms, mfu 0.01%
iter 1372: loss 1.9104, time 74.39ms, mfu 0.01%
iter 1373: loss 1.8593, time 82.33ms, mfu 0.01%
iter 1374: loss 1.9066, time 112.40ms, mfu 0.01%
iter 1375: loss 1.8517, time 409.30ms, mfu 0.01%
iter 1376: loss 1.7638, time 172.12ms, mfu 0.01%
iter 1377: loss 1.8865, time 124.76ms, mfu 0.01%
iter 1378: loss 1.8301, time 85.89ms, mfu 0.01%
iter 1379: loss 1.9391, time 96.17ms, mfu 0.01%
iter 1380: loss 1.7734, time 66.51ms, mfu 0.01%
iter 1381: loss 1.9339, time 94.28ms, mfu 0.01%
iter 1382: loss 1.7921, time 76.99ms, mfu 0.01%
iter 1383: loss 1.8849, time 75.34ms, mfu 0.01%
iter 1384: loss 1.8568, time 73.28ms, mfu 0.01%
iter 1385: loss 1.9190, time 42.07ms, mfu 0.02%
iter 1386: loss 1.7977, time 50.94ms, mfu 0.02%
iter 1387: loss 1.8599, time 58.29ms, mfu 0.02%
iter 1388: loss 1.8525, time 41.01ms, mfu 0.02%
iter 1389: loss 1.8533, time 57.39ms, mfu 0.02%
iter 1390: loss 1.9301, time 53.57ms, mfu 0.02%
iter 1391: loss 1.8160, time 53.51ms, mfu 0.02%
iter 1392: loss 1.8530, time 90.68ms, mfu 0.02%
iter 1393: loss 1.9791, time 57.41ms, mfu 0.02%
iter 1394: loss 1.8913, time 70.64ms, mfu 0.02%
iter 1395: loss 1.8677, time 79.20ms, mfu 0.02%
iter 1396: loss 1.8882, time 65.32ms, mfu 0.02%
iter 1397: loss 1.8208, time 73.05ms, mfu 0.02%
iter 1398: loss 1.7251, time 83.91ms, mfu 0.02%
iter 1399: loss 1.9319, time 49.63ms, mfu 0.02%
iter 1400: loss 1.8251, time 44.96ms, mfu 0.02%
iter 1401: loss 1.8336, time 33.30ms, mfu 0.02%
iter 1402: loss 1.9545, time 44.74ms, mfu 0.02%
iter 1403: loss 1.8206, time 49.10ms, mfu 0.02%
iter 1404: loss 1.9279, time 42.29ms, mfu 0.02%
iter 1405: loss 1.9249, time 34.15ms, mfu 0.02%
iter 1406: loss 1.8807, time 42.27ms, mfu 0.03%
iter 1407: loss 1.9173, time 52.77ms, mfu 0.03%
iter 1408: loss 1.8707, time 51.93ms, mfu 0.03%
iter 1409: loss 1.9065, time 41.54ms, mfu 0.03%
iter 1410: loss 1.8992, time 94.55ms, mfu 0.02%
iter 1411: loss 1.7874, time 73.02ms, mfu 0.02%
iter 1412: loss 1.8687, time 276.30ms, mfu 0.02%
iter 1413: loss 1.8950, time 435.07ms, mfu 0.02%
iter 1414: loss 1.8444, time 284.53ms, mfu 0.02%
iter 1415: loss 1.8862, time 171.89ms, mfu 0.02%
iter 1416: loss 1.8390, time 157.77ms, mfu 0.02%
iter 1417: loss 1.8496, time 427.51ms, mfu 0.02%
iter 1418: loss 1.9114, time 308.12ms, mfu 0.01%
iter 1419: loss 1.8822, time 118.16ms, mfu 0.01%
iter 1420: loss 1.8308, time 102.19ms, mfu 0.01%
iter 1421: loss 1.8732, time 168.32ms, mfu 0.01%
iter 1422: loss 1.8949, time 136.06ms, mfu 0.01%
iter 1423: loss 1.8320, time 236.96ms, mfu 0.01%
iter 1424: loss 1.7847, time 129.51ms, mfu 0.01%
iter 1425: loss 1.8736, time 128.36ms, mfu 0.01%
iter 1426: loss 1.9243, time 116.06ms, mfu 0.01%
iter 1427: loss 1.8880, time 86.63ms, mfu 0.01%
iter 1428: loss 1.8604, time 65.27ms, mfu 0.01%
iter 1429: loss 1.9600, time 60.47ms, mfu 0.01%
iter 1430: loss 1.8750, time 84.99ms, mfu 0.01%
iter 1431: loss 1.8466, time 103.13ms, mfu 0.01%
iter 1432: loss 2.0068, time 56.75ms, mfu 0.01%
iter 1433: loss 1.8885, time 106.88ms, mfu 0.01%
iter 1434: loss 1.9159, time 173.95ms, mfu 0.01%
iter 1435: loss 1.8457, time 120.44ms, mfu 0.01%
iter 1436: loss 1.9120, time 144.36ms, mfu 0.01%
iter 1437: loss 1.8213, time 87.41ms, mfu 0.01%
iter 1438: loss 1.9079, time 69.33ms, mfu 0.01%
iter 1439: loss 1.7653, time 71.06ms, mfu 0.01%
iter 1440: loss 1.7765, time 78.27ms, mfu 0.01%
iter 1441: loss 1.7510, time 84.67ms, mfu 0.01%
iter 1442: loss 1.8549, time 93.94ms, mfu 0.01%
iter 1443: loss 1.7976, time 128.99ms, mfu 0.01%
iter 1444: loss 1.8192, time 116.29ms, mfu 0.01%
iter 1445: loss 1.8632, time 469.94ms, mfu 0.01%
iter 1446: loss 1.8890, time 213.62ms, mfu 0.01%
iter 1447: loss 1.8887, time 111.40ms, mfu 0.01%
iter 1448: loss 2.0332, time 72.88ms, mfu 0.01%
iter 1449: loss 1.7804, time 68.72ms, mfu 0.01%
iter 1450: loss 1.8467, time 241.58ms, mfu 0.01%
iter 1451: loss 1.8490, time 67.78ms, mfu 0.01%
iter 1452: loss 1.8041, time 53.64ms, mfu 0.01%
iter 1453: loss 1.8066, time 45.01ms, mfu 0.02%
iter 1454: loss 1.7856, time 33.55ms, mfu 0.02%
iter 1455: loss 1.9288, time 60.59ms, mfu 0.02%
iter 1456: loss 1.7772, time 62.93ms, mfu 0.02%
iter 1457: loss 1.8652, time 80.26ms, mfu 0.02%
iter 1458: loss 1.7608, time 55.17ms, mfu 0.02%
iter 1459: loss 1.8071, time 57.31ms, mfu 0.02%
iter 1460: loss 1.8346, time 91.71ms, mfu 0.02%
iter 1461: loss 1.8516, time 155.62ms, mfu 0.02%
iter 1462: loss 1.8396, time 252.39ms, mfu 0.02%
iter 1463: loss 1.8830, time 214.09ms, mfu 0.02%
iter 1464: loss 1.9313, time 157.28ms, mfu 0.01%
iter 1465: loss 1.8594, time 112.23ms, mfu 0.01%
iter 1466: loss 1.8750, time 116.41ms, mfu 0.01%
iter 1467: loss 1.8087, time 88.55ms, mfu 0.01%
iter 1468: loss 1.7728, time 132.41ms, mfu 0.01%
iter 1469: loss 1.8967, time 130.58ms, mfu 0.01%
iter 1470: loss 1.8773, time 72.79ms, mfu 0.01%
iter 1471: loss 1.8480, time 71.23ms, mfu 0.01%
iter 1472: loss 1.8651, time 90.79ms, mfu 0.01%
iter 1473: loss 1.8508, time 102.01ms, mfu 0.01%
iter 1474: loss 1.8673, time 103.22ms, mfu 0.01%
iter 1475: loss 1.9003, time 83.31ms, mfu 0.01%
iter 1476: loss 2.0827, time 133.23ms, mfu 0.01%
iter 1477: loss 1.7993, time 140.53ms, mfu 0.01%
iter 1478: loss 1.8796, time 166.92ms, mfu 0.01%
iter 1479: loss 1.8845, time 129.89ms, mfu 0.01%
iter 1480: loss 1.8389, time 68.20ms, mfu 0.01%
iter 1481: loss 1.7949, time 70.03ms, mfu 0.01%
iter 1482: loss 1.7822, time 73.98ms, mfu 0.01%
iter 1483: loss 1.8141, time 47.25ms, mfu 0.02%
iter 1484: loss 1.8112, time 46.48ms, mfu 0.02%
iter 1485: loss 1.7876, time 70.82ms, mfu 0.02%
iter 1486: loss 1.8575, time 86.86ms, mfu 0.02%
iter 1487: loss 1.8741, time 143.38ms, mfu 0.02%
iter 1488: loss 1.7862, time 216.34ms, mfu 0.01%
iter 1489: loss 1.9402, time 560.40ms, mfu 0.01%
iter 1490: loss 1.8105, time 181.90ms, mfu 0.01%
iter 1491: loss 1.8227, time 104.43ms, mfu 0.01%
iter 1492: loss 1.7702, time 104.37ms, mfu 0.01%
iter 1493: loss 1.7748, time 137.13ms, mfu 0.01%
iter 1494: loss 1.8848, time 200.02ms, mfu 0.01%
iter 1495: loss 1.7646, time 97.84ms, mfu 0.01%
iter 1496: loss 1.7851, time 151.83ms, mfu 0.01%
iter 1497: loss 1.8713, time 141.12ms, mfu 0.01%
iter 1498: loss 1.7788, time 98.53ms, mfu 0.01%
iter 1499: loss 1.8476, time 214.28ms, mfu 0.01%
step 1500: train loss 1.8557, val loss 1.9225
saving checkpoint to out-shakespeare-char
iter 1500: loss 1.8624, time 2249.31ms, mfu 0.01%
iter 1501: loss 1.9029, time 55.20ms, mfu 0.01%
iter 1502: loss 1.8334, time 61.61ms, mfu 0.01%
iter 1503: loss 1.8723, time 72.71ms, mfu 0.01%
iter 1504: loss 1.8613, time 60.90ms, mfu 0.01%
iter 1505: loss 1.8844, time 84.17ms, mfu 0.01%
iter 1506: loss 1.7501, time 104.24ms, mfu 0.01%
iter 1507: loss 1.8751, time 109.48ms, mfu 0.01%
iter 1508: loss 2.0147, time 102.45ms, mfu 0.01%
iter 1509: loss 1.8591, time 106.66ms, mfu 0.01%
iter 1510: loss 1.7505, time 249.91ms, mfu 0.01%
iter 1511: loss 1.8415, time 119.36ms, mfu 0.01%
iter 1512: loss 1.7407, time 79.30ms, mfu 0.01%
iter 1513: loss 1.7791, time 150.20ms, mfu 0.01%
iter 1514: loss 1.8457, time 58.81ms, mfu 0.01%
iter 1515: loss 1.8212, time 72.60ms, mfu 0.01%
iter 1516: loss 1.8134, time 82.25ms, mfu 0.01%
iter 1517: loss 1.7699, time 102.02ms, mfu 0.01%
iter 1518: loss 1.8516, time 130.72ms, mfu 0.01%
iter 1519: loss 1.8508, time 161.02ms, mfu 0.01%
iter 1520: loss 1.8912, time 280.98ms, mfu 0.01%
iter 1521: loss 1.7793, time 537.68ms, mfu 0.01%
iter 1522: loss 1.8573, time 288.35ms, mfu 0.01%
iter 1523: loss 1.8069, time 268.01ms, mfu 0.01%
iter 1524: loss 1.7402, time 492.02ms, mfu 0.01%
iter 1525: loss 1.8282, time 405.22ms, mfu 0.01%
iter 1526: loss 1.8168, time 221.09ms, mfu 0.01%
iter 1527: loss 1.7843, time 65.57ms, mfu 0.01%
iter 1528: loss 1.8344, time 111.87ms, mfu 0.01%
iter 1529: loss 1.8107, time 93.58ms, mfu 0.01%
iter 1530: loss 1.8465, time 114.23ms, mfu 0.01%
iter 1531: loss 1.8449, time 162.65ms, mfu 0.01%
iter 1532: loss 1.8415, time 137.28ms, mfu 0.01%
iter 1533: loss 1.8430, time 91.84ms, mfu 0.01%
iter 1534: loss 1.8973, time 123.42ms, mfu 0.01%
iter 1535: loss 1.7893, time 87.40ms, mfu 0.01%
iter 1536: loss 1.8587, time 137.61ms, mfu 0.01%
iter 1537: loss 1.9075, time 84.01ms, mfu 0.01%
iter 1538: loss 1.8520, time 73.45ms, mfu 0.01%
iter 1539: loss 1.8734, time 253.39ms, mfu 0.01%
iter 1540: loss 1.7848, time 133.72ms, mfu 0.01%
iter 1541: loss 1.8395, time 88.00ms, mfu 0.01%
iter 1542: loss 1.7912, time 103.51ms, mfu 0.01%
iter 1543: loss 1.9071, time 107.95ms, mfu 0.01%
iter 1544: loss 1.8089, time 105.18ms, mfu 0.01%
iter 1545: loss 1.8766, time 103.19ms, mfu 0.01%
iter 1546: loss 1.7501, time 109.91ms, mfu 0.01%
iter 1547: loss 1.8623, time 55.00ms, mfu 0.01%
iter 1548: loss 1.8256, time 59.80ms, mfu 0.01%
iter 1549: loss 1.8087, time 38.81ms, mfu 0.02%
iter 1550: loss 1.8736, time 37.22ms, mfu 0.02%
iter 1551: loss 1.7196, time 40.48ms, mfu 0.02%
iter 1552: loss 1.8209, time 41.84ms, mfu 0.02%
iter 1553: loss 1.8825, time 49.60ms, mfu 0.02%
iter 1554: loss 1.7125, time 41.88ms, mfu 0.02%
iter 1555: loss 1.8519, time 36.08ms, mfu 0.02%
iter 1556: loss 1.7490, time 55.05ms, mfu 0.02%
iter 1557: loss 1.8797, time 67.94ms, mfu 0.02%
iter 1558: loss 1.7944, time 88.26ms, mfu 0.02%
iter 1559: loss 1.8864, time 115.40ms, mfu 0.02%
iter 1560: loss 1.8491, time 172.69ms, mfu 0.02%
iter 1561: loss 1.7478, time 156.20ms, mfu 0.02%
iter 1562: loss 1.7721, time 98.29ms, mfu 0.02%
iter 1563: loss 1.7795, time 68.84ms, mfu 0.02%
iter 1564: loss 1.7300, time 75.15ms, mfu 0.02%
iter 1565: loss 1.7812, time 81.37ms, mfu 0.02%
iter 1566: loss 1.8205, time 85.80ms, mfu 0.02%
iter 1567: loss 1.7802, time 159.33ms, mfu 0.02%
iter 1568: loss 1.7921, time 165.37ms, mfu 0.02%
iter 1569: loss 1.8613, time 135.39ms, mfu 0.01%
iter 1570: loss 1.8325, time 188.30ms, mfu 0.01%
iter 1571: loss 1.8301, time 82.53ms, mfu 0.01%
iter 1572: loss 1.8649, time 48.64ms, mfu 0.02%
iter 1573: loss 1.8089, time 59.36ms, mfu 0.02%
iter 1574: loss 1.6946, time 53.12ms, mfu 0.02%
iter 1575: loss 1.7825, time 37.78ms, mfu 0.02%
iter 1576: loss 1.8883, time 79.13ms, mfu 0.02%
iter 1577: loss 1.7559, time 44.57ms, mfu 0.02%
iter 1578: loss 1.7638, time 68.80ms, mfu 0.02%
iter 1579: loss 1.8210, time 59.50ms, mfu 0.02%
iter 1580: loss 1.7281, time 54.05ms, mfu 0.02%
iter 1581: loss 1.8286, time 66.53ms, mfu 0.02%
iter 1582: loss 1.8387, time 114.38ms, mfu 0.02%
iter 1583: loss 1.7638, time 137.92ms, mfu 0.02%
iter 1584: loss 1.7144, time 89.22ms, mfu 0.02%
iter 1585: loss 1.8740, time 98.52ms, mfu 0.02%
iter 1586: loss 1.8780, time 137.07ms, mfu 0.02%
iter 1587: loss 1.9013, time 90.38ms, mfu 0.02%
iter 1588: loss 1.8361, time 82.26ms, mfu 0.02%
iter 1589: loss 1.7214, time 110.79ms, mfu 0.02%
iter 1590: loss 1.7680, time 85.85ms, mfu 0.02%
iter 1591: loss 1.8756, time 136.97ms, mfu 0.01%
iter 1592: loss 1.6822, time 83.35ms, mfu 0.01%
iter 1593: loss 1.8340, time 315.27ms, mfu 0.01%
iter 1594: loss 1.7960, time 188.33ms, mfu 0.01%
iter 1595: loss 1.8945, time 285.79ms, mfu 0.01%
iter 1596: loss 1.8207, time 628.46ms, mfu 0.01%
iter 1597: loss 1.8352, time 587.42ms, mfu 0.01%
iter 1598: loss 1.7969, time 424.20ms, mfu 0.01%
iter 1599: loss 1.7746, time 145.06ms, mfu 0.01%
iter 1600: loss 1.7808, time 100.53ms, mfu 0.01%
iter 1601: loss 1.7328, time 70.10ms, mfu 0.01%
iter 1602: loss 1.8331, time 63.16ms, mfu 0.01%
iter 1603: loss 1.8316, time 61.41ms, mfu 0.01%
iter 1604: loss 1.8537, time 76.72ms, mfu 0.01%
iter 1605: loss 1.7347, time 115.90ms, mfu 0.01%
iter 1606: loss 1.8569, time 73.98ms, mfu 0.01%
iter 1607: loss 1.8159, time 37.61ms, mfu 0.02%
iter 1608: loss 1.8937, time 59.20ms, mfu 0.02%
iter 1609: loss 1.8270, time 56.24ms, mfu 0.02%
iter 1610: loss 1.8423, time 53.02ms, mfu 0.02%
iter 1611: loss 1.8658, time 51.99ms, mfu 0.02%
iter 1612: loss 1.8421, time 44.63ms, mfu 0.02%
iter 1613: loss 1.6999, time 71.46ms, mfu 0.02%
iter 1614: loss 1.8139, time 41.54ms, mfu 0.02%
iter 1615: loss 1.8105, time 57.17ms, mfu 0.02%
iter 1616: loss 1.8139, time 73.40ms, mfu 0.02%
iter 1617: loss 1.7697, time 63.76ms, mfu 0.02%
iter 1618: loss 1.7595, time 65.55ms, mfu 0.02%
iter 1619: loss 1.7615, time 75.38ms, mfu 0.02%
iter 1620: loss 1.8048, time 75.45ms, mfu 0.02%
iter 1621: loss 1.8572, time 149.54ms, mfu 0.02%
iter 1622: loss 1.8292, time 180.57ms, mfu 0.02%
iter 1623: loss 1.9528, time 97.73ms, mfu 0.02%
iter 1624: loss 1.7806, time 138.08ms, mfu 0.02%
iter 1625: loss 1.7796, time 151.54ms, mfu 0.02%
iter 1626: loss 1.8230, time 145.01ms, mfu 0.01%
iter 1627: loss 1.8743, time 208.54ms, mfu 0.01%
iter 1628: loss 1.7519, time 608.56ms, mfu 0.01%
iter 1629: loss 1.7744, time 563.32ms, mfu 0.01%
iter 1630: loss 1.7679, time 214.27ms, mfu 0.01%
iter 1631: loss 1.8786, time 240.95ms, mfu 0.01%
iter 1632: loss 1.7891, time 309.34ms, mfu 0.01%
iter 1633: loss 1.7429, time 219.27ms, mfu 0.01%
iter 1634: loss 1.7381, time 176.03ms, mfu 0.01%
iter 1635: loss 1.6885, time 164.34ms, mfu 0.01%
iter 1636: loss 1.8223, time 68.74ms, mfu 0.01%
iter 1637: loss 1.8580, time 48.04ms, mfu 0.01%
iter 1638: loss 1.8411, time 66.30ms, mfu 0.01%
iter 1639: loss 1.7807, time 54.80ms, mfu 0.01%
iter 1640: loss 1.8338, time 56.23ms, mfu 0.01%
iter 1641: loss 1.6963, time 62.40ms, mfu 0.01%
iter 1642: loss 1.8068, time 112.56ms, mfu 0.01%
iter 1643: loss 1.8575, time 82.14ms, mfu 0.01%
iter 1644: loss 1.8096, time 81.21ms, mfu 0.01%
iter 1645: loss 1.8229, time 81.82ms, mfu 0.01%
iter 1646: loss 1.8699, time 45.44ms, mfu 0.02%
iter 1647: loss 1.9380, time 57.74ms, mfu 0.02%
iter 1648: loss 1.8300, time 54.44ms, mfu 0.02%
iter 1649: loss 1.7637, time 58.19ms, mfu 0.02%
iter 1650: loss 1.8783, time 52.35ms, mfu 0.02%
iter 1651: loss 1.6766, time 67.63ms, mfu 0.02%
iter 1652: loss 1.8523, time 67.46ms, mfu 0.02%
iter 1653: loss 1.9616, time 53.70ms, mfu 0.02%
iter 1654: loss 1.7503, time 59.36ms, mfu 0.02%
iter 1655: loss 1.8328, time 123.28ms, mfu 0.02%
iter 1656: loss 1.7932, time 126.99ms, mfu 0.02%
iter 1657: loss 1.8028, time 104.72ms, mfu 0.02%
iter 1658: loss 1.7720, time 102.54ms, mfu 0.02%
iter 1659: loss 1.7657, time 90.10ms, mfu 0.02%
iter 1660: loss 1.7460, time 68.01ms, mfu 0.02%
iter 1661: loss 1.8526, time 84.43ms, mfu 0.02%
iter 1662: loss 1.7599, time 68.23ms, mfu 0.02%
iter 1663: loss 1.8023, time 77.10ms, mfu 0.02%
iter 1664: loss 1.8818, time 114.94ms, mfu 0.02%
iter 1665: loss 1.7177, time 74.70ms, mfu 0.02%
iter 1666: loss 1.8367, time 896.10ms, mfu 0.01%
iter 1667: loss 1.8443, time 938.82ms, mfu 0.01%
iter 1668: loss 1.6899, time 364.19ms, mfu 0.01%
iter 1669: loss 1.6838, time 175.90ms, mfu 0.01%
iter 1670: loss 1.8474, time 131.35ms, mfu 0.01%
iter 1671: loss 1.8514, time 93.61ms, mfu 0.01%
iter 1672: loss 1.8078, time 104.49ms, mfu 0.01%
iter 1673: loss 1.8371, time 126.52ms, mfu 0.01%
iter 1674: loss 1.8086, time 109.58ms, mfu 0.01%
iter 1675: loss 1.8085, time 108.37ms, mfu 0.01%
iter 1676: loss 1.7926, time 98.73ms, mfu 0.01%
iter 1677: loss 1.8536, time 95.02ms, mfu 0.01%
iter 1678: loss 1.7910, time 116.22ms, mfu 0.01%
iter 1679: loss 1.8483, time 101.31ms, mfu 0.01%
iter 1680: loss 1.8388, time 70.63ms, mfu 0.01%
iter 1681: loss 1.8637, time 63.25ms, mfu 0.01%
iter 1682: loss 1.7223, time 79.32ms, mfu 0.01%
iter 1683: loss 1.8121, time 114.68ms, mfu 0.01%
iter 1684: loss 1.8058, time 98.65ms, mfu 0.01%
iter 1685: loss 1.7476, time 92.47ms, mfu 0.01%
iter 1686: loss 1.8607, time 86.44ms, mfu 0.01%
iter 1687: loss 1.7972, time 68.60ms, mfu 0.01%
iter 1688: loss 1.7813, time 59.58ms, mfu 0.01%
iter 1689: loss 1.6947, time 59.80ms, mfu 0.02%
iter 1690: loss 1.8558, time 64.57ms, mfu 0.02%
iter 1691: loss 1.8517, time 63.34ms, mfu 0.02%
iter 1692: loss 1.9158, time 42.16ms, mfu 0.02%
iter 1693: loss 1.8096, time 66.04ms, mfu 0.02%
iter 1694: loss 1.8569, time 85.06ms, mfu 0.02%
iter 1695: loss 1.7337, time 66.12ms, mfu 0.02%
iter 1696: loss 1.7687, time 71.32ms, mfu 0.02%
iter 1697: loss 1.7989, time 79.13ms, mfu 0.02%
iter 1698: loss 1.8207, time 72.98ms, mfu 0.02%
iter 1699: loss 1.8542, time 65.18ms, mfu 0.02%
iter 1700: loss 1.7163, time 55.60ms, mfu 0.02%
iter 1701: loss 1.7893, time 70.82ms, mfu 0.02%
iter 1702: loss 1.8696, time 75.78ms, mfu 0.02%
iter 1703: loss 1.8218, time 65.83ms, mfu 0.02%
iter 1704: loss 1.5975, time 90.74ms, mfu 0.02%
iter 1705: loss 1.8014, time 238.29ms, mfu 0.02%
iter 1706: loss 1.7459, time 55.68ms, mfu 0.02%
iter 1707: loss 1.8655, time 79.84ms, mfu 0.02%
iter 1708: loss 1.8093, time 72.12ms, mfu 0.02%
iter 1709: loss 1.7775, time 63.31ms, mfu 0.02%
iter 1710: loss 1.8155, time 107.09ms, mfu 0.02%
iter 1711: loss 1.8890, time 161.77ms, mfu 0.02%
iter 1712: loss 1.9253, time 268.31ms, mfu 0.01%
iter 1713: loss 1.8318, time 394.71ms, mfu 0.01%
iter 1714: loss 1.9152, time 288.36ms, mfu 0.01%
iter 1715: loss 1.8138, time 122.33ms, mfu 0.01%
iter 1716: loss 1.8018, time 207.15ms, mfu 0.01%
iter 1717: loss 1.8206, time 241.07ms, mfu 0.01%
iter 1718: loss 1.7672, time 250.70ms, mfu 0.01%
iter 1719: loss 1.8134, time 118.29ms, mfu 0.01%
iter 1720: loss 1.8219, time 90.59ms, mfu 0.01%
iter 1721: loss 1.8645, time 176.98ms, mfu 0.01%
iter 1722: loss 1.8257, time 193.99ms, mfu 0.01%
iter 1723: loss 1.8010, time 98.91ms, mfu 0.01%
iter 1724: loss 1.7704, time 73.81ms, mfu 0.01%
iter 1725: loss 1.7934, time 85.39ms, mfu 0.01%
iter 1726: loss 1.7738, time 127.76ms, mfu 0.01%
iter 1727: loss 1.8267, time 118.42ms, mfu 0.01%
iter 1728: loss 1.8406, time 115.42ms, mfu 0.01%
iter 1729: loss 1.9355, time 81.96ms, mfu 0.01%
iter 1730: loss 1.8010, time 73.98ms, mfu 0.01%
iter 1731: loss 1.7289, time 64.39ms, mfu 0.01%
iter 1732: loss 1.6900, time 66.31ms, mfu 0.01%
iter 1733: loss 1.8331, time 64.91ms, mfu 0.01%
iter 1734: loss 1.7582, time 47.17ms, mfu 0.02%
iter 1735: loss 1.7563, time 42.41ms, mfu 0.02%
iter 1736: loss 1.8971, time 39.42ms, mfu 0.02%
iter 1737: loss 1.7855, time 62.90ms, mfu 0.02%
iter 1738: loss 1.7462, time 91.73ms, mfu 0.02%
iter 1739: loss 1.7415, time 166.45ms, mfu 0.02%
iter 1740: loss 1.7515, time 132.07ms, mfu 0.02%
iter 1741: loss 1.7446, time 168.81ms, mfu 0.02%
iter 1742: loss 1.8519, time 164.98ms, mfu 0.01%
iter 1743: loss 1.8005, time 345.96ms, mfu 0.01%
iter 1744: loss 1.7608, time 403.03ms, mfu 0.01%
iter 1745: loss 1.7233, time 417.99ms, mfu 0.01%
iter 1746: loss 1.7781, time 709.19ms, mfu 0.01%
iter 1747: loss 1.6983, time 207.68ms, mfu 0.01%
iter 1748: loss 1.8040, time 184.44ms, mfu 0.01%
iter 1749: loss 1.7471, time 76.49ms, mfu 0.01%
step 1750: train loss 1.7790, val loss 1.8921
saving checkpoint to out-shakespeare-char
iter 1750: loss 1.8041, time 926.42ms, mfu 0.01%
iter 1751: loss 1.9559, time 47.23ms, mfu 0.01%
iter 1752: loss 1.8112, time 68.45ms, mfu 0.01%
iter 1753: loss 1.6835, time 75.33ms, mfu 0.01%
iter 1754: loss 1.8415, time 70.96ms, mfu 0.01%
iter 1755: loss 1.8036, time 55.83ms, mfu 0.01%
iter 1756: loss 1.7677, time 63.66ms, mfu 0.01%
iter 1757: loss 1.7190, time 77.82ms, mfu 0.01%
iter 1758: loss 1.7417, time 69.48ms, mfu 0.02%
iter 1759: loss 1.7732, time 78.76ms, mfu 0.02%
iter 1760: loss 1.6994, time 45.30ms, mfu 0.02%
iter 1761: loss 1.7389, time 55.01ms, mfu 0.02%
iter 1762: loss 1.7722, time 165.05ms, mfu 0.02%
iter 1763: loss 1.8943, time 99.05ms, mfu 0.02%
iter 1764: loss 1.8158, time 60.24ms, mfu 0.02%
iter 1765: loss 1.7328, time 40.64ms, mfu 0.02%
iter 1766: loss 1.8004, time 47.60ms, mfu 0.02%
iter 1767: loss 1.8424, time 57.24ms, mfu 0.02%
iter 1768: loss 1.8857, time 53.18ms, mfu 0.02%
iter 1769: loss 1.7644, time 67.42ms, mfu 0.02%
iter 1770: loss 1.6505, time 47.98ms, mfu 0.02%
iter 1771: loss 1.7699, time 57.52ms, mfu 0.02%
iter 1772: loss 1.7587, time 77.62ms, mfu 0.02%
iter 1773: loss 1.8406, time 64.73ms, mfu 0.02%
iter 1774: loss 1.7990, time 45.49ms, mfu 0.02%
iter 1775: loss 1.7889, time 37.26ms, mfu 0.02%
iter 1776: loss 1.6292, time 30.20ms, mfu 0.02%
iter 1777: loss 1.8156, time 37.59ms, mfu 0.03%
iter 1778: loss 1.7504, time 106.21ms, mfu 0.02%
iter 1779: loss 1.9579, time 111.45ms, mfu 0.02%
iter 1780: loss 1.7315, time 48.26ms, mfu 0.02%
iter 1781: loss 1.8403, time 27.91ms, mfu 0.03%
iter 1782: loss 1.9166, time 39.70ms, mfu 0.03%
iter 1783: loss 1.7503, time 25.83ms, mfu 0.03%
iter 1784: loss 1.8315, time 35.14ms, mfu 0.03%
iter 1785: loss 1.7967, time 39.21ms, mfu 0.03%
iter 1786: loss 1.7845, time 69.15ms, mfu 0.03%
iter 1787: loss 1.8225, time 78.39ms, mfu 0.03%
iter 1788: loss 1.7811, time 54.73ms, mfu 0.03%
iter 1789: loss 1.7647, time 60.67ms, mfu 0.03%
iter 1790: loss 1.7741, time 48.28ms, mfu 0.03%
iter 1791: loss 1.7642, time 34.89ms, mfu 0.03%
iter 1792: loss 1.8247, time 44.37ms, mfu 0.03%
iter 1793: loss 1.8432, time 52.83ms, mfu 0.03%
iter 1794: loss 1.7787, time 44.60ms, mfu 0.03%
iter 1795: loss 1.7685, time 39.59ms, mfu 0.03%
iter 1796: loss 1.8958, time 45.96ms, mfu 0.03%
iter 1797: loss 1.7584, time 41.48ms, mfu 0.03%
iter 1798: loss 1.7609, time 85.95ms, mfu 0.03%
iter 1799: loss 1.7328, time 53.03ms, mfu 0.03%
iter 1800: loss 1.8415, time 37.94ms, mfu 0.03%
iter 1801: loss 1.8189, time 38.00ms, mfu 0.03%
iter 1802: loss 1.8043, time 43.28ms, mfu 0.03%
iter 1803: loss 1.6580, time 31.45ms, mfu 0.03%
iter 1804: loss 1.8217, time 45.34ms, mfu 0.03%
iter 1805: loss 1.7127, time 39.54ms, mfu 0.03%
iter 1806: loss 1.7373, time 47.49ms, mfu 0.03%
iter 1807: loss 1.8110, time 33.25ms, mfu 0.03%
iter 1808: loss 1.8539, time 40.13ms, mfu 0.03%
iter 1809: loss 1.8770, time 66.40ms, mfu 0.03%
iter 1810: loss 1.8603, time 32.13ms, mfu 0.03%
iter 1811: loss 1.7364, time 41.37ms, mfu 0.03%
iter 1812: loss 1.7575, time 29.68ms, mfu 0.03%
iter 1813: loss 1.8114, time 31.61ms, mfu 0.03%
iter 1814: loss 1.8269, time 24.11ms, mfu 0.03%
iter 1815: loss 1.8794, time 46.65ms, mfu 0.03%
iter 1816: loss 1.6855, time 35.60ms, mfu 0.03%
iter 1817: loss 1.7748, time 37.56ms, mfu 0.03%
iter 1818: loss 1.8222, time 31.83ms, mfu 0.03%
iter 1819: loss 1.6791, time 30.60ms, mfu 0.04%
iter 1820: loss 1.6907, time 40.73ms, mfu 0.03%
iter 1821: loss 1.7013, time 52.16ms, mfu 0.03%
iter 1822: loss 1.7329, time 61.01ms, mfu 0.03%
iter 1823: loss 1.7331, time 55.43ms, mfu 0.03%
iter 1824: loss 1.6313, time 38.61ms, mfu 0.03%
iter 1825: loss 1.6895, time 34.60ms, mfu 0.03%
iter 1826: loss 1.7253, time 52.57ms, mfu 0.03%
iter 1827: loss 1.7949, time 46.88ms, mfu 0.03%
iter 1828: loss 1.7864, time 34.64ms, mfu 0.03%
iter 1829: loss 1.9065, time 38.40ms, mfu 0.03%
iter 1830: loss 1.7614, time 48.96ms, mfu 0.03%
iter 1831: loss 1.8434, time 64.04ms, mfu 0.03%
iter 1832: loss 1.7082, time 51.17ms, mfu 0.03%
iter 1833: loss 1.8636, time 40.55ms, mfu 0.03%
iter 1834: loss 1.8524, time 32.28ms, mfu 0.03%
iter 1835: loss 1.8769, time 54.77ms, mfu 0.03%
iter 1836: loss 1.7676, time 57.54ms, mfu 0.03%
iter 1837: loss 1.8077, time 53.81ms, mfu 0.03%
iter 1838: loss 1.6216, time 45.89ms, mfu 0.03%
iter 1839: loss 1.7999, time 38.73ms, mfu 0.03%
iter 1840: loss 1.8017, time 39.10ms, mfu 0.03%
iter 1841: loss 1.7604, time 44.23ms, mfu 0.03%
iter 1842: loss 1.7712, time 37.56ms, mfu 0.03%
iter 1843: loss 1.8923, time 35.35ms, mfu 0.03%
iter 1844: loss 1.7598, time 44.90ms, mfu 0.03%
iter 1845: loss 1.8535, time 44.89ms, mfu 0.03%
iter 1846: loss 1.7269, time 55.11ms, mfu 0.03%
iter 1847: loss 1.6867, time 43.76ms, mfu 0.03%
iter 1848: loss 1.7332, time 42.82ms, mfu 0.03%
iter 1849: loss 1.8958, time 35.91ms, mfu 0.03%
iter 1850: loss 1.7283, time 25.72ms, mfu 0.03%
iter 1851: loss 1.7362, time 40.83ms, mfu 0.03%
iter 1852: loss 1.7231, time 36.68ms, mfu 0.03%
iter 1853: loss 1.8270, time 57.82ms, mfu 0.03%
iter 1854: loss 1.6889, time 32.45ms, mfu 0.03%
iter 1855: loss 1.6886, time 25.85ms, mfu 0.03%
iter 1856: loss 1.8231, time 39.82ms, mfu 0.03%
iter 1857: loss 1.8388, time 35.77ms, mfu 0.03%
iter 1858: loss 1.9075, time 28.16ms, mfu 0.03%
iter 1859: loss 1.7361, time 49.55ms, mfu 0.03%
iter 1860: loss 1.6949, time 31.12ms, mfu 0.03%
iter 1861: loss 1.6415, time 43.68ms, mfu 0.03%
iter 1862: loss 1.7298, time 38.23ms, mfu 0.03%
iter 1863: loss 1.7108, time 35.98ms, mfu 0.03%
iter 1864: loss 1.7105, time 36.03ms, mfu 0.03%
iter 1865: loss 1.8216, time 30.07ms, mfu 0.04%
iter 1866: loss 1.7632, time 27.60ms, mfu 0.04%
iter 1867: loss 1.7425, time 34.75ms, mfu 0.04%
iter 1868: loss 1.7308, time 31.29ms, mfu 0.04%
iter 1869: loss 1.8061, time 36.30ms, mfu 0.04%
iter 1870: loss 1.6711, time 61.04ms, mfu 0.03%
iter 1871: loss 1.7664, time 49.56ms, mfu 0.03%
iter 1872: loss 1.7578, time 52.85ms, mfu 0.03%
iter 1873: loss 1.7199, time 39.01ms, mfu 0.03%
iter 1874: loss 1.7120, time 59.83ms, mfu 0.03%
iter 1875: loss 1.6852, time 38.58ms, mfu 0.03%
iter 1876: loss 1.7393, time 38.81ms, mfu 0.03%
iter 1877: loss 1.7477, time 38.75ms, mfu 0.03%
iter 1878: loss 1.6768, time 46.26ms, mfu 0.03%
iter 1879: loss 1.7723, time 52.18ms, mfu 0.03%
iter 1880: loss 1.7548, time 54.79ms, mfu 0.03%
iter 1881: loss 1.7746, time 37.71ms, mfu 0.03%
iter 1882: loss 1.8441, time 43.82ms, mfu 0.03%
iter 1883: loss 1.6335, time 50.31ms, mfu 0.03%
iter 1884: loss 1.7074, time 57.01ms, mfu 0.03%
iter 1885: loss 1.7975, time 61.14ms, mfu 0.03%
iter 1886: loss 1.7024, time 61.63ms, mfu 0.03%
iter 1887: loss 1.8263, time 46.89ms, mfu 0.03%
iter 1888: loss 1.7169, time 63.42ms, mfu 0.03%
iter 1889: loss 1.7219, time 251.40ms, mfu 0.02%
iter 1890: loss 1.6987, time 141.98ms, mfu 0.02%
iter 1891: loss 1.7109, time 72.70ms, mfu 0.02%
iter 1892: loss 1.6612, time 79.83ms, mfu 0.02%
iter 1893: loss 1.7451, time 55.98ms, mfu 0.02%
iter 1894: loss 1.7400, time 44.79ms, mfu 0.02%
iter 1895: loss 1.7499, time 61.05ms, mfu 0.02%
iter 1896: loss 1.7203, time 33.96ms, mfu 0.02%
iter 1897: loss 1.7030, time 38.61ms, mfu 0.02%
iter 1898: loss 1.7733, time 42.52ms, mfu 0.03%
iter 1899: loss 1.7188, time 36.67ms, mfu 0.03%
iter 1900: loss 1.6849, time 42.54ms, mfu 0.03%
iter 1901: loss 1.7666, time 49.27ms, mfu 0.03%
iter 1902: loss 1.8024, time 40.79ms, mfu 0.03%
iter 1903: loss 1.8703, time 55.34ms, mfu 0.03%
iter 1904: loss 1.7309, time 52.07ms, mfu 0.03%
iter 1905: loss 1.7496, time 51.50ms, mfu 0.03%
iter 1906: loss 1.6487, time 79.82ms, mfu 0.03%
iter 1907: loss 1.8226, time 85.65ms, mfu 0.02%
iter 1908: loss 1.7343, time 70.76ms, mfu 0.02%
iter 1909: loss 1.8662, time 51.10ms, mfu 0.02%
iter 1910: loss 1.6221, time 38.95ms, mfu 0.02%
iter 1911: loss 1.7553, time 51.42ms, mfu 0.02%
iter 1912: loss 1.7277, time 48.72ms, mfu 0.02%
iter 1913: loss 1.7665, time 58.42ms, mfu 0.02%
iter 1914: loss 1.7239, time 32.92ms, mfu 0.03%
iter 1915: loss 1.8489, time 48.41ms, mfu 0.03%
iter 1916: loss 1.6891, time 40.44ms, mfu 0.03%
iter 1917: loss 1.7314, time 50.85ms, mfu 0.03%
iter 1918: loss 1.7947, time 47.85ms, mfu 0.03%
iter 1919: loss 1.6894, time 67.81ms, mfu 0.03%
iter 1920: loss 1.7789, time 64.92ms, mfu 0.02%
iter 1921: loss 1.7977, time 75.83ms, mfu 0.02%
iter 1922: loss 1.7670, time 78.61ms, mfu 0.02%
iter 1923: loss 1.7595, time 66.98ms, mfu 0.02%
iter 1924: loss 1.7274, time 105.16ms, mfu 0.02%
iter 1925: loss 1.7422, time 71.60ms, mfu 0.02%
iter 1926: loss 1.8686, time 64.17ms, mfu 0.02%
iter 1927: loss 1.9504, time 69.08ms, mfu 0.02%
iter 1928: loss 1.6605, time 39.50ms, mfu 0.02%
iter 1929: loss 1.7044, time 40.37ms, mfu 0.02%
iter 1930: loss 1.8036, time 50.72ms, mfu 0.02%
iter 1931: loss 1.7936, time 34.08ms, mfu 0.02%
iter 1932: loss 1.8269, time 48.92ms, mfu 0.02%
iter 1933: loss 1.7727, time 46.47ms, mfu 0.03%
iter 1934: loss 1.8015, time 58.80ms, mfu 0.02%
iter 1935: loss 1.7271, time 47.33ms, mfu 0.02%
iter 1936: loss 1.6927, time 54.55ms, mfu 0.02%
iter 1937: loss 1.8060, time 37.41ms, mfu 0.03%
iter 1938: loss 1.7350, time 44.48ms, mfu 0.03%
iter 1939: loss 1.7386, time 66.76ms, mfu 0.03%
iter 1940: loss 1.7062, time 56.56ms, mfu 0.03%
iter 1941: loss 1.6497, time 73.31ms, mfu 0.02%
iter 1942: loss 1.7515, time 76.60ms, mfu 0.02%
iter 1943: loss 1.7952, time 55.89ms, mfu 0.02%
iter 1944: loss 1.8112, time 46.43ms, mfu 0.02%
iter 1945: loss 1.7891, time 56.56ms, mfu 0.02%
iter 1946: loss 1.6995, time 48.52ms, mfu 0.02%
iter 1947: loss 1.7981, time 42.33ms, mfu 0.02%
iter 1948: loss 1.8106, time 53.29ms, mfu 0.02%
iter 1949: loss 1.6782, time 36.80ms, mfu 0.03%
iter 1950: loss 1.7474, time 52.79ms, mfu 0.03%
iter 1951: loss 1.8605, time 56.88ms, mfu 0.03%
iter 1952: loss 1.6390, time 33.44ms, mfu 0.03%
iter 1953: loss 1.8531, time 28.14ms, mfu 0.03%
iter 1954: loss 1.7255, time 31.82ms, mfu 0.03%
iter 1955: loss 1.8938, time 52.76ms, mfu 0.03%
iter 1956: loss 1.7512, time 47.52ms, mfu 0.03%
iter 1957: loss 1.7251, time 96.90ms, mfu 0.03%
iter 1958: loss 1.8246, time 79.00ms, mfu 0.03%
iter 1959: loss 1.7809, time 46.09ms, mfu 0.03%
iter 1960: loss 1.7511, time 49.00ms, mfu 0.03%
iter 1961: loss 1.7355, time 49.39ms, mfu 0.03%
iter 1962: loss 1.8752, time 57.41ms, mfu 0.03%
iter 1963: loss 1.6606, time 87.45ms, mfu 0.02%
iter 1964: loss 1.7129, time 48.26ms, mfu 0.02%
iter 1965: loss 1.8167, time 86.06ms, mfu 0.02%
iter 1966: loss 1.8038, time 54.41ms, mfu 0.02%
iter 1967: loss 1.9572, time 70.03ms, mfu 0.02%
iter 1968: loss 1.6740, time 53.85ms, mfu 0.02%
iter 1969: loss 1.7750, time 34.60ms, mfu 0.02%
iter 1970: loss 1.8327, time 43.59ms, mfu 0.03%
iter 1971: loss 1.7521, time 30.32ms, mfu 0.03%
iter 1972: loss 1.7338, time 40.82ms, mfu 0.03%
iter 1973: loss 1.7306, time 38.65ms, mfu 0.03%
iter 1974: loss 1.6931, time 55.24ms, mfu 0.03%
iter 1975: loss 1.7289, time 44.37ms, mfu 0.03%
iter 1976: loss 1.6929, time 38.91ms, mfu 0.03%
iter 1977: loss 1.8865, time 122.08ms, mfu 0.03%
iter 1978: loss 1.7603, time 29.94ms, mfu 0.03%
iter 1979: loss 1.7862, time 31.22ms, mfu 0.03%
iter 1980: loss 1.7613, time 31.17ms, mfu 0.03%
iter 1981: loss 1.7781, time 45.00ms, mfu 0.03%
iter 1982: loss 1.7722, time 58.90ms, mfu 0.03%
iter 1983: loss 1.7677, time 68.73ms, mfu 0.03%
iter 1984: loss 1.6993, time 49.81ms, mfu 0.03%
iter 1985: loss 1.7417, time 40.69ms, mfu 0.03%
iter 1986: loss 1.6848, time 40.62ms, mfu 0.03%
iter 1987: loss 1.7071, time 42.58ms, mfu 0.03%
iter 1988: loss 1.7093, time 37.06ms, mfu 0.03%
iter 1989: loss 1.6852, time 37.36ms, mfu 0.03%
iter 1990: loss 1.7025, time 36.80ms, mfu 0.03%
iter 1991: loss 1.7723, time 26.70ms, mfu 0.03%
iter 1992: loss 1.6626, time 42.91ms, mfu 0.03%
iter 1993: loss 1.8213, time 31.67ms, mfu 0.03%
iter 1994: loss 1.7564, time 101.68ms, mfu 0.03%
iter 1995: loss 1.7688, time 72.11ms, mfu 0.03%
iter 1996: loss 1.8214, time 55.39ms, mfu 0.03%
iter 1997: loss 1.7120, time 48.63ms, mfu 0.03%
iter 1998: loss 1.8759, time 39.14ms, mfu 0.03%
iter 1999: loss 1.9002, time 57.71ms, mfu 0.03%
step 2000: train loss 1.7648, val loss 1.8857
saving checkpoint to out-shakespeare-char
iter 2000: loss 1.6958, time 801.37ms, mfu 0.03%
