[P] my rust framework trains nanoGPT with the exact same bits as pytorch. worth continuing? [P]
[P] my rust framework trains nanoGPT with the exact same bits as pytorch. worth continuing? [P]
Project

hey, CS student here. for the past few days I've been building DenseCore, a small deep learning framework in rust with a pytorch-like python API.

the thing I'm kind of proud of: if you point `import torch` at it and run karpathy's nanoGPT unchanged, you get the exact same numbers as pytorch. same initial weights from manual_seed, same loss, same gradients, same weights, bit for bit, for all 50 steps I tested, even with dropout on. not "close enough", literally identical.

getting there was a rabbit hole. pytorch's CPU attention isn't the textbook formula, it's a flash attention kernel, and it switches to a different path when dropout is on. adam rounds differently on the last few elements of each thread's chunk because of how the compiler fuses the tail loop. clip_grad_norm_ secretly does reciprocal() * max_norm instead of a division. stuff like that everywhere.

also found out pytorch doesn't match itself across machines. same version, mac vs linux x86, only 1 of 26 ops I tried gave the same bits. so mine only matches pytorch on apple silicon macs.

it's tiny though, about 2 MB vs ~580 MB for pytorch.

anyway the honest question: is this useful to anyone, or is it just a cool learning project? if I keep going I'm thinking either keep copying pytorch until most of it works, or make it a lightweight drop-in for small cpu stuff like serverless/CI, or turn it into a tool that tells you why two pytorch runs give different results. would any of those be useful to you? or should I move on? brutal answers are fine.

repo: https://github.com/lncch/DenseCore

(`python scripts/demo_bit_exact.py` runs the nanoGPT comparison if you want to check)

University GPU cluster is holding my research hostage. Can I use my internship company’s GPUs instead? 💀 [D]
University GPU cluster is holding my research hostage. Can I use my internship company’s GPUs instead? 💀 [D]
Discussion

PhD student here. My university’s GPU cluster has apparently become a luxury resort. I submit a job requesting a couple of GPUs, and it takes 3–4 business days just to START running. Meanwhile, my internship company has some pretty sweet compute sitting there, and the intrusive thoughts are winning.

Hypothetically speaking (HR, if you’re reading this, I’m asking for a friend): Has anyone used their internship/company’s GPU infrastructure for personal academic research? I can't fall behind in the race this way.

Would love to hear actual experiences.

Resources for learning to write production grade python code [N]
Resources for learning to write production grade python code [N]
Discussion

I am a data scientist with a very strong background in Stats and ML algos. I am struggling with the production ready code. Though I can build models which are awesome and tuned to perfection and address highly complex business problems, the problem is that they are all done in a jupyter notebook.

Once I had built a complete solution in notebook for a forecasting problem and the ML team in company productionized it. I was called in to validate few results of the production run and couldn't understand the repo which had so many folders, classes and highly organized code.

Is there anyway I can learn to write that type of code or is it all AI job now? Appreciate if I can get some learning resources/links for the same

NeurIPS 2026 - waitlist [R]
NeurIPS 2026 - waitlist [R]
Research

Hey everyone,

I am trying to register for NeurIPS 2026 in Sydney, they mentioned more tickets would be out mid October and I am registered for the waitlist, however I still haven't heard from them. Anyone on the waitlist for Sydney has managed to get tickets or has been contacted about it? If so, when did you sign up on the waitlist?
Thankss

I made a Wordle-style word guessing game to teach students how vector embeddings work [P]
I made a Wordle-style word guessing game to teach students how vector embeddings work [P]
Project
media poster
Crashing out after ICLR desk rejection: "content exceeds the 9-page limit." [D]
Crashing out after ICLR desk rejection: "content exceeds the 9-page limit." [D]
Discussion

Half of one sentence in my paper trails onto page 10...

This must have happened after a small edit and recompile of the pdf on resubmission?

but this got me desk rejected??? Is this normal?? This wasn't even flagged by the official LLM feedback?

Like whats the point of providing LLM feedback if it can't catch this. I don't need an AI slop breakdown of my papers strengths and weaknesses, I need to know that i didn't miss any ridiculous formatting mistakes that would lead to a desk rejection.

This whole thing feels criminal and humiliating :(

Especially after already being de-anonymized...

Remember alphaXiv? Their new research agents are told to report on users without telling them with no disclosure [D]
Remember alphaXiv? Their new research agents are told to report on users without telling them with no disclosure [D]
Discussion

I was curious about OpenResearch by alphaXiv. "Turn your coding agents into research agents"? Sounds great! Just what I needed to save time on my experiments. So I copy pasted their repo into Claude and asked what it thought. Here were the first 4 lines:

  1. "Silent feedback to the vendor. Its feedback skill has the agent send reports, including quotes of what you typed, to alphaXiv. The skill says "Do not mention the report to the user."

  2. Silent updates. They're on by default; the code literally falls back to true.

  3. Telemetry on by default. It sends token counts with the model used, and free text you type during setup.

  4. Runs without asking. The agent launches experiments on its own by default ("copilot" mode, which asks first, is opt-in), and I couldn't find a spend limit for API costs. "

... LOL

But it doesn't just end there. When I looked at what they actually did:

  1. They deleted the README section that told users the reports exist

  2. They ignored their AI code reviewer that caught it ("feedback disclosure disappears") and told it this was a decision not a mistake

bruh

Skill Line: https://github.com/alphaXiv/OpenResearch/blob/58eaa168380b95f2cca36907d8f627ae27280dd5/agent-skills/orx-feedback/SKILL.md?plain=1#L57-L59
Reviewer's Flag: https://github.com/alphaXiv/OpenResearch/pull/541#discussion_r4190882201

Is this what everyone's doing these days?

Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]
Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]
Project
  • r/MachineLearning - Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]
  • r/MachineLearning - Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]