More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
Good research starts with picking your own problems, not the ones everyone else is chasing. Instead of absorbing topics from advisors or trending papers, decide what outcome you actually want and work backward to the experiments. That approach forces originality—John Schulman contrasts it with endlessly tweaking existing literature. Meanwhile, training your “taste” like a muscle means predicting experiment results before you run them, scoring your forecasts and adjusting over time. That habit sharpens intuition faster than passive reading ever will.
Your information diet matters. Don’t just scan arXiv’s hot list or Slack chatter—you’ll end up thinking what everyone else is thinking. Hunt down older work: mixture of experts dates back to 1991, LSTMs to 1997, and Shannon’s 1952 talk on shrinking hard problems still applies today. Branch out across fields—statistics, neuroscience, mechanism design—to spot dead-end papers early and spot promising angles before benchmarks do.
Writing everything down is nonnegotiable. Paul Graham and Feynman both used writing to expose hidden assumptions and guard against self-deception. Keep a running lab log—hypothesis, setup, expectation, result, belief update—and revisit it. Public essays pull undigested ideas into view; Chris Olah and Len Carter showed that clear blog posts can shape a field more than dense conference proceedings. That public record doubles as proof of your thought process.
Finally, tighten your loop between idea and result. Tooling isn’t junior work: one-command experiment launches and instant comparisons let you iterate faster. Overfit a single batch first, as Karpathy recommends, to flush out bugs in seconds, not hours. Stare at raw outputs—read a hundred failure cases by hand and cluster them. Most ML problems hide in data, not code. Run quick-and-dirty prototypes of every idea, kill most of them early, tune baselines mercilessly and ablate until you know which part actually moves the needle. That volume of rapid feedback is how you beat slow, polite progress.
Questions about this article
No questions yet.