#1 Predicting Synthetic Outputs: We’ve done this a few times, and typically coming up with and generating the synthetic prediction task takes <1 day. The time it takes to get a model to fit a synthetic task is more variable, though in the described case of predicting std(daytime HRs) — std(nighttime HRs), the model we were trying out was able to predict this task with low MAE.

#2 Visualizing activations: One of our interns built the weight extraction, storing, and visualization tool in about one month. Once he noticed dead neurons, he replaced ReLUs with Leaky ReLUs in <1 week.

#3 Gradient analysis: It’s hard to say, as the problem resolved itself in a later model that included many changes.

#4 Loss analysis: We usually spend a few days looking through losses and validating/invalidating hypotheses. Though we do not do this for all models we train; just those that are likely to launch or are markedly different (eg new data source or new prediction task)

Software Engineer and Runner

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store