Absolutely.

#1 Predicting Synthetic Outputs: We’ve done this a few times, and typically coming up with and generating the synthetic prediction task takes <1 day. The time it takes to get a model to fit a synthetic task is more variable, though in the described case of predicting std(daytime HRs) — std(nighttime HRs), the model we were trying out was able to predict this task with low MAE.

#2 Visualizing activations: One of our interns built the weight extraction, storing, and visualization tool in about one month. Once he noticed dead neurons, he replaced ReLUs with Leaky ReLUs in <1 week.

#3 Gradient analysis: It’s hard to say, as the problem resolved itself in a later model that included many changes.

#4 Loss analysis: We usually spend a few days looking through losses and validating/invalidating hypotheses. Though we do not do this for all models we train; just those that are likely to launch or are markedly different (eg new data source or new prediction task)