Optimization problems in machine learning have aspects that make them more challenging than the traditional settings, like stochasticity, and parameters with side-effects (e.g., the batch size and structure). The field has invented many different approaches to deal with these demands. Unfortunately - and intriguingly - this extra functionality seems to invariably necessitate the introduction of tuning parameters: step sizes, decay rates, cycle lengths, batch sampling distributions, and so on. Such parameters are not present, or at least not as prominent, in classic optimization methods. But getting them right is frequently crucial, and necessitates inconvenient human “babysitting”.
Recent work has increasingly tried to eliminate such fiddle factors, typically by statistical estimation. This also includes automatic selection of external parameters like the batch-size or -structure, which have not traditionally been treated as part of the optimization task. Several different strategies have now been proposed, but they are not always compatible with each other, and lack a common framework that would foster both conceptual and algorithmic interoperability. This workshop aims to provide a forum for the nascent community studying automating parameter-tuning in optimization routines.
Among the questions to be addressed by the workshop are:
The workshop is organized by Maren Mahsereci, Alex Davies and Philipp Hennig.
The workshop will be held on Saturday, 10 December, in Area 2
Time | Event | Material | |
---|---|---|---|
09:00-09:10 | — | Opening Remarks | |
09:10-09:30 | — | Matt Hoffman (DeepMind) | |
09:30-10:00 | — | David Duvenaud (U Toronto) | slides |
10:00-10:30 | — | Stephen J Wright (U of Wisconsin) | slides |
10.30-11.00 | — | Coffee Break | |
11:00-11:30 | — | Samantha Hansen (Spotify) | slides |
11:30-12:00 | — | Spotlights | (see below) |
12:00-12:45 | — | Poster Session | |
12:45-14:15 | — | Lunch Break | |
14:15-14:40 | — | Matteo Pirotta (Politecnico di Milano) | |
14:40-15:00 | — | Ameet Talwalkar (UCLA) | slides |
15:00-15:30 | — | Coffee Break | |
15:30-15:50 | — | Ali Rahimi (Google) | |
15:50-16.20 | — | Mark Schmidt (UBC) | |
16:20-17:00 | — | Panel Discussion |
(in alphabetical order, by first author’s surname)
We welcome contributions from theoretical treatments, empirical studies, and applications, of the above. The list is not exhaustive, and we also welcome submissions that draw upon highly related topics.
Submissions should be in the (new!) NIPS 2016 format, with a maximum of 4 pages (excluding references). Accepted papers will be made available online at the workshop website, and will be presented in a spotlight talk at the workshop itself, but the workshop proceedings can be considered non-archival. Explicitly: shorter versions of relevant papers submitted or published elsewhere are encouraged. Submissions need not be anonymous.
Please mail pdf submissions to mmahsereci@tue.mpg.de.