First tutorial session Vincent Dumoulin

January 15, 2015

Outline

1

Solution to the numpy + MNIST + MLP assignment

2

Git primer

3

Theano primer

4

Porting numpy + MNIST + MLP to theano + MNIST + MLP

Solution to numpy + MNIST + MLP Setup

x = [x1 , · · · , xA ]T ,   W1,1 · · · W1,A ··· ··· , W =  ··· WH,1 · · · WH,A   V1,1 · · · V1,H ··· , V =  ··· ··· VC,1 · · · VC,H h = σ(Wx + b),

t = [t1 , · · · , xC ]T , b = [b1 , · · · , bH ]T ,

c = [c1 , · · · , cC ]T , y = softmax(Vh + c)

L = −tT log y

Solution to numpy + MNIST + MLP Derivatives ∂L ∂V ∂L ∂c ∂L ∂W ∂L ∂b

= (y − t)hT , =y−t = [VT (y − t) h (1 − h)]xT , = VT (y − t) h (1 − h)

A complete derivation can be found here https://raw.githubusercontent.com/vdumoulin/ ift6266h15/master/assignments/01/solution.pdf

Solution to numpy + MNIST + MLP

The full python implementation can be found here https://raw.githubusercontent.com/vdumoulin/ ift6266h15/master/assignments/01/solution.py

Git primer Why version control? A nice and clean alternative to maintaining multiple versions of the same file using some sort of custom naming scheme (e.g. myfile_9.py) A way to end the fear of saving and quitting Keep a trace how your files change throughout development Revert back to older versions Manage multiple versions (branches) of your code at the same time Going further http://git-scm.com/book/en/v2/ Getting-Started-About-Version-Control

Git primer

What is git? Distributed version control system No checking out single files: local version fully mirrors the repository No central authority on what is the true codebase

Takes snapshots of the state of a repository at a given time Intelligent about not duplicating information from one snapshot to another Going further http://git-scm.com/book/en/v2/Getting-Started-Git-Basics

Git primer

What is Github? A place to host your git repositories Makes it easy to Share code with others Keep track of other people’s code Modify other people’s code (forking) Collaborate with other people on common code

Technically no different from your own machine: Both can pull and push changes Both host a fully functional version of your repository

Git primer Creating a repository on Github

Git primer Identifying the repository URL

Cloning a git repository > git clone

Git primer

Putting a file under version control Create a dummy file Check the status of the repository: > git status

Add the file to version control: > git add my_dummy_file.py

Commit the newly-added file: > git commit -m "Add new dummy file to repository"

Git primer

Commit changes to a file Change something in your file Stage the changes: > git add my_dummy_file.py

Commit the changes: > git commit -m "Add stuff to dummy file"

Git primer Pull changes on Github Run > git pull origin master

Push changes on Github Pull the latest changes from your Github repo: > git pull origin master

Push your changes to Github: > git push origin master

Git primer

Going further with Git http://git-scm.com/book/en/v2

Theano primer What is Theano? From Theano’s online documentation: Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Does symbolic computation and differentiation (i.e. the end result of differentiation is itself a symbolic expression) Very similar to numpy with respect to its interface Allows doing numerical computation in a high-level language (Python) while still retaining the speed of low-level languages (like C) Allows the generation of efficient CPU and GPU code transparently

Theano primer

Typical Theano workflow 1

Instantiate symbolic variables

2

Build a computation graph out of those variables

3

Compile a function with the symbolic variables as input and the output of the computation graph as output

4

Call the compiled function with numerical inputs

Theano primer Theano vs. numpy Theano interface is very similar to numpy interface numpy arrays are automatically converted to constant symbolic variables when used inside a computation graph You can manipulate Theano symbolic variables in the same way you’d manipulate numpy arrays Going further: Theano’s basic interface http://deeplearning.net/software/theano/ library/tensor/basic.html

Theano primer Types of symbolic variables TensorVariable Its value is unspecified at graph creation and can change from one call of the compiled function to another (e.g. x and y in y = 3x − 2). Not persistent across function calls TensorConstant Its value is specified at graph creation and does not change from one call of the compiled funtion to another (e.g. 3 and −2 in y = 3x − 2) TensorSharedVariable Its value is specified at graph creation but is bound to change from one call of the compiled function to another (e.g. a and b in y = ax + b in a regression setting where some x and y pairs have been observed). Persistent across fuction calls

Theano primer Examples Listing 1: Simple algebra import theano import theano.tensor as T # 1. Instantiate symbolic variables x = T.vector(name=’x’) y = T.vector(name=’y’) # 2. Build a computation graph z = x + y # 3. Compile a callable function f = theano.function(inputs=[x, y], outputs=z) # 4. Call the function using numerical inputs print f([1, 2], [3, 4])

Theano primer Examples Listing 2: Gradient computation import theano import theano.tensor as T # 1. Instantiate symbolic variables x = T.vector(name=’x’) # 2. Build a computation graph z = (x ** 2).sum() d_z_d_x = T.grad(z, x) # 3. Compile a callable function f = theano.function(inputs=[x], outputs=d_z_d_x) # 4. Call the function using numerical inputs print f([1, 2])

Theano primer Examples Listing 3: Linear regression import theano import theano.tensor as T x = T.scalar(name=’x’); t = T.scalar(name=’t’) a = theano.shared(-1.0, name=’a’) b = theano.shared(0.0, name=’b’) y = a * x + b mse = (y - t) ** 2 grad_a, grad_b = T.grad(mse, [a, b]) f = theano.function(inputs=[x, t], outputs=mse, updates={a: a - 0.01 * grad_a, b: b - 0.01 * grad_b}) print [f(1, 5)) for i in xrange(10)]

Theano primer

Going further: online Theano tutorial http://deeplearning.net/software/theano/ tutorial/index.html#tutorial

## here - GitHub

Jan 15, 2015 - ift6266h15/master/assignments/01/solution.pdf ... the same file using some sort of custom naming scheme (e.g. myfile_9.py). A way to end the fear of saving and quitting. Keep a trace how your files change throughout development. Revert back to ... symbolic variables when used inside a computation graph.

#### Recommend Documents

here - GitHub
Word boundary (Perl, Java, .Net) ... Group re as a unit (Perl, Java, . .... If cond then re1, else re2. re2 is optional. .... Lazy quantifiers may be slower than greedy.

Here - GitHub
Mar 29, 2017 - 5.2 Basis for creating a system of equations from a single ODE . .... (weighted 20%) and a final exam (weighted 50%). 6 .... If you submit a PDF, please also submit the source-files used to generate the ...... Source: https: // commons

here - GitHub
Sep 14, 2015 - Highlights. 1 optimizationBenchmarking tool for evaluating and comparing ...... in artificial intelligence, logic, theoretical computer science, and various application ...... can automatically be compiled to PDF [86],ifaLATEX compiler

here - GitHub
can start it for free, but at some point you need to pay to advance through (not sure of the ... R for Everyone, Jared Lander, http://www.amazon.com/Everyone-Advanced-Analytics-Graphics-Addison-Wesley/ ... ISLR%20First%20Printing.pdf.

here - GitHub
Feb 16, 2016 - 6. 2 Low Level System Information. 7. 2.1 Machine Interface . .... devspecs/abi386-4.pdf, which describes the Linux IA-32 ABI for proces- ...... rameters described for the usual personality routine below, plus an additional.

here - GitHub
Oct 13, 2015 - Thus, I will need the email address that you reliably check. The default is whatever ... as a PDF and sending them to me. You will be rewarded in ... Part of good mathematics is its beautiful presentation. Thus, there will be a 1â ..

here
Jan 1, 2015 - Adjusting the viewfinder (diopter-adjustment) [10] ...... If the product is connected to a laptop computer that is not connected to a power source, ...

here
This thesis investigates using an approach based on the Actors paradigm for ... The analysis of these comparisons indicates that for certain existing ...... practical implementation reasons and to ease reasoning about message passing. ..... return R

here
Nov 28, 2008 - dimensional data points which lie in a very high-dimensional ... high-dimensional data in artificial intelligence, pattern recognition, computer vision, data mining, ...... Isomap and HLLE is destroyed, if we project them onto the ...

here
List of Abbreviations xi ...... When the agent is in the infectious state, this contact list is used to determine ..... The meaning of the fields in elements of the distributions are as follows. ...... using multiple, which require restating the func

here
Nov 28, 2008 - Tsinghua National Laboratory for Information Science and ... Chunxia Zhang is with the Software School, School of Computer Science, Beijing Institute of Technology, Beijing 100081, ..... local coordinates after local coordinatization.

here
Jun 7, 2015 - people must be met, not the needs of political and military elites. ... We see military commanders, each with their own agendas, not necessarily ...

here
Jan 1, 2015 - If the product is connected to a laptop computer that is not ...... iPhone and iPad are trademarks of Apple Inc., registered in the U.S. and other ...

here - Juror13
Nov 12, 2014 - 2 S v Pillay1977 (4) SA 531 (A) at 534 H- 535 A; S v Fazzie and Others 1964 (4) SA 673 (A). 3 S v Nkosi and Another 2011 (2) SACR 482 SCA ...

HERE - Lawctopus
Relevant experience is essential and a degree level qualification in a law or social ... Bachelors degree + Masters in Social Science or Humanities (Law degree ...

here - Juror13
Nov 12, 2014 - The State applies for leave to appeal the sentence imposed on. the conviction of culpable ... to a different conclusion on sentence, the Court of appeal would not be entitled to ..... that a non-custodial sentence would send a wrong me

'Newporters will learn here and become champions here' - The ...
Sheldon Whitehouse, D-R. ... Sheldon Whitehouse, D-R.I., Gov. Gina ... 'Newporters will learn here and become champions here' - The Newport Daily News.pdf.

here - NFL Labor
Run Like the Wind â Fans will have the opportunity to test their speed by running alongside a 40-yard long LED screen featuring NFL stars running the 40-yard dash. â¢ AFC & NFC Row/Team Houses â All 32 NFL clubs have a home at Draft Town at thei

here
Welcomes Dan Bourke - Irish Music Masterclass. * Continues Slow sessions ... and Rant will perform at 8.30pm followed by a mighty session. Tommy Carty Slow ...

here
ARGONAUT INSURANCE COMPANY. Respondent. APPLICATION OF CALFORNA WORKERS COMPENSATION. INSTITUTE FOR STATUS AS AMICUS ...

Winter is here! - Groups
Page 1. Winter is here! Color the boy and the snowman. Can you name all the winter clothes?

here - Chris Oatley
didn't want realism in our stgle. The same for the effects design that is based on design elements we found in. Chinese art, like clouds, fire, water etc.

here
Information Technology Code Enforce TeÅ¿t Parks ReCreation Planning ... See County Organizational Operational Policy Ordinance for powers, duties and ...