1 FIG. 1: output
log-probability plot for output
Plot of log-probability vs iterations for
0.05
log-probability
0.10 0.15 0.20 0.25 0.30 0
200
400
600 Iteration
800
train exp/chain/tdnn7o_sp valid exp/chain/tdnn7o_sp
1000
2
r-dimension average-(value, derivative) and rms-oderivative percentiles for prefina
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 2: Per-dimension average-(value, derivative) and rms-oderivative percentiles for prefinal-chain.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.010 0.005
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
3
er-dimension average-(value, derivative) and rms-oderivative percentiles for prefina
Oderivative-RectifiedLinear
Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 3: Per-dimension average-(value, derivative) and rms-oderivative percentiles for prefinal-xent.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
1.0 0.5
0.004 0.002
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
4
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 4: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn1.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
1.0 0.5 0.0
0.10 0.05
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
5
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 5: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn10.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.010 0.005
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
6
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn
Oderivative-RectifiedLinear
Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 6: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn11.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.0075 0.0050 0.0025
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
7
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 7: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn2.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
1.0 0.5
0.02 0.01 0.00
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
8
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 8: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn3.relu
1.5 1.0 0.5 0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.006 0.004 0.002
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
9
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 9: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn4.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
1.00 0.75 0.50 0.25
0.02 0.01 0.00
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
10
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 10: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn5.relu
1.5 1.0 0.5 0.0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.010 0.005
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
11
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear
Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 11: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn6.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.0075 0.0050 0.0025
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
12
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear
Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 12: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn7.relu
1.5 1.0 0.5 0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.0075 0.0050 0.0025
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
13
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 13: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn8.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.010 0.005
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
14
Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdn
Oderivative-RectifiedLinear
Derivative-RectifiedLinear
Value-RectifiedLinear
FIG. 14: Per-dimension average-(value, derivative) and rms-oderivative percentiles for tdnn9.relu
1 0
0
200
400
600
800
1000
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.75 0.50 0.25
0.0075 0.0050 0.0025
[1]/tdnn7o_sp
[1]:exp/chain 5th percentile 50th percentile
95th percentile
Parameter differences at output-xent.affine
Relative Parameter Differences Parameter Differences
FIG. 15: Parameter differences at output-xent.affine
15 10 5 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.6 0.4 0.2 0.0
Parameter Differences exp/chain/tdnn7o_sp
15
16
Parameter differences at output.affine
Relative Parameter Differences
Parameter Differences
FIG. 16: Parameter differences at output.affine
8 6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.4 0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
17
Parameter differences at prefinal-chain-l
Relative Parameter Differences
Parameter Differences
FIG. 17: Parameter differences at prefinal-chain-l
4 3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.25 0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
18
Relative Parameter Differences Parameter Differences
FIG. 18: prefinal-chain.affine
Parameter differences at prefinal-chain.affine Parameter differences at
1.75 1.50 1.25 1.00 0.75
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.06 0.05 0.04 0.03 0.02
Parameter Differences exp/chain/tdnn7o_sp
19
Parameter differences at prefinal-l
Relative Parameter Differences
Parameter Differences
FIG. 19: Parameter differences at prefinal-l
4 3 2 1
0.30 0.25 0.20 0.15 0.10 0.05
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
Parameter Differences exp/chain/tdnn7o_sp
20
Parameter differences at prefinal-xent-l
Relative Parameter Differences
Parameter Differences
FIG. 20: Parameter differences at prefinal-xent-l
4 3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.25 0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
21
Relative Parameter Differences Parameter Differences
FIG. 21: prefinal-xent.affine
Parameter differences at prefinal-xent.affine Parameter differences at
1.5 1.0 0.5
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.04 0.03 0.02
Parameter Differences exp/chain/tdnn7o_sp
22
Parameter differences at tdnn1.affine
Relative Parameter Differences Parameter Differences
FIG. 22: Parameter differences at tdnn1.affine
3.0 2.5 2.0 1.5 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.08 0.06 0.04
Parameter Differences exp/chain/tdnn7o_sp
23
Parameter differences at tdnn10.affine
Relative Parameter Differences Parameter Differences
FIG. 23: Parameter differences at tdnn10.affine
1.6 1.4 1.2 1.0 0.8 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.06 0.04 0.02
Parameter Differences exp/chain/tdnn7o_sp
24
Parameter differences at tdnn10l
Relative Parameter Differences
Parameter Differences
FIG. 24: Parameter differences at tdnn10l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
25
Parameter differences at tdnn10l0
Relative Parameter Differences
Parameter Differences
FIG. 25: Parameter differences at tdnn10l0
4 3 2 1
0.25
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
26
Parameter differences at tdnn11.affine
Relative Parameter Differences Parameter Differences
FIG. 26: Parameter differences at tdnn11.affine
3.5 3.0 2.5 2.0 1.5 1.0
0.07 0.06 0.05 0.04 0.03
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
Parameter Differences exp/chain/tdnn7o_sp
27
Parameter differences at tdnn11l
Relative Parameter Differences
Parameter Differences
FIG. 27: Parameter differences at tdnn11l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
28
Parameter differences at tdnn11l0
Relative Parameter Differences
Parameter Differences
FIG. 28: Parameter differences at tdnn11l0
3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
29
Parameter differences at tdnn2.affine
Relative Parameter Differences Parameter Differences
FIG. 29: Parameter differences at tdnn2.affine
3.0 2.5 2.0 1.5 1.0
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.07 0.06 0.05 0.04 0.03
Parameter Differences exp/chain/tdnn7o_sp
30
Parameter differences at tdnn2l
Relative Parameter Differences
Parameter Differences
FIG. 30: Parameter differences at tdnn2l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.4 0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
31
Parameter differences at tdnn2l0
Relative Parameter Differences
Parameter Differences
FIG. 31: Parameter differences at tdnn2l0
4 3 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.25 0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
32
Parameter differences at tdnn3.affine
Relative Parameter Differences Parameter Differences
FIG. 32: Parameter differences at tdnn3.affine
2.0 1.5 1.0 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.08 0.06 0.04 0.02
Parameter Differences exp/chain/tdnn7o_sp
33
Parameter differences at tdnn3l
Relative Parameter Differences
Parameter Differences
FIG. 33: Parameter differences at tdnn3l
4 3 2 1
0.25 0.20 0.15 0.10 0.05
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
Parameter Differences exp/chain/tdnn7o_sp
34
Parameter differences at tdnn4.affine
Relative Parameter Differences Parameter Differences
FIG. 34: Parameter differences at tdnn4.affine
2.00 1.75 1.50 1.25 1.00 0.75 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.08 0.06 0.04 0.02
Parameter Differences exp/chain/tdnn7o_sp
35
Parameter differences at tdnn4l
Relative Parameter Differences
Parameter Differences
FIG. 35: Parameter differences at tdnn4l
6 4 2
0.4
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
36
Parameter differences at tdnn4l0
Relative Parameter Differences
Parameter Differences
FIG. 36: Parameter differences at tdnn4l0
4 3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.25 0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
37
Parameter differences at tdnn5.affine
Relative Parameter Differences Parameter Differences
FIG. 37: Parameter differences at tdnn5.affine
2.25 2.00 1.75 1.50 1.25 1.00 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.08 0.06 0.04
Parameter Differences exp/chain/tdnn7o_sp
38
Parameter differences at tdnn5l
Relative Parameter Differences
Parameter Differences
FIG. 38: Parameter differences at tdnn5l
4 3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.25 0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
39
Parameter differences at tdnn6.affine
Relative Parameter Differences Parameter Differences
FIG. 39: Parameter differences at tdnn6.affine
1.8 1.6 1.4 1.2 1.0 0.8
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.07 0.06 0.05 0.04 0.03 0.02
Parameter Differences exp/chain/tdnn7o_sp
40
Parameter differences at tdnn6l
Relative Parameter Differences
Parameter Differences
FIG. 40: Parameter differences at tdnn6l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
41
Parameter differences at tdnn6l0
Relative Parameter Differences
Parameter Differences
FIG. 41: Parameter differences at tdnn6l0
4 3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.25 0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
42
Parameter differences at tdnn7.affine
Relative Parameter Differences Parameter Differences
FIG. 42: Parameter differences at tdnn7.affine
2.5 2.0 1.5 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.07 0.06 0.05 0.04 0.03
Parameter Differences exp/chain/tdnn7o_sp
43
Parameter differences at tdnn7l
Relative Parameter Differences
Parameter Differences
FIG. 43: Parameter differences at tdnn7l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
44
Parameter differences at tdnn7l0
Relative Parameter Differences
Parameter Differences
FIG. 44: Parameter differences at tdnn7l0
3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
45
Parameter differences at tdnn8.affine
Relative Parameter Differences Parameter Differences
FIG. 45: Parameter differences at tdnn8.affine
1.6 1.4 1.2 1.0 0.8
0.07 0.06 0.05 0.04 0.03 0.02
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
Parameter Differences exp/chain/tdnn7o_sp
46
Parameter differences at tdnn8l
Relative Parameter Differences
Parameter Differences
FIG. 46: Parameter differences at tdnn8l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
47
Parameter differences at tdnn8l0
Relative Parameter Differences
Parameter Differences
FIG. 47: Parameter differences at tdnn8l0
4 3 2 1
0.25
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
48
Parameter differences at tdnn9.affine
Relative Parameter Differences Parameter Differences
FIG. 48: Parameter differences at tdnn9.affine
2.5 2.0 1.5 1.0
0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.08 0.06 0.04
Parameter Differences exp/chain/tdnn7o_sp
49
Parameter differences at tdnn9l
Relative Parameter Differences
Parameter Differences
FIG. 49: Parameter differences at tdnn9l
6 4 2 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.3 0.2 0.1
Parameter Differences exp/chain/tdnn7o_sp
50
Parameter differences at tdnn9l0
Relative Parameter Differences
Parameter Differences
FIG. 50: Parameter differences at tdnn9l0
3 2 1 0
200
400
600
800
1000
0
200
400
600 Iteration
800
1000
0.20 0.15 0.10 0.05
Parameter Differences exp/chain/tdnn7o_sp
