17 He Initialization

8.12

These functions are used in setting the initial values in a given theta based on the shapes of the tensors desired in the theta.

procedure
(init-theta θ-shapes) → (listof tensor?)
θ-shapes : (listof shape?)

Returns a list of tensors with shapes defined by θ-shapes with the following intialization rules.

Tensors of rank 1 are initialized to contain only 0.0.
Tensors of rank 2 are initialized to random numbers drawn from a normal distribution with a mean of 0.0 and a variance of (/ 2 fan-in) where fan-in is the last member of the shape.
Tensors of rank 3 are initialized to random numbers drawn from a normal distribution with a mean of 0.0 and a variance of (/ 2 fan-in) where fan-in is the product of the last two members of the shape.

procedure
(init-shape s) → tensor?
s : shape?

Returns a tensor of shape s according to the rules described in init-theta.

procedure
(zero-tensor s) → tensor?
s : shape?

Returns a tensor of shape s with only 0.0s as nested scalars in it.

1	Overview
2	Entry points
3	List functions
4	Tensor functions
5	Extended Functions
6	Automatic Differentiation
7	Differentiable extended numerical functions
8	Non-differentiable extended numerical functions
9	Base-rank (non-extended) differentiable functions
10	Boolean comparison functions
11	Tensorized comparison functions
12	Hyperparameters
13	Gradient Descent Functions and Hyperparameters
14	Layer functions
15	Loss Functions
16	Building blocks for neural networks
17	He Initialization
18	Random number functions
19	Models and Accuracy
20	Logging
21	Utilities
22	Setting tensor implementations