description: Create prior for Variational layers with kernel and bias.
View source on GitHub |
Create prior for Variational layers with kernel and bias.
tfp.experimental.nn.util.make_kernel_bias_prior_spike_and_slab(
kernel_shape, bias_shape, kernel_initializer=None, bias_initializer=None,
kernel_batch_ndims=0, bias_batch_ndims=0, dtype=tf.float32,
kernel_name='prior_kernel', bias_name='prior_bias'
)
Note: Distribution scale is inversely related to regularization strength.
Consider a "Normal" prior; bigger scale corresponds to less L2 regularization.
I.e.,
python
scale = (2. * l2weight)**-0.5
l2weight = scale**-2. / 2.
have a similar regularizing effect.
The std. deviation of each of the component distributions returned by this
function is approximately 1415
(or approximately l2weight = 25e-6
). In
other words this prior is extremely "weak".
Args | |
---|---|
kernel_shape
|
... |
bias_shape
|
... |
kernel_initializer
|
Ignored.
Default value: None (i.e., tf.initializers.glorot_uniform() ).
|
bias_initializer
|
Ignored.
Default value: None (i.e., tf.initializers.zeros() ).
|
kernel_batch_ndims
|
...
Default value: 0 .
|
bias_batch_ndims
|
...
Default value: 0 .
|
dtype
|
...
Default value: tf.float32 .
|
kernel_name
|
...
Default value: "prior_kernel" .
|
bias_name
|
...
Default value: "prior_bias" .
|
Returns | |
---|---|
kernel_and_bias_distribution
|
... |