Table of Contents
Shortcuts

# torch.nn.utils.spectral_norm

torch.nn.utils.spectral_norm(module, name='weight', n_power_iterations=1, eps=1e-12, dim=None)

  Applies spectral normalization to a parameter in the given module.

  W S N=W σ(W),σ(W)=max⁡h:h≠0∥W h∥2∥h∥2\mathbf{W}_{SN} = \dfrac{\mathbf{W}}{\sigma(\mathbf{W})}, \sigma(\mathbf{W}) = \max_{\mathbf{h}: \mathbf{h} \ne 0} \dfrac{\|\mathbf{W} \mathbf{h}\|_2}{\|\mathbf{h}\|_2} W SN​=σ(W)W​,σ(W)=h:h=0 max​∥h∥2​∥Wh∥2​​

  Spectral normalization stabilizes the training of discriminators (critics) in Generative Adversarial Networks (GANs) by rescaling the weight tensor with spectral norm σ\sigma σ of the weight matrix calculated using power iteration method. If the dimension of the weight tensor is greater than 2, it is reshaped to 2D in power iteration method to get spectral norm. This is implemented via a hook that calculates spectral norm and rescales weight before every `forward()` call.

  See Spectral Normalization for Generative Adversarial Networks .

  Parameters:

    - module (nn.Module) – containing module

    - name (str, optional) – name of weight parameter

    - n_power_iterations (int, optional) – number of power iterations to calculate spectral norm

    - eps (float, optional) – epsilon for numerical stability in calculating norms

    - dim (int, optional) – dimension corresponding to number of outputs, the default is `0`, except for modules that are instances of ConvTranspose{1,2,3}d, when it is `1`

  Returns:

    The original module with the spectral norm hook

  Return type:

    T_module

  Note

  This function has been reimplemented as `torch.nn.utils.parametrizations.spectral_norm()` using the new parametrization functionality in `torch.nn.utils.parametrize.register_parametrization()`. Please use the newer version. This function will be deprecated in a future version of PyTorch.

  Example:

      >>> m = spectral_norm(nn.Linear(20, 40))
      >>> m
      Linear(in_features=20, out_features=40, bias=True)
      >>> m.weight_u.size()
      torch.Size([40])

## Docs

Access comprehensive developer documentation for PyTorch

View Docs

## Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

## Resources

Find development resources and get your questions answered

View Resources

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook’s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.