Table of Contents
Shortcuts

# torch.backends

torch.backends controls the behavior of various backends that PyTorch supports.

These backends include:

- `torch.backends.cpu`

- `torch.backends.cuda`

- `torch.backends.cudnn`

- `torch.backends.mps`

- `torch.backends.mkl`

- `torch.backends.mkldnn`

- `torch.backends.openmp`

- `torch.backends.opt_einsum`

- `torch.backends.xeon`

## torch.backends.cpu

torch.backends.cpu.get_cpu_capability()

  Returns cpu capability as a string value.

  Possible values: - “DEFAULT” - “VSX” - “Z VECTOR” - “NO AVX” - “AVX2” - “AVX512”

  Return type:

    str

## torch.backends.cuda

torch.backends.cuda.is_built()

  Returns whether PyTorch is built with CUDA support. Note that this doesn’t necessarily mean CUDA is available; just that if this PyTorch binary were run a machine with working CUDA drivers and devices, we would be able to use it.

torch.backends.cuda.matmul.allow_tf32

  A `bool` that controls whether TensorFloat-32 tensor cores may be used in matrix multiplications on Ampere or newer GPUs. See TensorFloat-32(TF32) on Ampere devices.

torch.backends.cuda.matmul.allow_fp16_reduced_precision_reduction

  A `bool` that controls whether reduced precision reductions (e.g., with fp16 accumulation type) are allowed with fp16 GEMMs.

torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction

  A `bool` that controls whether reduced precision reductions are allowed with bf16 GEMMs.

torch.backends.cuda.cufft_plan_cache

  `cufft_plan_cache` contains the cuFFT plan caches for each CUDA device. Query a specific device i’s cache via torch.backends.cuda.cufft_plan_cache[i].

  torch.backends.cuda.cufft_plan_cache.size

    A readonly `int` that shows the number of plans currently in a cuFFT plan cache.

  torch.backends.cuda.cufft_plan_cache.max_size

    A `int` that controls the capacity of a cuFFT plan cache.

  torch.backends.cuda.cufft_plan_cache.clear()

    Clears a cuFFT plan cache.

torch.backends.cuda.preferred_linalg_library(backend=None)

  Warning

  This flag is experimental and subject to change.

  When PyTorch runs a CUDA linear algebra operation it often uses the cuSOLVER or MAGMA libraries, and if both are available it decides which to use with a heuristic. This flag (a `str`) allows overriding those heuristics.

  - If “cusolver” is set then cuSOLVER will be used wherever possible.

  - If “magma” is set then MAGMA will be used wherever possible.

  - If “default” (the default) is set then heuristics will be used to pick between cuSOLVER and MAGMA if both are available.

  - When no input is given, this function returns the currently preferred library.

  - User may use the environment variable TORCH_LINALG_PREFER_CUSOLVER=1 to set the preferred library to cuSOLVER globally. This flag only sets the initial value of the preferred library and the preferred library may still be overridden by this function call later in your script.

  Note: When a library is preferred other libraries may still be used if the preferred library doesn’t implement the operation(s) called. This flag may achieve better performance if PyTorch’s heuristic library selection is incorrect for your application’s inputs.

  Currently supported linalg operators:

  - `torch.linalg.inv()`

  - `torch.linalg.inv_ex()`

  - `torch.linalg.cholesky()`

  - `torch.linalg.cholesky_ex()`

  - `torch.cholesky_solve()`

  - `torch.cholesky_inverse()`

  - `torch.linalg.lu_factor()`

  - `torch.linalg.lu()`

  - `torch.linalg.lu_solve()`

  - `torch.linalg.qr()`

  - `torch.linalg.eigh()`

  - `torch.linalg.eighvals()`

  - `torch.linalg.svd()`

  - `torch.linalg.svdvals()`

  Return type:

    _LinalgBackend

class torch.backends.cuda.SDPBackend(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

  Enum class for the scaled dot product attention backends.

  Warning

  This class is in beta and subject to change.

  This class needs to stay aligned with the enum defined in: pytorch/aten/src/ATen/native/transformers/sdp_utils_cpp.h

torch.backends.cuda.flash_sdp_enabled()

  Warning

  This flag is beta and subject to change.

  Returns whether flash scaled dot product attention is enabled or not.

torch.backends.cuda.enable_mem_efficient_sdp(enabled)

  Warning

  This flag is beta and subject to change.

  Enables or disables memory efficient scaled dot product attention.

torch.backends.cuda.mem_efficient_sdp_enabled()

  Warning

  This flag is beta and subject to change.

  Returns whether memory efficient scaled dot product attention is enabled or not.

torch.backends.cuda.enable_flash_sdp(enabled)

  Warning

  This flag is beta and subject to change.

  Enables or disables flash scaled dot product attention.

torch.backends.cuda.math_sdp_enabled()

  Warning

  This flag is beta and subject to change.

  Returns whether math scaled dot product attention is enabled or not.

torch.backends.cuda.enable_math_sdp(enabled)

  Warning

  This flag is beta and subject to change.

  Enables or disables math scaled dot product attention.

torch.backends.cuda.sdp_kernel(enable_flash=True, enable_math=True, enable_mem_efficient=True)

  Warning

  This flag is beta and subject to change.

  This context manager can be used to temporarily enable or disable any of the three backends for scaled dot product attention. Upon exiting the context manager, the previous state of the flags will be restored.

## torch.backends.cudnn

torch.backends.cudnn.version()

  Returns the version of cuDNN

torch.backends.cudnn.is_available()

  Returns a bool indicating if CUDNN is currently available.

torch.backends.cudnn.enabled

  A `bool` that controls whether cuDNN is enabled.

torch.backends.cudnn.allow_tf32

  A `bool` that controls where TensorFloat-32 tensor cores may be used in cuDNN convolutions on Ampere or newer GPUs. See TensorFloat-32(TF32) on Ampere devices.

torch.backends.cudnn.deterministic

  A `bool` that, if True, causes cuDNN to only use deterministic convolution algorithms. See also `torch.are_deterministic_algorithms_enabled()` and `torch.use_deterministic_algorithms()`.

torch.backends.cudnn.benchmark

  A `bool` that, if True, causes cuDNN to benchmark multiple convolution algorithms and select the fastest.

torch.backends.cudnn.benchmark_limit

  A `int` that specifies the maximum number of cuDNN convolution algorithms to try when torch.backends.cudnn.benchmark is True. Set benchmark_limit to zero to try every available algorithm. Note that this setting only affects convolutions dispatched via the cuDNN v8 API.

## torch.backends.mps

torch.backends.mps.is_available()

  Returns a bool indicating if MPS is currently available.

  Return type:

    bool

torch.backends.mps.is_built()

  Returns whether PyTorch is built with MPS support. Note that this doesn’t necessarily mean MPS is available; just that if this PyTorch binary were run a machine with working MPS drivers and devices, we would be able to use it.

  Return type:

    bool

## torch.backends.mkl

torch.backends.mkl.is_available()

  Returns whether PyTorch is built with MKL support.

class torch.backends.mkl.verbose(enable)

  On-demand oneMKL verbosing functionality To make it easier to debug performance issues, oneMKL can dump verbose messages containing execution information like duration while executing the kernel. The verbosing functionality can be invoked via an environment variable named MKL_VERBOSE. However, this methodology dumps messages in all steps. Those are a large amount of verbose messages. Moreover, for investigating the performance issues, generally taking verbose messages for one single iteration is enough. This on-demand verbosing functionality makes it possible to control scope for verbose message dumping. In the following example, verbose messages will be dumped out for the second inference only.

      import torch
      model(data)
      with torch.backends.mkl.verbose(torch.backends.mkl.VERBOSE_ON):
          model(data)

  Parameters:

    level – Verbose level - `VERBOSE_OFF`: Disable verbosing - `VERBOSE_ON`: Enable verbosing

## torch.backends.mkldnn

torch.backends.mkldnn.is_available()

  Returns whether PyTorch is built with MKL-DNN support.

class torch.backends.mkldnn.verbose(level)

  On-demand oneDNN (former MKL-DNN) verbosing functionality To make it easier to debug performance issues, oneDNN can dump verbose messages containing information like kernel size, input data size and execution duration while executing the kernel. The verbosing functionality can be invoked via an environment variable named DNNL_VERBOSE. However, this methodology dumps messages in all steps. Those are a large amount of verbose messages. Moreover, for investigating the performance issues, generally taking verbose messages for one single iteration is enough. This on-demand verbosing functionality makes it possible to control scope for verbose message dumping. In the following example, verbose messages will be dumped out for the second inference only.

      import torch
      model(data)
      with torch.backends.mkldnn.verbose(torch.backends.mkldnn.VERBOSE_ON):
          model(data)

  Parameters:

    level – Verbose level - `VERBOSE_OFF`: Disable verbosing - `VERBOSE_ON`: Enable verbosing - `VERBOSE_ON_CREATION`: Enable verbosing, including oneDNN kernel creation

## torch.backends.openmp

torch.backends.openmp.is_available()

  Returns whether PyTorch is built with OpenMP support.

## torch.backends.opt_einsum

torch.backends.opt_einsum.is_available()

  Returns a bool indicating if opt_einsum is currently available.

  Return type:

    bool

torch.backends.opt_einsum.get_opt_einsum()

  Returns the opt_einsum package if opt_einsum is currently available, else None.

  Return type:

    Any

torch.backends.opt_einsum.enabled

  A :class:`bool` that controls whether opt_einsum is enabled (`True` by default). If so, torch.einsum will use opt_einsum (https://optimized-einsum.readthedocs.io/en/stable/path_finding.html) if available to calculate an optimal path of contraction for faster performance.

  If opt_einsum is not available, torch.einsum will fall back to the default contraction path of left to right.

torch.backends.opt_einsum.strategy

  A :class:`str` that specifies which strategies to try when `torch.backends.opt_einsum.enabled` is `True`. By default, torch.einsum will try the “auto” strategy, but the “greedy” and “optimal” strategies are also supported. Note that the “optimal” strategy is factorial on the number of inputs as it tries all possible paths. See more details in opt_einsum’s docs (https://optimized-einsum.readthedocs.io/en/stable/path_finding.html).

## torch.backends.xeon

## Docs

Access comprehensive developer documentation for PyTorch

View Docs

## Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

## Resources

Find development resources and get your questions answered

View Resources

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook’s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.