Table of Contents
Shortcuts

# torch.autograd.Function.forward

static Function.forward(ctx, *args, **kwargs)

  This function is to be overridden by all subclasses. There are two ways to define forward:

  Usage 1 (Combined forward and ctx):

      @staticmethod
      def forward(ctx: Any, *args: Any, **kwargs: Any) -> Any:
          pass

  - It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types).

  - See Combined or separate forward() and setup_context() for more details

  Usage 2 (Separate forward and ctx):

      @staticmethod
      def forward(*args: Any, **kwargs: Any) -> Any:
          pass

      @staticmethod
      def setup_context(ctx: Any, inputs: Tuple[Any, ...], output: Any) -> None:
          pass

  - The forward no longer accepts a ctx argument.

  - Instead, you must also override the `torch.autograd.Function.setup_context()` staticmethod to handle setting up the `ctx` object. `output` is the output of the forward, `inputs` are a Tuple of inputs to the forward.

  - See Extending torch.autograd for more details

  The context can be used to store arbitrary data that can be then retrieved during the backward pass. Tensors should not be stored directly on ctx (though this is not currently enforced for backward compatibility). Instead, tensors should be saved either with `ctx.save_for_backward()` if they are intended to be used in `backward` (equivalently, `vjp`) or `ctx.save_for_forward()` if they are intended to be used for in `jvp`.

  Return type:

    Any

## Docs

Access comprehensive developer documentation for PyTorch

View Docs

## Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

## Resources

Find development resources and get your questions answered

View Resources

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook’s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.