How to make input variable as trainable parameter in a neural network?

Question

I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given output.

Answer 1

It's just a typical optimization problem. You want to optimize function \begin{equation} f(\theta, x) \end{equation} with fixed parameters $\theta$ (network weights) and optimization variable $x$. Typically the approach is consisted of choosing a search direction and direction step. For deciding search direction you can use the gradient of objective with respect to decision variable and have an update of form \begin{equation} x = x - \alpha \nabla_x f(\theta, x) \end{equation} where $\nabla_x f(\theta, x)$ is the gradient. You can also use Newton or Quasi-Newton methods of form \begin{equation} x = x - \alpha B^{-1}\nabla_x f(\theta, x) \end{equation} where $B$ is a Hessian or approximate Hessian of $f(\theta, x)$ with respect to $x$. Step length parameter is usually decided with line search or you can also use trust region approaches which decide step direction with constraints to it's length. For more details about optimization you can consult Numerical Optimization by Nocedal and Wright