{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# A simple example (unconstrained)\n", "\n", "## Define your problem\n", "\n", "Let's start with a simple problem of minimizing $x_1^4 + x_2^4$ with respect to $x_1$ and $x_2$.\n", "\n", "The mathematical problem statement is: \n", "\n", "\n", "$$\n", "\\underset{x_1, x_2 \\in \\mathbb{R}}{\\text{minimize}} \\quad x_1^4 + x_2^4\n", "$$\n", "\n", "We know the solution of this problem is $x_1=0$, and $x_2=0$.\n", "However, we start from an intial guess of $x_1=0.3$, and $x_2=0.3$ for the purposes of this tutorial.\n", "\n", "The problem is written in modOpt using the **Problem()** class as follows:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from modopt import Problem\n", "\n", "\n", "class X4(Problem):\n", " def initialize(self, ):\n", " # Name your problem\n", " self.problem_name = 'x^4'\n", "\n", " def setup(self):\n", " # Add design variables of your problem\n", " self.add_design_variables('x',\n", " shape=(2, ),\n", " vals=np.array([.3, .3]))\n", " self.add_objective('f')\n", "\n", " def setup_derivatives(self):\n", " # Declare objective gradient and its shape\n", " self.declare_objective_gradient(wrt='x', )\n", "\n", " # Compute the value of the objective with given design variable values\n", " def compute_objective(self, dvs, obj):\n", " obj['f'] = np.sum(dvs['x']**4)\n", "\n", " def compute_objective_gradient(self, dvs, grad):\n", " grad['x'] = 4 * dvs['x']**3" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Develop/Build your optimization algorithm\n", "\n", "Here we look at the **steepest descent** algorithm for unconstrained problems. \n", "We will later (in the next section) use it to solve the unconstrained optimization problem defined above.\n", "\n", "For a general unconstrained optimization problem stated as: \n", "\n", "$$\n", "\\underset{x \\in \\mathbb{R^n}}{\\text{minimize}} \\quad f(x)\n", "$$\n", "\n", "the steepest descent algorithms computes the new iterate recursively by using the formula\n", "\n", "$$\n", "x_{k+1} = x_{k} - \\nabla f(x_k) .\n", "$$\n", "\n", "Given an initial guess $x_0$, we can write an optimizer using the steepest descent algorithm using the **Optimizer()** class in modOpt as follows:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import time\n", "from modopt import Optimizer\n", "\n", "\n", "class SteepestDescent(Optimizer):\n", " def initialize(self):\n", "\n", " # Name your algorithm\n", " self.solver_name = 'steepest_descent'\n", "\n", " self.obj = self.problem._compute_objective\n", " self.grad = self.problem._compute_objective_gradient\n", "\n", " self.options.declare('maxiter', default=1000, types=int)\n", " self.options.declare('opt_tol', default=1e-5, types=float)\n", " # Enable user to specify, as a list, which among the available outputs\n", " # need to be written to output files\n", " self.options.declare('readable_outputs', types=list, default=[])\n", "\n", " # Specify format of outputs available from your optimizer after each iteration\n", " self.available_outputs = {\n", " 'itr': int,\n", " 'obj': float,\n", " # for arrays from each iteration, shapes need to be declared\n", " 'x': (float, (self.problem.nx, )),\n", " 'opt': float,\n", " 'time': float,\n", " }\n", "\n", " def setup(self):\n", " # Instantiate any modules you need for the algorithm\n", " pass\n", "\n", " def solve(self):\n", " nx = self.problem.nx\n", " x = self.problem.x0\n", " opt_tol = self.options['opt_tol']\n", " maxiter = self.options['maxiter']\n", "\n", " obj = self.obj\n", " grad = self.grad\n", "\n", " start_time = time.time()\n", "\n", " # Setting intial values for initial iterates\n", " x_k = x * 1.\n", " f_k = obj(x_k)\n", " g_k = grad(x_k)\n", "\n", " # Iteration counter\n", " itr = 0\n", "\n", " # Optimality\n", " opt = np.linalg.norm(g_k)\n", "\n", " # Initializing outputs\n", " self.update_outputs(itr=0,\n", " x=x_k,\n", " obj=f_k,\n", " opt=opt,\n", " time=time.time() - start_time)\n", "\n", " while (opt > opt_tol and itr < maxiter):\n", " itr_start = time.time()\n", " itr += 1\n", "\n", " # ALGORITHM STARTS HERE\n", " # >>>>>>>>>>>>>>>>>>>>>\n", "\n", " p_k = -g_k\n", "\n", " x_k += p_k\n", " f_k = obj(x_k)\n", " g_k = grad(x_k)\n", "\n", " opt = np.linalg.norm(g_k)\n", "\n", " # <<<<<<<<<<<<<<<<<<<\n", " # ALGORITHM ENDS HERE\n", "\n", " # Append arrays inside outputs dict with new values from the current iteration\n", " self.update_outputs(itr=itr,\n", " x=x_k,\n", " obj=f_k,\n", " opt=opt,\n", " time=time.time() - start_time)\n", "\n", " self.total_time = time.time() - start_time\n", "\n", " self.results = {\n", " 'x': x_k,\n", " 'objective': f_k,\n", " 'optimality': opt,\n", " 'itr': itr,\n", " 'time': self.total_time\n", " }\n", "\n", " # Run post-processing for the Optimizer() base class\n", " self.run_post_processing()\n", "\n", " return self.results" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "The **Optimizer()** class records all the data needed using the `outputs` dictionary.\n", "\n", "## Solve your problem using your optimizer\n", "\n", "Now that we have modeled the problem and developed the optimizer, the task remaining is to solve the problem with the optimizer.\n", "For this, we need to set up our optimizer with the problem and pass in optimizer-specific parameters. \n", "Default values will be assumed if the optimizer parameters are not passed in." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Setting objective name as \"f\".\n", "\n", "----------------------------------------------------------------------------\n", "Derivative type | Calc norm | FD norm | Abs error norm | Rel error norm \n", "----------------------------------------------------------------------------\n", "\n", "Gradient | 1.5274e-01 | 1.5274e-01 | 7.6367e-07 | 4.9999e-06 \n", "----------------------------------------------------------------------------\n", "\n", "\n", "\tSolution from modOpt:\n", "\t----------------------------------------------------------------------------------------------------\n", "\tProblem : x^4\n", "\tSolver : steepest_descent\n", "\tobjective : 2.830456199167557e-06\n", "\toptimality : 0.00023211051832047348\n", "\titr : 100\n", "\ttime : 0.14033794403076172\n", "\ttotal_callbacks : 206\n", "\tobj_evals : 104\n", "\tgrad_evals : 102\n", "\thess_evals : 0\n", "\tcon_evals : 0\n", "\tjac_evals : 0\n", "\treused_callbacks : 0\n", "\tout_dir : x^4_outputs/2025-01-20_16.44.55.032044\n", "\t----------------------------------------------------------------------------------------------------\n", "\n", "=========================================================================\n", " modOpt summary table: \n", "=========================================================================\n", " # itr obj opt time \n", " 0 0 1.620000E-02 1.527351E-01 7.700920E-05 \n", " 1 1 2.717909E-03 4.003858E-02 2.296209E-03 \n", " 2 2 1.435827E-03 2.481013E-02 4.526138E-03 \n", " 3 3 9.123600E-04 1.765742E-02 6.705999E-03 \n", " 4 4 6.383038E-04 1.350744E-02 8.321047E-03 \n", " 5 5 4.744946E-04 1.081374E-02 9.446144E-03 \n", " 6 6 3.679275E-04 8.935609E-03 1.051021E-02 \n", " 7 7 2.943483E-04 7.558728E-03 1.154304E-02 \n", " 8 8 2.412398E-04 6.510874E-03 1.259208E-02 \n", " 9 9 2.015609E-04 5.689934E-03 1.377201E-02 \n", " 10 10 1.710840E-04 5.031639E-03 1.494718E-02 \n", " 11 11 1.471372E-04 4.493599E-03 1.942706E-02 \n", " 12 12 1.279604E-04 4.046775E-03 2.194524E-02 \n", " 13 13 1.123533E-04 3.670640E-03 2.305794E-02 \n", " 14 14 9.947358E-05 3.350297E-03 2.406907E-02 \n", " 15 15 8.871518E-05 3.074687E-03 2.492404E-02 \n", " 16 16 7.963259E-05 2.835441E-03 2.573705E-02 \n", " 17 17 7.189216E-05 2.626114E-03 2.662206E-02 \n", " 18 18 6.523983E-05 2.441671E-03 2.771997E-02 \n", " 19 19 5.947928E-05 2.278121E-03 2.871823E-02 \n", " 20 20 5.445680E-05 2.132267E-03 2.956796E-02 \n", " 21 21 5.005061E-05 2.001518E-03 3.026724E-02 \n", " 22 22 4.616319E-05 1.883755E-03 3.272009E-02 \n", " 23 23 4.271564E-05 1.777226E-03 3.614116E-02 \n", " 24 24 3.964361E-05 1.680476E-03 3.793526E-02 \n", " 25 25 3.689416E-05 1.592285E-03 3.982806E-02 \n", " 26 26 3.442338E-05 1.511620E-03 4.081607E-02 \n", " 27 27 3.219464E-05 1.437607E-03 4.203224E-02 \n", " 28 28 3.017715E-05 1.369497E-03 4.326415E-02 \n", " 29 29 2.834488E-05 1.306648E-03 4.417014E-02 \n", " 30 30 2.667573E-05 1.248504E-03 4.515886E-02 \n", " 31 31 2.515080E-05 1.194583E-03 4.615092E-02 \n", " 32 32 2.375385E-05 1.144466E-03 4.722309E-02 \n", " 33 33 2.247088E-05 1.097786E-03 4.851007E-02 \n", " 34 34 2.128977E-05 1.054219E-03 5.262589E-02 \n", " 35 35 2.019996E-05 1.013480E-03 5.471587E-02 \n", " 36 36 1.919224E-05 9.753190E-04 5.635214E-02 \n", " 37 37 1.825852E-05 9.395107E-04 5.748510E-02 \n", " 38 38 1.739172E-05 9.058564E-04 5.873799E-02 \n", " 39 39 1.658555E-05 8.741779E-04 6.049609E-02 \n", " 40 40 1.583446E-05 8.443159E-04 6.139398E-02 \n", " 41 41 1.513354E-05 8.161272E-04 6.257200E-02 \n", " 42 42 1.447839E-05 7.894827E-04 6.385422E-02 \n", " 43 43 1.386509E-05 7.642661E-04 6.530786E-02 \n", " 44 44 1.329015E-05 7.403720E-04 6.627011E-02 \n", " 45 45 1.275042E-05 7.177050E-04 6.801510E-02 \n", " 46 46 1.224307E-05 6.961781E-04 6.909013E-02 \n", " 47 47 1.176556E-05 6.757124E-04 7.039428E-02 \n", " 48 48 1.131557E-05 6.562357E-04 7.137203E-02 \n", " 49 49 1.089103E-05 6.376822E-04 7.259488E-02 \n", " 50 50 1.049005E-05 6.199915E-04 7.373691E-02 \n", " 51 51 1.011091E-05 6.031083E-04 7.483220E-02 \n", " 52 52 9.752052E-06 5.869816E-04 7.602406E-02 \n", " 53 53 9.412042E-06 5.715647E-04 7.730889E-02 \n", " 54 54 9.089584E-06 5.568144E-04 7.816195E-02 \n", " 55 55 8.783485E-06 5.426910E-04 7.914805E-02 \n", " 56 56 8.492654E-06 5.291576E-04 7.998013E-02 \n", " 57 57 8.216089E-06 5.161802E-04 8.074498E-02 \n", " 58 58 7.952870E-06 5.037272E-04 8.181000E-02 \n", " 59 59 7.702148E-06 4.917693E-04 8.313012E-02 \n", " 60 60 7.463144E-06 4.802793E-04 8.475709E-02 \n", " 61 61 7.235136E-06 4.692319E-04 8.579993E-02 \n", " 62 62 7.017457E-06 4.586034E-04 8.690786E-02 \n", " 63 63 6.809492E-06 4.483720E-04 8.872819E-02 \n", " 64 64 6.610669E-06 4.385171E-04 9.033203E-02 \n", " 65 65 6.420459E-06 4.290195E-04 9.153104E-02 \n", " 66 66 6.238371E-06 4.198613E-04 9.412193E-02 \n", " 67 67 6.063947E-06 4.110257E-04 9.561396E-02 \n", " 68 68 5.896761E-06 4.024970E-04 9.689593E-02 \n", " 69 69 5.736419E-06 3.942604E-04 9.796000E-02 \n", " 70 70 5.582550E-06 3.863020E-04 1.006241E-01 \n", " 71 71 5.434810E-06 3.786089E-04 1.021061E-01 \n", " 72 72 5.292877E-06 3.711687E-04 1.037152E-01 \n", " 73 73 5.156450E-06 3.639700E-04 1.047208E-01 \n", " 74 74 5.025246E-06 3.570019E-04 1.057360E-01 \n", " 75 75 4.899003E-06 3.502541E-04 1.066272E-01 \n", " 76 76 4.777472E-06 3.437171E-04 1.074870E-01 \n", " 77 77 4.660422E-06 3.373816E-04 1.082661E-01 \n", " 78 78 4.547634E-06 3.312391E-04 1.093841E-01 \n", " 79 79 4.438903E-06 3.252814E-04 1.102772E-01 \n", " 80 80 4.334037E-06 3.195008E-04 1.112063E-01 \n", " 81 81 4.232854E-06 3.138900E-04 1.122870E-01 \n", " 82 82 4.135184E-06 3.084421E-04 1.133740E-01 \n", " 83 83 4.040865E-06 3.031505E-04 1.144960E-01 \n", " 84 84 3.949746E-06 2.980090E-04 1.176901E-01 \n", " 85 85 3.861683E-06 2.930117E-04 1.198022E-01 \n", " 86 86 3.776540E-06 2.881529E-04 1.219893E-01 \n", " 87 87 3.694190E-06 2.834274E-04 1.228631E-01 \n", " 88 88 3.614511E-06 2.788301E-04 1.241779E-01 \n", " 89 89 3.537390E-06 2.743562E-04 1.256919E-01 \n", " 90 90 3.462717E-06 2.700009E-04 1.269281E-01 \n", " 91 91 3.390390E-06 2.657601E-04 1.280401E-01 \n", " 92 92 3.320312E-06 2.616295E-04 1.289370E-01 \n", " 93 93 3.252389E-06 2.576050E-04 1.299410E-01 \n", " 94 94 3.186535E-06 2.536831E-04 1.309121E-01 \n", " 95 95 3.122665E-06 2.498599E-04 1.319849E-01 \n", " 96 96 3.060702E-06 2.461321E-04 1.331420E-01 \n", " 97 97 3.000569E-06 2.424964E-04 1.344991E-01 \n", " 98 98 2.942195E-06 2.389495E-04 1.356480E-01 \n", " 99 99 2.885513E-06 2.354885E-04 1.373892E-01 \n", " 100 100 2.830456E-06 2.321105E-04 1.387231E-01 \n", "=========================================================================\n", "\n", "\n", "100\n", "[0.03449107 0.03449107]\n", "0.14033794403076172\n", "2.830456199167557e-06\n", "0.00023211051832047348\n" ] } ], "source": [ "# Set your optimality tolerance\n", "opt_tol = 1E-8\n", "# Set maximum optimizer iteration limit\n", "maxiter = 100\n", "\n", "prob = X4()\n", "\n", "# Set up your optimizer with your problem and pass in optimizer parameters\n", "# And declare outputs to be stored\n", "optimizer = SteepestDescent(prob,\n", " opt_tol=opt_tol,\n", " maxiter=maxiter,\n", " readable_outputs=['itr', 'obj', 'x', 'opt', 'time'])\n", "\n", "# Check first derivatives at the initial guess, if needed\n", "optimizer.check_first_derivatives(prob.x0)\n", "\n", "# Solve your optimization problem\n", "optimizer.solve()\n", "\n", "# Print results of optimization (summary_table contains information from each iteration)\n", "optimizer.print_results(summary_table=True)\n", "\n", "# Print any output that was declared\n", "# Since the arrays are long, here we only print the last entry and\n", "# verify it with the print_results() above\n", "\n", "print('\\n')\n", "print(optimizer.results['itr'])\n", "print(optimizer.results['x'])\n", "print(optimizer.results['time'])\n", "print(optimizer.results['objective'])\n", "print(optimizer.results['optimality'])" ] } ], "metadata": { "kernelspec": { "display_name": "venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }