{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\nFirst Optimization\n==================\n\nQuadratic Function Example\n--------------------------\n\nUsually, Optuna is used to optimize hyper-parameters, but as an example,\nlet us directly optimize a quadratic function in an IPython shell.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import optuna"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The objective function is what will be optimized.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def objective(trial):\n    x = trial.suggest_uniform('x', -10, 10)\n    return (x - 2) ** 2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This function returns the value of $(x - 2)^2$. Our goal is to find the value of ``x``\nthat minimizes the output of the ``objective`` function. This is the \"optimization.\"\nDuring the optimization, Optuna repeatedly calls and evaluates the objective function with\ndifferent values of ``x``.\n\nA :class:`~optuna.trial.Trial` object corresponds to a single execution of the objective\nfunction and is internally instantiated upon each invocation of the function.\n\nThe `suggest` APIs (for example, :func:`~optuna.trial.Trial.suggest_float`) are called\ninside the objective function to obtain parameters for a trial.\n:func:`~optuna.trial.Trial.suggest_float` selects parameters uniformly within the range\nprovided. In our example, from $-10$ to $10$.\n\nTo start the optimization, we create a study object and pass the objective function to method\n:func:`~optuna.study.Study.optimize` as follows.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study = optuna.create_study()\nstudy.optimize(objective, n_trials=100)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can get the best parameter as follows.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(study.best_params)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can see that the ``x`` value found by Optuna is close to the optimal value of ``2``.\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-info\"><h4>Note</h4><p>When used to search for hyper-parameters in machine learning,\n    usually the objective function would return the loss or accuracy\n    of the model.</p></div>\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Study Object\n------------\n\nLet us clarify the terminology in Optuna as follows:\n\n* **Trial**: A single call of the objective function\n* **Study**: An optimization session, which is a set of trials\n* **Parameter**: A variable whose value is to be optimized, such as ``x`` in the above example\n\nIn Optuna, we use the study object to manage optimization.\nMethod :func:`~optuna.study.create_study` returns a study object.\nA study object has useful properties for analyzing the optimization outcome.\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the best parameter:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.best_params"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the best value:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.best_value"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the best trial:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.best_trial"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get all trials:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.trials"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the number of trials:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "len(study.trials)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "By executing :func:`~optuna.study.Study.optimize` again, we can continue the optimization.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.optimize(objective, n_trials=100)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the updated number of trials:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "len(study.trials)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.7"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}