{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c3109b7e-0092-47a3-aca2-288e745731ba",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Lecture 11"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0357b50-2962-423e-85f1-80217ec3d134",
   "metadata": {},
   "source": [
    "#### Announcements\n",
    "\n",
    "* Problems for today: P11\n",
    "\n",
    "* Artifact showcase!\n",
    "\n",
    "* Today's tea: Nepal Imperial Black"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38efd636-c09e-4d1f-a828-33e0fcc75686",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### Goals\n",
    "* Be able to explain why panorama input images need to be taken from the same position\n",
    "* Understand the mathematics of perspective image formation under a pinhole camera model, and associated terminology:\n",
    "    * Center of projection, projection plane, focal length, optical axis\n",
    "* Know what is meant by world coordinates, camera coordinates, and image coordinates.\n",
    "* Understand the relationship between depth and disparity in a simple rectified stero camera pair"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ed0970a-cf75-497c-beba-f716066da9c0",
   "metadata": {},
   "source": [
    "### Stitching 360 Panoramas\n",
    "\n",
    "Can we make a 360 panorama with the tools we have?\n",
    "\n",
    "A useful perspective: homographies are 3x3 linear transformations on planar images, which then get projected back onto a single plane."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff70da82-3173-45ef-9ee1-47e9da0ff1ec",
   "metadata": {},
   "source": [
    "#### Outline\n",
    "\n",
    "* segue from panoramas with image formation in mind\n",
    "\n",
    "  * aside about spherical stitching\n",
    "\n",
    "* panoramas need a common COP - why?\n",
    "\n",
    "  * if not, then position depends on depth\n",
    "  * hang on a minute, can we use that?\n",
    "\n",
    "* Pinhole camera model\n",
    "\n",
    "  * 1053cam mk I: not a camera\n",
    "  * 1053cam mk II: pinhole camera\n",
    "  * 1053cam mk III: math camera, with terminology\n",
    "    * pinhole = center of projection\n",
    "    * image plane = projection plane\n",
    "    * focal length\n",
    "    * optical axis\n",
    "    * camera coordinates\n",
    "    * image coordinates\n",
    "  * **HW1**\n",
    "\n",
    "* Where we're headed: the camera matrix - one matrix to project them all!\n",
    "\n",
    "  * 3D world points to 2D pixel coordinates via\n",
    "    * Extrinsics: world to camera\n",
    "    * Projection: 3D to 2D\n",
    "    * Intrinsics: camera to image\n",
    "  * Enables: depth from disparity / stereo; sfm/mvs, slam, ...\n",
    "\n",
    "* Pinhole projection\n",
    "\n",
    "  * f=1 case\n",
    "    * 10th grade way - geometry; **HW2** \n",
    "    * 15th grade way - linear algebra; **HW3** \n",
    "\n",
    "* Camera coordiantes: P2 where points normalize onto the image plane\n",
    "\n",
    "  * f=$f$ case:\n",
    "    * 10th grade way - geometry; **HW4**\n",
    "    * 15th grade way - linear algebra; **HW5**\n",
    "\n",
    "* Depth from disparity: 10th grade way for the simplest case\n",
    "\n",
    "  * **HW6-7**\n",
    "\n",
    "\n",
    "(likely save for next time):\n",
    "\n",
    "* Rectified stereo:\n",
    "\n",
    "  * depth from disparity reduces stereo vision to the correspondence problem\n",
    "\n",
    "  * assumed a simple case: this is the **rectified case** where (assumptions)\n",
    "\n",
    "  * correspondence - sounds familiar, but now it's dense. some metrics:\n",
    "\n",
    "    * SSD - sum of squared differences\n",
    "    * CC - cross-correlation: filter the right scanline with the left patch; where product is highest, call it a match; in practice, use NCC instead:\n",
    "    * NCC - normalized cross-correlation: standardize (subtract mean, divide by std) patches before multiplication to add invariance to photometric changes\n",
    "\n",
    "  * The **cost volume**: given a matching cost c:\n",
    "\n",
    "    ```\n",
    "    for i in rows:\n",
    "      for j in columns:\n",
    "        for d in disparities:\n",
    "          C[i, j, d] = c(img1[i,j], img2[i,j+d])\n",
    "    ```\n",
    "\n",
    "    (note that c will usually look at a patch around img[i,j])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}