kuhyx/WUT_Computer_Science
1
0
Fork 0
mirror of https://github.com/kuhyx/WUT_Computer_Science.git synced 2026-11-10 20:47:09 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
37cc00fa21
WUT_Computer_Science/Programming/twm_4/TWM_KerasIntro.ipynb
Krzysztof kuhy Rudnicki 1fd1de2f82
Some checks are pending
md-naming / markdown naming convention (push) Waiting to run
Details
pre-commit / pre-commit (push) Waiting to run
Details
runnability / runnability (push) Waiting to run
Details
twm_4: zero findings, with the notebook's outputs proved untouched 
228 ruff findings to 0 across the notebook, its partial export and twm.py, and
mypy --strict is clean on both .py files.

A notebook is a deliverable that happens to contain code, so the check that
mattered was not "does it still run" -- it needs a GPU, MNIST, CIFAR-10 and a
slice of Quick, Draw! to run at all -- but "is anything except the code still
exactly as submitted". Hashing every key of the .ipynb EXCEPT cell `source`
gives 60242e116d12549d11c93f7f before the first edit and after the last one,
across four passes and a `ruff format`. The file also round-trips byte-for-byte
through json.dumps(indent=1, ensure_ascii=False), which is what made editing
cells programmatically safe rather than a reformat of all 141 of them; the
diff touches 38.

print() became sys.stdout.write() rather than logging. In a notebook the print
IS the output, and a logger with no handler configured would have silently
emptied every cell -- the exact regression scripts/check_refactor_safety.py
was written for. matprint's `print(x, end="  ")` and bare `print()` are
preserved as writes with and without a trailing newline.

Both `show_samples` and `show_samples_rgb` took (indices, preds, images,
labels, count, names), which is six arguments and a mutable default. They now
take a frozen `Samples` holding the last four, which also removed the same
four-value tuple from seven call sites. Reviewing that diff is how I caught
that I had transcribed `real_score = preds[sample][real_id]` as
`samples.labels[sample]` -- a bug I introduced and then fixed before commit.

Other real things, not just rule satisfaction: `np.in1d` is removed in NumPy 2
(NPY201) and is now `np.isin(...).ravel()`; `test_model` computed
correct_indices and incorrect_indices and used neither; two cells grew `img`
inside a loop so it was only bound by the `else` branch; and the Quick, Draw!
downloader now builds a URL whose https scheme is a literal, which is what
S310 was asking for.

The README claimed the .py was "an nbconvert export of that notebook". It is
an export of the CIFAR-10 section only -- 28 of 81 code cells -- and has been
edited since. It says that now, and says the two are separate artifacts rather
than one being regenerable from the other.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 08:10:31 +02:00

2745 lines
81 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "c9tFwxET8YY5"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"# gjfklg"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "M9PoJm628YY_"
},
"source": [
"# Introduction to Deep Learning with Keras and TensorFlow\n",
"\n",
"Based on excelent work by **[Daniel Moser (UT Southwestern Medical Center)](https://github.com/AviatorMoser/keras-mnist-tutorial)**, Resources: **[Xavier Snelgrove](https://github.com/wxs/keras-mnist-tutorial), [Yash Katariya](https://github.com/yashk2810/MNIST-Keras)**"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lh0dsQg08YZA"
},
"source": [
"To help you understand the fundamentals of deep learning, this demo will walk through the basic steps of building two toy models for classifying handwritten numbers with accuracies surpassing 95%. The first model will be a basic fully-connected neural network, and the second model will be a deeper network that introduces the concepts of convolution and pooling."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LJVXVJ6_8YZB"
},
"source": [
"## The Task for the AI\n",
"\n",
"Our goal is to construct and train an artificial neural network on thousands of images of handwritten digits so that it may successfully identify others when presented. The data that will be incorporated is the MNIST database which contains 60,000 images for training and 10,000 test images. We will use the Keras Python API with TensorFlow as the backend."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3eVu55U98YZC"
},
"source": [
"<img src=\"https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/mnist.png\" >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ms_a4hzY8YZC"
},
"source": [
"## Prerequisite Python Modules\n",
"\n",
"First, some software needs to be loaded into the Python environment."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "D7Bns2Lo8YZD",
"outputId": "be00cd49-743b-4aec-97f5-18df00e73a98"
},
"outputs": [
{
"ename": "ModuleNotFoundError",
"evalue": "No module named 'tensorflow'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[1], line 5\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mmatplotlib\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mpyplot\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mplt\u001b[39;00m \u001b[38;5;66;03m# MATLAB like plotting routines\u001b[39;00m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mrandom\u001b[39;00m \u001b[38;5;66;03m# for generating random numbers\u001b[39;00m\n\u001b[0;32m----> 5\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mtensorflow\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mtf\u001b[39;00m\n\u001b[1;32m 6\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mkeras\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mdatasets\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m mnist \u001b[38;5;66;03m# MNIST dataset is included in Keras%\u001b[39;00m\n\u001b[1;32m 7\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mkeras\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmodels\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m Sequential \u001b[38;5;66;03m# Model type to be used\u001b[39;00m\n",
"\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow'"
]
}
],
"source": [
"import itertools\n",
"import random # for generating random numbers\n",
"import sys\n",
"import urllib.request\n",
"from dataclasses import dataclass\n",
"from pathlib import Path\n",
"\n",
"import matplotlib.pyplot as plt # MATLAB like plotting routines\n",
"import numpy as np # advanced math library\n",
"import seaborn as sns\n",
"import tensorflow as tf\n",
"from keras import Model\n",
"from keras.datasets import mnist # MNIST dataset is included in Keras%\n",
"from keras.layers import (\n",
" Activation,\n",
" Dense,\n",
" Dropout,\n",
") # Types of layers to be used in our model\n",
"from keras.models import Sequential # Model type to be used\n",
"from keras.utils import to_categorical # NumPy related tools\n",
"from matplotlib import patheffects\n",
"from sklearn.manifold import TSNE\n",
"from sklearn.metrics import confusion_matrix\n",
"from sklearn.preprocessing import OneHotEncoder\n",
"\n",
"# print() would need a logging handler configured to survive this file being\n",
"# exported and run as a script; sys.stdout.write shows the same text in a cell.\n",
"sys.stdout.write(f\"{tf.__version__}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QygDQ7Ch8YZG"
},
"source": [
"## Loading Training Data\n",
"\n",
"The MNIST dataset is conveniently bundled within Keras, and we can easily analyze some of its features in Python."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "SdZZph6i8YZG",
"outputId": "0bccfb2e-0ac0-4e45-8a2b-879ee5c6860b"
},
"outputs": [],
"source": [
"# The MNIST data is split between 60,000 28 x 28 pixel training images\n",
"# and 10,000 28 x 28 pixel images\n",
"(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
"\n",
"sys.stdout.write(f\"X_train shape {X_train.shape}\\n\")\n",
"sys.stdout.write(f\"y_train shape {y_train.shape}\\n\")\n",
"sys.stdout.write(f\"X_test shape {X_test.shape}\\n\")\n",
"sys.stdout.write(f\"y_test shape {y_test.shape}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9v5sjP468YZJ"
},
"source": [
"Using matplotlib, we can plot some sample images from the training set directly into this Jupyter Notebook. We can egzamine the interclass variability - how many different ways of writing the same digit there are!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 811
},
"id": "Ov1T_A6X8YZJ",
"outputId": "5527cbf1-7ab1-4035-ef38-1fbaad1a425c"
},
"outputs": [],
"source": [
"plt.rcParams[\"figure.figsize\"] = (9, 9) # Make the figures a bit bigger\n",
"\n",
"CLASS_COUNT = 10\n",
"\n",
"\n",
"def visualize_classes(images: np.ndarray, labels: np.ndarray) -> None:\n",
" \"\"\"Show ten samples of each class as one wide strip.\n",
"\n",
" The strips are collected in a list and concatenated once. Growing the\n",
" result in place only bound it in the loop's `else` branch, which reads as\n",
" a use-before-assignment and is what ruff flagged.\n",
" \"\"\"\n",
" strips = []\n",
" for class_id in range(CLASS_COUNT):\n",
" img_batch = images[labels == class_id][0:10]\n",
" strips.append(\n",
" np.reshape(\n",
" img_batch,\n",
" (img_batch.shape[0] * img_batch.shape[1], img_batch.shape[2]),\n",
" )\n",
" )\n",
" plt.figure(figsize=(10, 20))\n",
" plt.axis(\"off\")\n",
" plt.imshow(np.concatenate(strips, axis=1), cmap=\"gray\")\n",
"\n",
"\n",
"visualize_classes(X_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uGjX5ZD68YZM"
},
"source": [
"Let's examine a single digit a little closer, and print out the array representing the last digit."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "wW_rrJfj8YZN",
"outputId": "58260b7d-47fd-4fcd-dd49-92d73823b9d5"
},
"outputs": [],
"source": [
"# just a little function for pretty printing a matrix\n",
"def matprint(mat: np.ndarray, fmt: str = \"g\") -> None:\n",
" \"\"\"Print `mat` with every column padded to its own widest entry.\"\"\"\n",
" col_maxes = [max([len((\"{:\" + fmt + \"}\").format(x)) for x in col]) for col in mat.T]\n",
" for row in mat:\n",
" for i, value in enumerate(row):\n",
" # Two trailing spaces and no newline: this builds one line per row.\n",
" sys.stdout.write((\"{:\" + str(col_maxes[i]) + fmt + \"} \").format(value))\n",
" sys.stdout.write(\"\\n\")\n",
"\n",
"\n",
"# now print!\n",
"matprint(X_train[-1])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "s3koPAgMdzoG"
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "vQPPE2Bq8YZP"
},
"source": [
"Each pixel is an 8-bit integer from 0-255. 0 is full black, while 255 is full white. This what we call a single-channel pixel. It's called monochrome.\n",
"\n",
"*Fun-fact! Your computer screen has three channels for each pixel: red, green, blue. Each of these channels also likely takes an 8-bit integer. 3 channels -- 24 bits total -- 16,777,216 possible colors!*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z09IKy7PR9M9"
},
"source": [
"## More input data nalysis\n",
"\n",
"\n",
"Visualizing high-dimensional data by projecting it into a low-dimensional space is a classic operation that anyone working with data has probably done at least once in their life. There are a huge variety of methods for reducing dimensionality, but one very popular method is t-SNE, a method proposed by Geoffry Hinton’s group back in 2008. [ _more..._ ](https://mlexplained.com/2018/09/14/paper-dissected-visualizing-data-using-t-sne-explained/)\n",
"\n",
"Code by [Zaid Alyafeai](https://github.com/zaidalyafeai/Notebooks) [MIT license]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "59m4-pjCSBgM"
},
"outputs": [],
"source": [
"RS = 19238\n",
"\n",
"class_names = [str(clid) for clid in range(CLASS_COUNT)]\n",
"\n",
"\n",
"def scatter(points: np.ndarray, colors: np.ndarray) -> None:\n",
" \"\"\"Scatter 2D `points` coloured by class, with each class labelled once.\"\"\"\n",
" # We choose a color palette with seaborn.\n",
" palette = np.array(sns.color_palette(\"hls\", CLASS_COUNT))\n",
"\n",
" # We create a scatter plot.\n",
" plt.figure(figsize=(8, 8))\n",
" ax = plt.subplot(aspect=\"equal\")\n",
" ax.scatter(points[:, 0], points[:, 1], lw=0, s=40, c=palette[colors.astype(\"int\")])\n",
" plt.xlim(-25, 25)\n",
" plt.ylim(-25, 25)\n",
" ax.axis(\"off\")\n",
" ax.axis(\"tight\")\n",
"\n",
" # We add the labels for each digit.\n",
" for i in range(CLASS_COUNT):\n",
" # Position of each label.\n",
" xtext, ytext = np.median(points[colors == i, :], axis=0)\n",
" txt = ax.text(xtext, ytext, class_names[i], fontsize=15)\n",
" txt.set_path_effects(\n",
" [patheffects.Stroke(linewidth=5, foreground=\"w\"), patheffects.Normal()]\n",
" )\n",
"\n",
"\n",
"def plot_tsne(features: np.ndarray, labels: np.ndarray) -> None:\n",
" \"\"\"Project `features` to two dimensions with t-SNE and scatter them.\"\"\"\n",
" sys.stdout.write(\"calculating tsne ...\\n\")\n",
" proj = TSNE(random_state=RS, learning_rate=200, init=\"random\").fit_transform(\n",
" features\n",
" )\n",
" scatter(proj, labels)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 670
},
"id": "1_gbsFQ3Srw_",
"outputId": "175c8344-ebf4-4900-b702-bbf9072af429"
},
"outputs": [],
"source": [
"X = np.reshape(X_train, (X_train.shape[0], 28 * 28))[0:2000]\n",
"y = y_train[0:2000]\n",
"plot_tsne(X, y)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UFWr5nQo8YZQ"
},
"source": [
"## Formatting the input data layer\n",
"\n",
"Instead of a 28 x 28 matrix, we build our network to accept a 784-length vector.\n",
"\n",
"Each image needs to be then reshaped (or flattened) into a vector. We'll also normalize the inputs to be in the range [0-1] rather than [0-255]. Normalizing inputs is generally recommended, so that any additional dimensions (for other network architectures) are of the same scale."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "B-b0XGKd8YZQ"
},
"source": [
"<img src='https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/flatten.png' >"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0ge5s-pT8YZR",
"outputId": "5695385b-3845-4ced-ab70-2d085a0417e2"
},
"outputs": [],
"source": [
"X_train = X_train.reshape(\n",
" 60000, 784\n",
") # reshape 60,000 28 x 28 matrices into 60,000 784-length vectors.\n",
"X_test = X_test.reshape(\n",
" 10000, 784\n",
") # reshape 10,000 28 x 28 matrices into 10,000 784-length vectors.\n",
"\n",
"X_train = X_train.astype(\"float32\") # change integers to 32-bit floating point numbers\n",
"X_test = X_test.astype(\"float32\")\n",
"\n",
"X_train /= (\n",
" 255 # normalize each value for each pixel for the entire vector for each input\n",
")\n",
"X_test /= 255\n",
"\n",
"sys.stdout.write(f\"Training matrix shape {X_train.shape}\\n\")\n",
"sys.stdout.write(f\"Testing matrix shape {X_test.shape}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sS8jHU6V8YZT"
},
"source": [
"We then modify our classes (unique digits) to be in the one-hot (categorical) format, i.e.\n",
"\n",
"```\n",
"0 -> [1, 0, 0, 0, 0, 0, 0, 0, 0]\n",
"1 -> [0, 1, 0, 0, 0, 0, 0, 0, 0]\n",
"2 -> [0, 0, 1, 0, 0, 0, 0, 0, 0]\n",
"etc.\n",
"```\n",
"\n",
"If the final output of our network is very close to one of these classes, then it is most likely that class. For example, if the final output is:\n",
"\n",
"```\n",
"[0, 0.94, 0, 0, 0, 0, 0.06, 0, 0]\n",
"```\n",
"then it is most probable that the image is that of the digit `1`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AFvPkCFw8YZU"
},
"outputs": [],
"source": [
"nb_classes = 10 # number of unique digits\n",
"\n",
"Y_train = to_categorical(y_train, nb_classes)\n",
"Y_test = to_categorical(y_test, nb_classes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "q_B8ZH5G8YZX"
},
"source": [
"# Building a 3-layer fully connected network\n",
"\n",
"<img src=\"https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/figure.png\" />"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "M-uOWlRv8YZX"
},
"outputs": [],
"source": [
"# The Sequential model is a linear stack of layers and is very common.\n",
"\n",
"model = Sequential()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iyoUd9WE8YZa"
},
"source": [
"## The first hidden layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "IdgP4lai8YZb"
},
"outputs": [],
"source": [
"# The first hidden layer is a set of 512 nodes (artificial neurons).\n",
"# Each node will receive an element from each input vector and apply some\n",
"# weight and bias to it.\n",
"\n",
"model.add(\n",
" Dense(512, input_shape=(784,))\n",
") # (784,) is not a typo -- that represents a 784 length vector!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "c1MiY3u68YZd"
},
"outputs": [],
"source": [
"# An \"activation\" is a non-linear function applied to the output of the layer\n",
"# above. It checks the new value of the node, and decides whether that\n",
"# artifical neuron has fired.\n",
"# The Rectified Linear Unit (ReLU) converts all negative inputs to nodes in the\n",
"# next layer to be zero. Those inputs are then not considered to be fired.\n",
"# Positive values of a node are unchanged.\n",
"\n",
"model.add(Activation(\"relu\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tQCRmiMI8YZg"
},
"source": [
"$$f(x) = max (0,x)$$\n",
"<img src = 'https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/relu.jpg' >"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kudeX_kN8YZg"
},
"outputs": [],
"source": [
"# Dropout zeroes a selection of random outputs (i.e., disables their activation)\n",
"# Dropout helps protect the model from memorizing or \"overfitting\" the training data.\n",
"model.add(Dropout(0.2))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PY5XiXYc8YZj"
},
"source": [
"## Adding the second hidden layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Tnz-mpIS8YZj"
},
"outputs": [],
"source": [
"# The second hidden layer appears identical to our first layer.\n",
"# However, instead of each of the 512-node receiving 784-inputs from the input\n",
"# image data, they receive 512 inputs from the output of the first 512-node\n",
"# layer.\n",
"\n",
"model.add(Dense(512))\n",
"model.add(Activation(\"relu\"))\n",
"model.add(Dropout(0.2))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vkeHSS0h8YZm"
},
"source": [
"## The Final Output Layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cndhIgBK8YZm"
},
"outputs": [],
"source": [
"# The final layer of 10 neurons in fully-connected to the previous 512-node\n",
"# layer. The final layer of a FCN should be equal to the number of desired\n",
"# classes (10 in this case).\n",
"model.add(Dense(CLASS_COUNT))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "02Zk2VWL8YZp"
},
"outputs": [],
"source": [
"# The \"softmax\" activation represents a probability distribution over K\n",
"# different possible outcomes. Its values are all non-negative and sum to 1.\n",
"\n",
"model.add(Activation(\"softmax\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "YReJLqWL8YZr",
"outputId": "80207ffb-cf67-42a5-decb-25812126daa3"
},
"outputs": [],
"source": [
"# Summarize the built model\n",
"\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UcndsYyk8YZv"
},
"source": [
"## Compiling the model\n",
"\n",
"Keras is built on top of Theano and TensorFlow. Both packages allow you to define a *computation graph* in Python, which then compiles and runs efficiently on the CPU or GPU without the overhead of the Python interpreter.\n",
"\n",
"When compiing a model, Keras asks you to specify your **loss function** and your **optimizer**. The loss function we'll use here is called *categorical cross-entropy*, and is a loss function well-suited to comparing two probability distributions.\n",
"\n",
"Our predictions are probability distributions across the ten different digits (e.g. \"we're 80% confident this image is a 3, 10% sure it's an 8, 5% it's a 2, etc.\"), and the target is a probability distribution with 100% for the correct category, and 0 for everything else. The cross-entropy is a measure of how different your predicted distribution is from the target distribution. [More detail at Wikipedia](https://en.wikipedia.org/wiki/Cross_entropy)\n",
"\n",
"The optimizer helps determine how quickly the model learns through **gradient descent**. The rate at which descends a gradient is called the **learning rate**."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Wsv1Z9LC8YZw"
},
"source": [
"<img src = \"https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/gradient_descent.png\" >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vZWeBx-58YZx"
},
"source": [
"<img src = \"https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/learning_rate.png\" >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ptVA6aSg8YZy"
},
"source": [
"So are smaller learning rates better? Not quite! It's important for an optimizer not to get stuck in local minima while neglecting the global minimum of the loss function. Sometimes that means trying a larger learning rate to jump out of a local minimum."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zSGiajLT8YZ1"
},
"source": [
"<img src = 'https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/complicated_loss_function.png' >"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7RUmQxfp8YZ2"
},
"outputs": [],
"source": [
"# Let's use the Adam optimizer for learning\n",
"adam = tf.optimizers.Adam(learning_rate=0.001)\n",
"model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d-u8EqW48YZ3"
},
"source": [
"## Train the model!\n",
"This is the fun part!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rTynqX-H8YZ4"
},
"source": [
"The batch size determines over how much data per step is used to compute the loss function, gradients, and back propagation. Large batch sizes allow the network to complete it's training faster; however, there are other factors beyond training speed to consider.\n",
"\n",
"Too large of a batch size smoothes the local minima of the loss function, causing the optimizer to settle in one because it thinks it found the global minimum.\n",
"\n",
"Too small of a batch size creates a very noisy loss function, and the optimizer may never find the global minimum.\n",
"\n",
"So a good batch size may take some trial and error to find!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "MxhTSlBI8YZ5",
"outputId": "1428ff5e-d1c9-4297-c9db-d01e17797af3"
},
"outputs": [],
"source": [
"model.fit(X_train, Y_train, batch_size=128, epochs=5, verbose=1)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YufwyCC38YZ6"
},
"source": [
"The two numbers, in order, represent the value of the loss function of the network on the training set, and the overall accuracy of the network on the training data. But how does it do on data it did not train on?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wQORnbe08YZ7"
},
"source": [
"## Evaluate Model's Accuracy on Test Data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "SX3n6p108YZ7",
"outputId": "882d8060-ff99-4802-b0c4-18ce6542bf33"
},
"outputs": [],
"source": [
"score = model.evaluate(X_test, Y_test)\n",
"sys.stdout.write(f\"Test score: {score[0]}\\n\")\n",
"sys.stdout.write(f\"Test accuracy: {score[1]}\\n\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NYD0_NjusHRL"
},
"outputs": [],
"source": [
"def plot_confusion_matrix(\n",
" cm: np.ndarray,\n",
" classes: list[str],\n",
" *,\n",
" normalize: bool = False,\n",
" title: str = \"Confusion matrix\",\n",
" cmap: object = plt.cm.Blues,\n",
") -> None:\n",
" \"\"\"Print and plot the confusion matrix.\n",
"\n",
" Normalization can be applied by setting `normalize=True`.\n",
" \"\"\"\n",
" if normalize:\n",
" cm = cm.astype(\"float\") / cm.sum(axis=1)[:, np.newaxis]\n",
" sys.stdout.write(\"Normalized confusion matrix\\n\")\n",
" else:\n",
" sys.stdout.write(\"Confusion matrix, without normalization\\n\")\n",
"\n",
" sys.stdout.write(f\"{cm}\\n\")\n",
"\n",
" plt.imshow(cm, interpolation=\"nearest\", cmap=cmap)\n",
" plt.title(title)\n",
" plt.colorbar()\n",
" tick_marks = np.arange(len(classes))\n",
" plt.xticks(tick_marks, classes, rotation=45)\n",
" plt.yticks(tick_marks, classes)\n",
"\n",
" fmt = \".2f\" if normalize else \"d\"\n",
" thresh = cm.max() / 2.0\n",
" for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
" plt.text(\n",
" j,\n",
" i,\n",
" format(cm[i, j], fmt),\n",
" horizontalalignment=\"center\",\n",
" color=\"white\" if cm[i, j] > thresh else \"black\",\n",
" )\n",
"\n",
" plt.ylabel(\"True label\")\n",
" plt.xlabel(\"Predicted label\")\n",
" plt.tight_layout()\n",
"\n",
"\n",
"@dataclass(frozen=True)\n",
"class Samples:\n",
" \"\"\"A model's output, with the images and true labels it was scored on.\"\"\"\n",
"\n",
" scores: np.ndarray\n",
" images: np.ndarray\n",
" labels: np.ndarray\n",
" names: list[str] | None = None"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yWaT13H78YZ9"
},
"source": [
"### Inspecting the output\n",
"\n",
"It's always a good idea to inspect the output and make sure everything looks sane. Here we'll look at some examples it gets right, and some examples it gets wrong."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "LFECu0nE8YZ-",
"outputId": "b16aba6d-811f-47b2-a0db-e53489fe5df1"
},
"outputs": [],
"source": [
"# The predict_classes function outputs the highest probability class\n",
"# according to the trained classifier for each input example.\n",
"predict = model.predict(X_test)\n",
"predicted_classes = np.argmax(predict, axis=1)\n",
"\n",
"# Check which items we got right / wrong\n",
"correct_indices = np.nonzero(predicted_classes == y_test)[0]\n",
"incorrect_indices = np.nonzero(predicted_classes != y_test)[0]\n",
"cnf_matrix = confusion_matrix(y_test, predicted_classes)\n",
"\n",
"class_names = [str(i) for i in range(10)]\n",
"\n",
"# Plot non-normalized confusion matrix\n",
"plt.figure()\n",
"plot_confusion_matrix(\n",
" cnf_matrix, classes=class_names, title=\"Confusion matrix, without normalization\"\n",
")\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "d4tL55qnBfAA"
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ct14A0rsD-Ja"
},
"source": [
"**Correct predictions**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 906
},
"id": "ew6oWSoY8YaA",
"outputId": "8de14653-7da4-4120-96a1-b823954df616"
},
"outputs": [],
"source": [
"def show_samples(indices: np.ndarray, samples: Samples, *, count: int = 3) -> None:\n",
" \"\"\"Plot a `count` x `count` grid of digits, titled predicted vs expected.\"\"\"\n",
" plt.figure()\n",
" for i, sample in enumerate(indices[: count**2]):\n",
" pred_id = int(np.argmax(samples.scores[sample]))\n",
" real_id = int(samples.labels[sample])\n",
" pred_score = samples.scores[sample][pred_id]\n",
" real_score = samples.scores[sample][real_id]\n",
" plt.subplot(count, count, i + 1)\n",
" plt.imshow(\n",
" samples.images[sample].reshape(28, 28), cmap=\"gray\", interpolation=\"none\"\n",
" )\n",
" plt.axis(\"off\")\n",
" if samples.names:\n",
" predicted_label: str | int = samples.names[pred_id]\n",
" real_label: str | int = samples.names[real_id]\n",
" else:\n",
" predicted_label = pred_id\n",
" real_label = real_id\n",
" plt.title(\n",
" f\"P: {predicted_label} ({pred_score:.2f})\\n\"\n",
" f\"E: {real_label} ({real_score:.2f})\"\n",
" )\n",
"\n",
" plt.tight_layout()\n",
"\n",
"\n",
"show_samples(correct_indices, Samples(predict, X_test, y_test), count=5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fhZp92ZxEGn4"
},
"source": [
"**Wrong predictions**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 906
},
"id": "FrAIwG1CEFxe",
"outputId": "bd8f88cd-5a58-4fce-8674-4a6ceb9d4d2e"
},
"outputs": [],
"source": [
"show_samples(incorrect_indices, Samples(predict, X_test, y_test), count=5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6ZzALozi8YaD"
},
"source": [
"# Try experimenting with the batch size!\n",
"\n",
" * How does increasing the batch size to 10,000 affect the training time and test accuracy?\n",
" * How about a batch size of 32?\n",
" * Is there any difference in results between the students? If so - why?\n",
" * Experiment with the learning rate in the optimizer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uPUgO1BY8YaD"
},
"source": [
"# Introducing Convolution! What is it?"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vs75ypBn8YaE"
},
"source": [
"Before, we built a network that accepts the normalized pixel values of each value and operates soley on those values. What if we could instead feed different features (e.g. **curvature, edges**) of each image into a network, and have the network learn which features are important for classifying an image?\n",
"\n",
"This possible through convolution! Convolution applies **kernels** (filters) that traverse through each image and generate **feature maps**."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eVIKBobD8YaF"
},
"source": [
"<img src = 'https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/convolution.gif' >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gYJDfK4p8YaG"
},
"source": [
"In the above example, the image is a 5 x 5 matrix and the kernel going over it is a 3 x 3 matrix. A dot product operation takes place between the image and the kernel and the convolved feature is generated. Each kernel in a CNN learns a different characteristic of an image.\n",
"\n",
"Kernels are often used in photoediting software to apply blurring, edge detection, sharpening, etc."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qm9CKpnr8YaH"
},
"source": [
"<img src = 'https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/kernels.png' >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QdHIoa-E8YaH"
},
"source": [
"Kernels in deep learning networks are used in similar ways, i.e. highlighting some feature. Combined with a system called **max pooling**, the non-highlighted elements are discarded from each feature map, leaving only the features of interest, reducing the number of learned parameters, and decreasing the computational cost (e.g. system memory)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ceCEMpHf8YaI"
},
"source": [
"<img src = 'https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/max_pooling.png' >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "poGu943u8YaJ"
},
"source": [
"We can also take convolutions of convolutions -- we can stack as many convolutions as we want, as long as there are enough pixels to fit a kernel.\n",
"\n",
"*Warning: What you may find down there in those deep convolutions may not appear recognizable to you.*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "u7KmEU3m8YaJ"
},
"source": [
"<img src = 'https://github.com/wut-mpg/keras-mnist-tutorial/raw/master/go_deeper.jpg' >"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "h88R-Bsa8YaK"
},
"source": [
"## Building a \"Deep\" Convolutional Neural Network"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qoXIbzAG8YaL"
},
"outputs": [],
"source": [
"# import some additional tools\n",
"\n",
"from keras.layers import (\n",
" BatchNormalization,\n",
" Conv2D,\n",
" Flatten,\n",
" MaxPooling2D,\n",
")\n",
"from keras.preprocessing.image import ImageDataGenerator"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "w0MBTlCL8YaM"
},
"outputs": [],
"source": [
"# Reload the MNIST data\n",
"(X_train, y_train), (X_test, y_test) = mnist.load_data()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "TbBjjLil8YaO",
"outputId": "f6cf7711-1959-4d80-89f9-5769b86000ca"
},
"outputs": [],
"source": [
"# Again, do some formatting\n",
"# Except we do not flatten each image into a 784-length vector because we want\n",
"# to perform convolutions first\n",
"\n",
"X_train = X_train.reshape(\n",
" 60000, 28, 28, 1\n",
") # add an additional dimension to represent the single-channel\n",
"X_test = X_test.reshape(10000, 28, 28, 1)\n",
"\n",
"X_train = X_train.astype(\"float32\") # change integers to 32-bit floating point numbers\n",
"X_test = X_test.astype(\"float32\")\n",
"\n",
"X_train /= (\n",
" 255 # normalize each value for each pixel for the entire vector for each input\n",
")\n",
"X_test /= 255\n",
"\n",
"sys.stdout.write(f\"Training matrix shape {X_train.shape}\\n\")\n",
"sys.stdout.write(f\"Testing matrix shape {X_test.shape}\\n\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "aNNglEKV8YaR"
},
"outputs": [],
"source": [
"# one-hot format classes\n",
"\n",
"nb_classes = 10 # number of unique digits\n",
"\n",
"Y_train = to_categorical(y_train, nb_classes)\n",
"Y_test = to_categorical(y_test, nb_classes)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "buX5gLwP8YaT"
},
"outputs": [],
"source": [
"model = Sequential() # Linear stacking of layers\n",
"\n",
"# Convolution Layer 1\n",
"model.add(\n",
" Conv2D(16, (5, 5), input_shape=(28, 28, 1))\n",
") # 16 different 5x5 kernels -- so 16 feature maps\n",
"model.add(Activation(\"relu\")) # activation\n",
"model.add(MaxPooling2D(pool_size=(2, 2))) # Pool the max values over a 2x2 kernel\n",
"\n",
"# Convolution Layer 2\n",
"model.add(Conv2D(32, (5, 5))) # 32 different 5x5 kernels -- so 32 feature maps\n",
"model.add(Activation(\"relu\")) # activation\n",
"model.add(MaxPooling2D(pool_size=(2, 2))) # Pool the max values over a 2x2 kernel\n",
"\n",
"model.add(Flatten()) # Flatten final output matrix into a vector\n",
"\n",
"# Fully Connected Layer\n",
"model.add(Dense(128)) # 128 FC nodes\n",
"model.add(Activation(\"relu\")) # activation\n",
"\n",
"# Fully Connected Layer\n",
"model.add(Dense(10)) # final 10 FC nodes\n",
"model.add(Activation(\"softmax\")) # softmax activation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "srtd-OZV8YaV",
"outputId": "c9607879-bd3d-4480-b26b-dc5e916ef3e5"
},
"outputs": [],
"source": [
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "TJ5vnJet8YaX"
},
"outputs": [],
"source": [
"# we'll use the same optimizer\n",
"adam = tf.optimizers.Adam(learning_rate=0.001)\n",
"model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WShqV9h28Yaa"
},
"outputs": [],
"source": [
"# data augmentation prevents overfitting by slightly changing the data randomly\n",
"# Keras has a great built-in feature to do automatic augmentation\n",
"\n",
"gen = ImageDataGenerator(\n",
" rotation_range=8,\n",
" width_shift_range=0.08,\n",
" shear_range=0.3,\n",
" height_shift_range=0.08,\n",
" zoom_range=0.08,\n",
" validation_split=0.2,\n",
")\n",
"\n",
"test_gen = ImageDataGenerator()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VsGRzqqd8Yab"
},
"outputs": [],
"source": [
"# We can then feed our augmented data in batches\n",
"# Besides loss function considerations as before, this method actually results\n",
"# in significant memory savings because we are actually LOADING the data into\n",
"# the network in batches before processing each batch\n",
"\n",
"# Before the data was all loaded into memory, but then processed in batches.\n",
"\n",
"train_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"training\")\n",
"valid_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"validation\")\n",
"test_generator = test_gen.flow(X_test, Y_test, batch_size=128)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_DXSGa-z8Yae",
"outputId": "47a5dfcf-fa1b-4de4-fc02-a2f388169490"
},
"outputs": [],
"source": [
"# We can now train our model which is fed data by our batch loader\n",
"# Steps per epoch should always be total size of the set divided by the batch size\n",
"\n",
"# SIGNIFICANT MEMORY SAVINGS (important for larger, deeper networks)\n",
"\n",
"model.fit(\n",
" train_generator,\n",
" steps_per_epoch=48000 // 128,\n",
" epochs=5,\n",
" verbose=1,\n",
" validation_data=valid_generator,\n",
" validation_steps=12000 // 128,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "6YZaV3U-8Yah",
"outputId": "291dcaae-95b9-4db3-ee3a-f472e90ebcda"
},
"outputs": [],
"source": [
"score = model.evaluate(X_test, Y_test)\n",
"sys.stdout.write(f\"Test score: {score[0]}\\n\")\n",
"sys.stdout.write(f\"Test accuracy: {score[1]}\\n\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "WqvF3eS2pnLp",
"outputId": "0594401a-4050-42fa-e198-6912ccad62cc"
},
"outputs": [],
"source": [
"# The predict_classes function outputs the highest probability class\n",
"# according to the trained classifier for each input example.\n",
"predict = model.predict(X_test)\n",
"predicted_classes = np.argmax(predict, axis=1)\n",
"\n",
"# Check which items we got right / wrong\n",
"correct_indices = np.nonzero(predicted_classes == y_test)[0]\n",
"\n",
"incorrect_indices = np.nonzero(predicted_classes != y_test)[0]\n",
"\n",
"\n",
"cnf_matrix = confusion_matrix(y_test, predicted_classes)\n",
"\n",
"class_names = [str(i) for i in range(10)]\n",
"\n",
"# Plot non-normalized confusion matrix\n",
"plt.figure()\n",
"plot_confusion_matrix(\n",
" cnf_matrix, classes=class_names, title=\"Confusion matrix, without normalization\"\n",
")\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 906
},
"id": "Wn7FDt7DqSKB",
"outputId": "b0ad9e24-9992-4ec4-c920-3d5a88c3941c"
},
"outputs": [],
"source": [
"show_samples(correct_indices, Samples(predict, X_test, y_test), count=5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 906
},
"id": "r0m_gom9qL3o",
"outputId": "999960b7-d8cd-4760-8aab-d301af027ac7"
},
"outputs": [],
"source": [
"show_samples(incorrect_indices, Samples(predict, X_test, y_test), count=5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RsY5uPgU8Yak"
},
"source": [
"## Great results!\n",
"\n",
"But wouldn't it be nice if we could visualize those convolutions so that we can see what the model is seeing?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BgbunToX8Yal"
},
"outputs": [],
"source": [
"def visualize2(\n",
" model: Sequential, layer: object, img: np.ndarray, r: int, c: int\n",
") -> None:\n",
" \"\"\"Plot the `r` x `c` feature maps `layer` produces for a single image.\"\"\"\n",
" # expand dimensions so that it represents a single 'sample'\n",
" img = np.expand_dims(img, axis=0)\n",
" outputs = [layer.output]\n",
" feature_model = Model(inputs=model.inputs, outputs=outputs)\n",
" # get feature map for first hidden layer\n",
" feature_maps = feature_model.predict(img)\n",
" for fmap in feature_maps:\n",
" # plot all 64 maps in an 8x8 squares\n",
" ix = 1\n",
" for _ in range(r):\n",
" for _ in range(c):\n",
" # specify subplot and turn of axis\n",
" ax = plt.subplot(r, c, ix)\n",
" ax.set_xticks([])\n",
" ax.set_yticks([])\n",
" # plot filter channel in grayscale\n",
" plt.imshow(fmap[:, :, ix - 1], cmap=\"gray\")\n",
" ix += 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 770
},
"id": "ZiDemlgp8Yan",
"outputId": "6255c19a-48d8-4589-859a-bf7743f7a0e0"
},
"outputs": [],
"source": [
"plt.figure()\n",
"img = X_test[11]\n",
"plt.imshow(img[:, :, 0], cmap=\"gray\", interpolation=\"none\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 747
},
"id": "UklZv03C8Yao",
"outputId": "6795a13c-d664-4336-a4ff-b46374dbe45b"
},
"outputs": [],
"source": [
"visualize2(model, model.layers[1], img, 4, 4)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 670
},
"id": "gt8S9bzR8Yar",
"outputId": "938b929e-7cdc-4deb-f99f-096a75acf35a"
},
"outputs": [],
"source": [
"visualize2(model, model.layers[4], img, 4, 8)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "okUztZOd79by"
},
"source": [
"# Transfer learning"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8MRwLZgGhK-K"
},
"source": [
"## Funkcje pomocnicze"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1AIiBfX78luv"
},
"outputs": [],
"source": [
"# Loading the specified subset of digits from the MNIST dataset\n",
"def get_mnist(\n",
" digits: list[int],\n",
") -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n",
" \"\"\"Reload MNIST keeping only `digits`, scaled to [0, 1] with a channel axis.\"\"\"\n",
" # Reload the MNIST data\n",
" (x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
"\n",
" train_ids = np.isin(y_train, digits).ravel()\n",
" test_ids = np.isin(y_test, digits).ravel()\n",
" x_train = x_train[train_ids]\n",
" x_test = x_test[test_ids]\n",
" y_train = y_train[train_ids]\n",
" y_test = y_test[test_ids]\n",
"\n",
" x_train = np.expand_dims(x_train, axis=-1)\n",
" x_test = np.expand_dims(x_test, axis=-1)\n",
"\n",
" x_train = x_train.astype(\n",
" \"float32\"\n",
" ) # change integers to 32-bit floating point numbers\n",
" x_test = x_test.astype(\"float32\")\n",
"\n",
" x_train /= (\n",
" 255 # normalize each value for each pixel for the entire vector for each input\n",
" )\n",
" x_test /= 255\n",
"\n",
" return x_train, y_train, x_test, y_test"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Zf0T6tdPBC70"
},
"outputs": [],
"source": [
"# Preparing the base model\n",
"def prepare_model(input_shape: tuple[int, ...], class_count: int) -> Sequential:\n",
" \"\"\"Build the three-convolution-block CNN this section reuses.\"\"\"\n",
" model = Sequential() # Linear stacking of layers\n",
"\n",
" # Convolution Layer 1\n",
" model.add(Conv2D(16, (3, 3), padding=\"same\", input_shape=input_shape))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
" # Convolution Layer 2\n",
" model.add(Conv2D(32, (3, 3), padding=\"same\"))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
" # Convolution Layer 3\n",
" model.add(Conv2D(64, (3, 3), padding=\"same\"))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
" model.add(Flatten())\n",
"\n",
" # Fully Connected Layer\n",
" model.add(Dense(64))\n",
" model.add(Activation(\"relu\"))\n",
"\n",
" # Fully Connected Layer\n",
" model.add(Dense(class_count))\n",
" model.add(Activation(\"softmax\"))\n",
"\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "EX6bXUPBGDyx"
},
"outputs": [],
"source": [
"# One-hot encoding of non-consecutive labels\n",
"def one_hot(labels: np.ndarray) -> np.ndarray:\n",
" \"\"\"One-hot encode `labels`, which need not be 0..n-1.\"\"\"\n",
" encoder = OneHotEncoder(categories=\"auto\")\n",
" column = labels.reshape(-1, 1)\n",
" output = encoder.fit_transform(column)\n",
" return output.toarray()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-VP_9oFFiE6a"
},
"outputs": [],
"source": [
"def test_model(\n",
" model: Sequential,\n",
" x_test: np.ndarray,\n",
" y_test_onehot: np.ndarray,\n",
" y_test: np.ndarray,\n",
" digits: list[int],\n",
") -> None:\n",
" \"\"\"Score `model` on the test split and plot its confusion matrix.\"\"\"\n",
" score = model.evaluate(x_test, y_test_onehot)\n",
" sys.stdout.write(f\"Test score: {score[0]}\\n\")\n",
" sys.stdout.write(f\"Test accuracy: {score[1]}\\n\")\n",
"\n",
" # The predict_classes function outputs the highest probability class\n",
" # according to the trained classifier for each input example.\n",
" predicted = model.predict(x_test)\n",
" predicted_classes = np.argmax(predicted, axis=1)\n",
"\n",
" actual_classes = [digits[x] for x in predicted_classes]\n",
"\n",
" cnf_matrix = confusion_matrix(y_test, actual_classes)\n",
"\n",
" class_names = [str(i) for i in digits]\n",
"\n",
" # Plot non-normalized confusion matrix\n",
" plt.figure()\n",
" plot_confusion_matrix(\n",
" cnf_matrix, classes=class_names, title=\"Confusion matrix, without normalization\"\n",
" )\n",
"\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Fi7BwsX8h5Dm"
},
"source": [
"## Trening modelu bazowego: cyfry 0..4"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ZyAj3Kk4Bw4d",
"outputId": "5c383cb2-d515-40c1-a1f6-4f45f10c029b"
},
"outputs": [],
"source": [
"digits = [0, 1, 2, 3, 4]\n",
"\n",
"X_train, y_train, X_test, y_test = get_mnist(digits)\n",
"\n",
"Y_train = one_hot(y_train)\n",
"Y_test = one_hot(y_test)\n",
"\n",
"model = prepare_model((28, 28, 1), 5)\n",
"\n",
"adam = tf.optimizers.Adam(learning_rate=0.001)\n",
"model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])\n",
"\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BjEZRMnvChne"
},
"outputs": [],
"source": [
"gen = ImageDataGenerator(\n",
" rotation_range=8,\n",
" width_shift_range=0.08,\n",
" shear_range=0.3,\n",
" height_shift_range=0.08,\n",
" zoom_range=0.08,\n",
" validation_split=0.2,\n",
")\n",
"\n",
"test_gen = ImageDataGenerator()\n",
"\n",
"train_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"training\")\n",
"valid_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"validation\")\n",
"test_generator = test_gen.flow(X_test, Y_test, batch_size=128)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "byGKttIXDfE3",
"outputId": "b07c62a8-030a-4da4-b947-e97140bb796a"
},
"outputs": [],
"source": [
"model.fit_generator(\n",
" train_generator,\n",
" steps_per_epoch=25000 // 128,\n",
" epochs=5,\n",
" verbose=1,\n",
" validation_data=valid_generator,\n",
" validation_steps=5000 // 128,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "3sUGcowEMLHk",
"outputId": "7b1de02c-647a-4640-d247-fab34a416ef8"
},
"outputs": [],
"source": [
"test_model(model, X_test, Y_test, y_test, digits)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0fkbvsjbiIi5"
},
"source": [
"## Transfer wiedzy do nowego modelu: cyfry 5..9"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MfJ3UmNrMwfC"
},
"outputs": [],
"source": [
"# Extract specified range of layers to a new sequential model\n",
"def extract_layers(\n",
" main_model: Sequential, starting_layer_ix: int, ending_layer_ix: int\n",
") -> Sequential:\n",
" \"\"\"Return a new Sequential holding layers `starting_layer_ix`..`ending_layer_ix`.\"\"\"\n",
" # create an empty model\n",
" new_model = Sequential()\n",
" for ix in range(starting_layer_ix, ending_layer_ix + 1):\n",
" curr_layer = main_model.get_layer(index=ix)\n",
" # copy this layer over to the new model\n",
" new_model.add(curr_layer)\n",
" return new_model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aAY_aLnOiTct"
},
"source": [
"Nowy model stworzony jest na bazie wytrenowanego modelu bazowego. Część odpowiedzialna za detekcję cech w obrazie (warstwy splotowe) są wykorzystywane jako podstawa dla nowej sieci. Część klasyfikacyjna jest dodawana na nowo, z losowo zainicjowanymi wagami.\n",
"\n",
"Warstwy splotowe były wstępnie wytrenowane w modelu bazowym, więc są zamrażane. Treningowi podlega tylko część klasyfikacyjna."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "qzDfXKIMNXMK",
"outputId": "9aa913b2-1b7a-4db5-ee26-f2aca777fad1"
},
"outputs": [],
"source": [
"new_model = extract_layers(model, 0, 9)\n",
"\n",
"# Fully Connected Layer\n",
"new_model.add(Dense(64))\n",
"new_model.add(Activation(\"relu\"))\n",
"\n",
"# Fully Connected Layer\n",
"new_model.add(Dense(5))\n",
"new_model.add(Activation(\"softmax\"))\n",
"\n",
"for ix in range(9 + 1):\n",
" new_model.get_layer(index=ix).trainable = False\n",
"\n",
"new_digits = [5, 6, 7, 8, 9]\n",
"\n",
"X_train, y_train, X_test, y_test = get_mnist(new_digits)\n",
"\n",
"Y_train = one_hot(y_train)\n",
"Y_test = one_hot(y_test)\n",
"\n",
"new_model.compile(\n",
" loss=\"categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.legacy.Adam(),\n",
" metrics=[\"accuracy\"],\n",
")\n",
"\n",
"new_model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "quqhC141PBTO"
},
"outputs": [],
"source": [
"gen = ImageDataGenerator(\n",
" rotation_range=8,\n",
" width_shift_range=0.08,\n",
" shear_range=0.3,\n",
" height_shift_range=0.08,\n",
" zoom_range=0.08,\n",
" validation_split=0.2,\n",
")\n",
"\n",
"test_gen = ImageDataGenerator()\n",
"\n",
"train_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"training\")\n",
"valid_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"validation\")\n",
"test_generator = test_gen.flow(X_test, Y_test, batch_size=128)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oqy-_NxblqDy"
},
"source": [
"Trening nowego modelu"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "l0H5XRxWPKvC",
"outputId": "8724009f-078d-4667-c14e-087b53968956"
},
"outputs": [],
"source": [
"new_model.fit_generator(\n",
" train_generator,\n",
" steps_per_epoch=24000 // 128,\n",
" epochs=5,\n",
" verbose=1,\n",
" validation_data=valid_generator,\n",
" validation_steps=6000 // 128,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "FRuh3MN-PeDr",
"outputId": "228c9b51-95d9-4660-c5ef-c60f95edefcb"
},
"outputs": [],
"source": [
"test_model(new_model, X_test, Y_test, y_test, new_digits)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2zgQd-nJjU2F"
},
"source": [
"## Fine-tuning"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7kctU-jel8k-"
},
"source": [
"Po wytrenowaniu części klasyfikacyjnej można ponownie odmrozić wszystkie warstwy i przeprowadzić dotrenowanie pełnej sieci na nowych danych, bez ryzyka związanego z dużymi gradientami w pierwszych fazach uczenia."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "yu4ks06pR1aO",
"outputId": "a37535ab-3964-4dd1-95c5-891d25d85de3"
},
"outputs": [],
"source": [
"for ix in range(9 + 1):\n",
" new_model.get_layer(index=ix).trainable = True\n",
"\n",
"new_model.compile(\n",
" loss=\"categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.legacy.Adam(),\n",
" metrics=[\"accuracy\"],\n",
")\n",
"new_model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "g9x5tXRzS9yE",
"outputId": "48a4bd8e-bfae-4a96-931d-c7ac84b4000d"
},
"outputs": [],
"source": [
"new_model.fit_generator(\n",
" train_generator,\n",
" steps_per_epoch=24000 // 128,\n",
" epochs=2,\n",
" verbose=1,\n",
" validation_data=valid_generator,\n",
" validation_steps=6000 // 128,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "pbfKwpJ9TI-e",
"outputId": "27617a2a-1b2d-4c29-b041-c0369bd2a112"
},
"outputs": [],
"source": [
"test_model(new_model, X_test, Y_test, y_test, new_digits)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e6ssF6eHeHro"
},
"source": [
"# Test on different dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "s00LerSteLv5",
"outputId": "8d26e551-10a6-43f8-cd66-32e4d94249c6"
},
"outputs": [],
"source": [
"!wget https://raw.githubusercontent.com/googlecreativelab/quickdraw-dataset/master/categories.txt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "H4d-lVtqgzgC"
},
"outputs": [],
"source": [
"# The scheme is spelled out at each use site rather than folded into this\n",
"# constant, so that the URL handed to urlopen is visibly https and nothing\n",
"# else -- which is what S310 asks for.\n",
"QUICKDRAW_HOST = \"storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap\"\n",
"\n",
"\n",
"def download_and_load(\n",
" class_names: list[str],\n",
" test_split: float = 0.2,\n",
" max_items_per_class: int = 10000,\n",
") -> tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:\n",
" \"\"\"Fetch the Quick, Draw! bitmaps for `class_names` and split them.\"\"\"\n",
" root = Path(\"data\")\n",
" root.mkdir(parents=True, exist_ok=True)\n",
"\n",
" sys.stdout.write(\"downloading ...\\n\")\n",
" for c in class_names:\n",
" cc = c.replace(\"_\", \"%20\")\n",
" path = f\"https://{QUICKDRAW_HOST}/{cc}.npy\"\n",
" sys.stdout.write(f\"{path}\\n\")\n",
" destination = root / f\"{cc}.npy\"\n",
" if not destination.exists():\n",
" with urllib.request.urlopen(path) as response:\n",
" destination.write_bytes(response.read())\n",
" else:\n",
" sys.stdout.write(\"Already downloaded\\n\")\n",
" sys.stdout.write(\"loading ...\\n\")\n",
"\n",
" # initialize variables\n",
" x = np.empty([0, 784])\n",
" y = np.empty([0])\n",
"\n",
" # load each data file\n",
" for idx, name in enumerate(class_names):\n",
" quoted = name.replace(\"_\", \"%20\")\n",
" data = np.load(root / f\"{quoted}.npy\")\n",
" data = data[0:max_items_per_class, :]\n",
" labels = np.full(data.shape[0], idx)\n",
"\n",
" x = np.concatenate((x, data), axis=0)\n",
" y = np.append(y, labels)\n",
"\n",
" # randomize the dataset\n",
" permutation = np.random.default_rng().permutation(y.shape[0])\n",
" x = x[permutation, :]\n",
" y = y[permutation]\n",
"\n",
" # reshape and inverse the colors\n",
" x = 255 - np.reshape(x, (x.shape[0], 28, 28))\n",
"\n",
" # separate into training and testing\n",
" test_size = int(x.shape[0] / 100 * (test_split * 100))\n",
"\n",
" x_test = x[0:test_size, :]\n",
" y_test = y[0:test_size]\n",
"\n",
" x_train = x[test_size : x.shape[0], :]\n",
" y_train = y[test_size : y.shape[0]]\n",
"\n",
" sys.stdout.write(f\"Training Data : {x_train.shape[0]}\\n\")\n",
" sys.stdout.write(f\"Testing Data : {x_test.shape[0]}\\n\")\n",
" return x_train, y_train, x_test, y_test"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kAiJYOJBgXup"
},
"outputs": [],
"source": [
"# set your random seed value, put any number here\n",
"RANDOM_SEED = 1234"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "CsfWTt6NeUEJ",
"outputId": "7330a224-fb28-4df3-adf0-6ae4ef6aa9dd"
},
"outputs": [],
"source": [
"all_class_names = Path(\"categories.txt\").read_text().splitlines()\n",
"\n",
"random.seed(RANDOM_SEED)\n",
"\n",
"all_class_names = [x.strip().replace(\" \", \"_\") for x in all_class_names]\n",
"\n",
"nb_classes = 10\n",
"\n",
"# select random 10 classes\n",
"class_names = random.sample(all_class_names, nb_classes)\n",
"\n",
"# The lab also suggests a deliberately hard fixed set instead of a random one:\n",
"# wheel, pizza, smiley_face, apple, potato, basketball, soccer_ball, brain,\n",
"# clock and circle.\n",
"\n",
"sys.stdout.write(f\"{class_names}\\n\")\n",
"\n",
"X_train, y_train, X_test, y_test = download_and_load(class_names)\n",
"\n",
"X_train = X_train.reshape(\n",
" X_train.shape[0], 28, 28, 1\n",
") # add an additional dimension to represent the single-channel\n",
"X_test = X_test.reshape(X_test.shape[0], 28, 28, 1)\n",
"\n",
"X_train = X_train.astype(\"float32\") # change integers to 32-bit floating point numbers\n",
"X_test = X_test.astype(\"float32\")\n",
"\n",
"X_train /= (\n",
" 255 # normalize each value for each pixel for the entire vector for each input\n",
")\n",
"X_test /= 255\n",
"\n",
"Y_train = to_categorical(y_train, nb_classes)\n",
"Y_test = to_categorical(y_test, nb_classes)\n",
"\n",
"visualize_classes(X_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "W5Jm2OPyk4FF"
},
"source": [
"## Create the model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1k0Qnajnk3ey"
},
"outputs": [],
"source": [
"model = Sequential() # Linear stacking of layers\n",
"\n",
"# Convolution Layer 1\n",
"model.add(\n",
" Conv2D(64, (5, 5), input_shape=(28, 28, 1))\n",
") # 16 different 5x5 kernels -- so 16 feature maps\n",
"model.add(BatchNormalization()) # weights normalization\n",
"model.add(Activation(\"relu\")) # activation\n",
"model.add(MaxPooling2D(pool_size=(2, 2))) # Pool the max values over a 2x2 kernel\n",
"\n",
"\n",
"# Convolution Layer 2\n",
"model.add(Conv2D(32, (5, 5))) # 32 different 5x5 kernels -- so 32 feature maps\n",
"model.add(BatchNormalization()) # weights normalization\n",
"model.add(Activation(\"relu\")) # activation\n",
"model.add(MaxPooling2D(pool_size=(2, 2))) # Pool the max values over a 2x2 kernel\n",
"\n",
"model.add(Flatten()) # Flatten final output matrix into a vector\n",
"\n",
"# Fully Connected Layer\n",
"model.add(Dense(128)) # 128 FC nodes\n",
"model.add(Activation(\"relu\")) # activation\n",
"\n",
"# Fully Connected Layer\n",
"model.add(Dense(10)) # final 10 FC nodes\n",
"model.add(Activation(\"softmax\")) # softmax activation\n",
"\n",
"\n",
"# we'll use the same optimizer\n",
"adam = tf.optimizers.Adam(learning_rate=0.001)\n",
"model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "HmR_tm0gmR0u"
},
"outputs": [],
"source": [
"# data augmentation prevents overfitting by slightly changing the data randomly\n",
"# Keras has a great built-in feature to do automatic augmentation\n",
"\n",
"gen = ImageDataGenerator(\n",
" rotation_range=8,\n",
" width_shift_range=0.08,\n",
" shear_range=0.3,\n",
" height_shift_range=0.08,\n",
" zoom_range=0.08,\n",
" validation_split=0.2,\n",
")\n",
"\n",
"test_gen = ImageDataGenerator()\n",
"\n",
"train_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"training\")\n",
"valid_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"validation\")\n",
"test_generator = test_gen.flow(X_test, Y_test, batch_size=128)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 338
},
"id": "p-jeOuI8nfl9",
"outputId": "868981c3-74be-4120-b11b-11f45abf6293"
},
"outputs": [],
"source": [
"model.fit(\n",
" train_generator,\n",
" steps_per_epoch=64000 // 128,\n",
" epochs=5,\n",
" verbose=1,\n",
" validation_data=valid_generator,\n",
" validation_steps=16000 // 128,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "OlzMJklYooX8"
},
"outputs": [],
"source": [
"score = model.evaluate(X_test, Y_test)\n",
"sys.stdout.write(f\"Test score: {score[0]}\\n\")\n",
"sys.stdout.write(f\"Test accuracy: {score[1]}\\n\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "sAhuIjARoqhp"
},
"outputs": [],
"source": [
"# The predict_classes function outputs the highest probability class\n",
"# according to the trained classifier for each input example.\n",
"predicted = model.predict(X_test)\n",
"predicted_classes = np.argmax(predicted, axis=1)\n",
"\n",
"# Check which items we got right / wrong\n",
"correct_indices = np.nonzero(predicted_classes == y_test)[0]\n",
"\n",
"incorrect_indices = np.nonzero(predicted_classes != y_test)[0]\n",
"\n",
"cnf_matrix = confusion_matrix(y_test, predicted_classes)\n",
"\n",
"# Plot non-normalized confusion matrix\n",
"plt.figure()\n",
"plot_confusion_matrix(\n",
" cnf_matrix, classes=class_names, title=\"Confusion matrix, without normalization\"\n",
")\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BKDRjvPFow6L"
},
"outputs": [],
"source": [
"show_samples(correct_indices, Samples(predicted, X_test, y_test, class_names), count=5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "lgtxWrarps6b"
},
"outputs": [],
"source": [
"show_samples(\n",
" incorrect_indices, Samples(predicted, X_test, y_test, class_names), count=5\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "r29_OvwJtyM7"
},
"source": [
"# CIFAR-10"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nuo0JHXDzxT7"
},
"source": [
"## Ładowanie zbioru danych"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "pPrx-UWet0Ng",
"outputId": "15d71d6b-ff5e-4538-87e6-9ca8bdf1e8a1"
},
"outputs": [],
"source": [
"from keras.datasets import cifar10\n",
"\n",
"(X_train, y_train), (X_test, y_test) = cifar10.load_data()\n",
"\n",
"X_train = X_train.astype(\"float32\") # change integers to 32-bit floating point numbers\n",
"X_test = X_test.astype(\"float32\")\n",
"\n",
"X_train /= (\n",
" 255 # normalize each value for each pixel for the entire vector for each input\n",
")\n",
"X_test /= 255\n",
"\n",
"y_train = y_train.reshape((1, -1))[0]\n",
"y_test = y_test.reshape((1, -1))[0]\n",
"\n",
"sys.stdout.write(f\"Training matrix shape {X_train.shape} {y_train.shape}\\n\")\n",
"sys.stdout.write(f\"Testing matrix shape {X_test.shape} {y_test.shape}\\n\")\n",
"\n",
"# one-hot format classes\n",
"\n",
"nb_classes = 10\n",
"\n",
"Y_train = to_categorical(y_train, nb_classes)\n",
"Y_test = to_categorical(y_test, nb_classes)\n",
"\n",
"cifar_names = [\n",
" \"airplane\",\n",
" \"automobile\",\n",
" \"bird\",\n",
" \"cat\",\n",
" \"deer\",\n",
" \"dog\",\n",
" \"frog\",\n",
" \"horse\",\n",
" \"ship\",\n",
" \"truck\",\n",
"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q2LGp6AVzzqC"
},
"source": [
"## Podgląd zbioru treningowego"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 828
},
"id": "o6OJ7XPdxe1i",
"outputId": "ce27c176-70c1-48f6-b176-dd2ab2fca32e"
},
"outputs": [],
"source": [
"strips = []\n",
"for class_id in range(CLASS_COUNT):\n",
" img_batch = X_train[y_train == class_id][0:10]\n",
" strips.append(\n",
" np.reshape(\n",
" img_batch,\n",
" (\n",
" img_batch.shape[0] * img_batch.shape[1],\n",
" img_batch.shape[2],\n",
" img_batch.shape[3],\n",
" ),\n",
" )\n",
" )\n",
"plt.figure(figsize=(10, 20))\n",
"plt.axis(\"off\")\n",
"plt.imshow(np.concatenate(strips, axis=1), cmap=\"gray\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6EF56o2gz3Mz"
},
"source": [
"## Przygotowanie modelu"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "6UvabhsguqWc"
},
"outputs": [],
"source": [
"def generate_model() -> Sequential:\n",
" \"\"\"Build the minimal variant: one conv layer straight into the classifier.\"\"\"\n",
" model = Sequential() # Linear stacking of layers\n",
"\n",
" # Convolution Layer 1\n",
" model.add(Conv2D(16, (3, 3), input_shape=(32, 32, 3)))\n",
" model.add(Activation(\"relu\"))\n",
"\n",
" # ...\n",
"\n",
" model.add(Flatten()) # Flatten final output matrix into a vector\n",
"\n",
" # ...\n",
"\n",
" # Fully Connected Layer\n",
" model.add(Dense(10)) # final 10 FC nodes\n",
" model.add(Activation(\"softmax\")) # softmax activation\n",
"\n",
" model.summary()\n",
"\n",
" adam = tf.optimizers.Adam(learning_rate=0.001)\n",
" model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])\n",
"\n",
" return model\n",
"\n",
"\n",
"def generate_model_default() -> Sequential:\n",
" \"\"\"Build the same architecture again, kept as the baseline to compare against.\"\"\"\n",
" model = Sequential() # Linear stacking of layers\n",
"\n",
" # Convolution Layer 1\n",
" model.add(Conv2D(16, (3, 3), input_shape=(32, 32, 3)))\n",
" model.add(Activation(\"relu\"))\n",
"\n",
" # ...\n",
"\n",
" model.add(Flatten()) # Flatten final output matrix into a vector\n",
"\n",
" # ...\n",
"\n",
" # Fully Connected Layer\n",
" model.add(Dense(10)) # final 10 FC nodes\n",
" model.add(Activation(\"softmax\")) # softmax activation\n",
"\n",
" model.summary()\n",
"\n",
" adam = tf.optimizers.Adam(learning_rate=0.001)\n",
" model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])\n",
"\n",
" return model\n",
"\n",
"\n",
"def generate_model_gemini() -> Sequential:\n",
" \"\"\"Build the variant Gemini proposed: two conv blocks and heavy dropout.\"\"\"\n",
" model = Sequential()\n",
"\n",
" # Convolutional Layers with Max Pooling\n",
" model.add(Conv2D(32, (3, 3), activation=\"relu\", input_shape=(32, 32, 3)))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
" model.add(Conv2D(64, (3, 3), activation=\"relu\"))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
" model.add(Dropout(0.25)) # Regularization\n",
"\n",
" # Flatten and Fully Connected Layers\n",
" model.add(Flatten())\n",
" model.add(Dense(128, activation=\"relu\"))\n",
" model.add(Dropout(0.5)) # Regularization\n",
" model.add(Dense(10, activation=\"softmax\"))\n",
"\n",
" # Model Compilation\n",
" model.compile(\n",
" loss=\"categorical_crossentropy\",\n",
" optimizer=\"adam\", # Consider trying other optimizers\n",
" metrics=[\"accuracy\"],\n",
" )\n",
"\n",
" return model\n",
"\n",
"\n",
"def generate_model_chat() -> Sequential:\n",
" \"\"\"Build the variant ChatGPT proposed: four conv layers with batch norm.\"\"\"\n",
" model = Sequential() # Linear stacking of layers\n",
"\n",
" # Convolution Layer 1\n",
" model.add(Conv2D(32, (3, 3), padding=\"same\", input_shape=(32, 32, 3)))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(BatchNormalization())\n",
"\n",
" # Convolution Layer 2\n",
" model.add(Conv2D(32, (3, 3)))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
" model.add(Dropout(0.6))\n",
"\n",
" # Convolution Layer 3\n",
" model.add(Conv2D(64, (3, 3), padding=\"same\"))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(BatchNormalization())\n",
"\n",
" # Convolution Layer 4\n",
" model.add(Conv2D(64, (3, 3)))\n",
" model.add(Activation(\"relu\"))\n",
" model.add(MaxPooling2D(pool_size=(2, 2)))\n",
" model.add(Dropout(0.6))\n",
"\n",
" # Flattening the convolutions\n",
" model.add(Flatten())\n",
"\n",
" # Fully Connected Layer\n",
" model.add(Dense(512)) # Large fully connected layer\n",
" model.add(Activation(\"relu\"))\n",
" model.add(BatchNormalization())\n",
" model.add(Dropout(0.6))\n",
"\n",
" # Output Layer\n",
" model.add(Dense(10)) # final 10 FC nodes\n",
" model.add(Activation(\"softmax\")) # softmax activation\n",
"\n",
" model.summary()\n",
"\n",
" # Compile the model\n",
" adam = tf.optimizers.Adam(learning_rate=0.001)\n",
" model.compile(loss=\"categorical_crossentropy\", optimizer=adam, metrics=[\"accuracy\"])\n",
"\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "PSQkAq9IvJsS",
"outputId": "c3330c24-bf8b-439f-c087-b9395b346060"
},
"outputs": [],
"source": [
"model = generate_model()\n",
"model_default = generate_model_default()\n",
"model_gemini = generate_model_gemini()\n",
"model_chat = generate_model_chat()\n",
"models = [model_chat]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YMtJVU_oz55g"
},
"source": [
"## Trening"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "sjZaiBLJvQkP"
},
"outputs": [],
"source": [
"gen = ImageDataGenerator(\n",
" rotation_range=8,\n",
" width_shift_range=0.08,\n",
" shear_range=0.3,\n",
" height_shift_range=0.08,\n",
" zoom_range=0.08,\n",
" validation_split=0.2,\n",
")\n",
"\n",
"train_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"training\")\n",
"valid_generator = gen.flow(X_train, Y_train, batch_size=128, subset=\"validation\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "MlMHmbOsvbWP",
"outputId": "70259da6-b0d3-4bda-9dcb-85dce8aa6ec0"
},
"outputs": [],
"source": [
"# Max 20 epoch\n",
"for model in models:\n",
" model.fit(\n",
" train_generator,\n",
" steps_per_epoch=40000 // 128,\n",
" epochs=20,\n",
" verbose=1,\n",
" validation_data=valid_generator,\n",
" validation_steps=10000 // 128,\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mN5zKMDNz8Jp"
},
"source": [
"## Test"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "lLZORONWvqex",
"outputId": "c75bf7d8-1be8-4ddf-fe24-a2f6f3637dca"
},
"outputs": [],
"source": [
"for trained in models:\n",
" score = trained.evaluate(X_test, Y_test)\n",
" sys.stdout.write(f\"Test score: {score[0]}\\n\")\n",
" sys.stdout.write(f\"Test accuracy: {score[1]}\\n\")\n",
"\n",
" # The predict_classes function outputs the highest probability class\n",
" # according to the trained classifier for each input example.\n",
" predicted = trained.predict(X_test)\n",
" predicted_classes = np.argmax(predicted, axis=1)\n",
"\n",
" # Check which items we got right / wrong\n",
" correct_indices = np.nonzero(predicted_classes == y_test)[0]\n",
"\n",
" incorrect_indices = np.nonzero(predicted_classes != y_test)[0]\n",
"\n",
" cnf_matrix = confusion_matrix(y_test, predicted_classes)\n",
"\n",
" class_names = [str(i) for i in range(CLASS_COUNT)]\n",
"\n",
" # Plot non-normalized confusion matrix\n",
" plt.figure()\n",
" plot_confusion_matrix(\n",
" cnf_matrix, classes=class_names, title=\"Confusion matrix, without normalization\"\n",
" )\n",
"\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "ktfHkYc1zHsY"
},
"outputs": [],
"source": [
"def show_samples_rgb(indices: np.ndarray, samples: Samples, *, count: int = 3) -> None:\n",
" \"\"\"Plot a `count` x `count` grid of RGB samples, predicted vs expected.\"\"\"\n",
" plt.figure()\n",
" for i, sample in enumerate(indices[: count**2]):\n",
" pred_id = int(np.argmax(samples.scores[sample]))\n",
" real_id = int(samples.labels[sample])\n",
" pred_score = samples.scores[sample][pred_id]\n",
" real_score = samples.scores[sample][real_id]\n",
" plt.subplot(count, count, i + 1)\n",
" plt.imshow(samples.images[sample], interpolation=\"none\")\n",
" plt.axis(\"off\")\n",
" if samples.names:\n",
" predicted_label: str | int = samples.names[pred_id]\n",
" real_label: str | int = samples.names[real_id]\n",
" else:\n",
" predicted_label = pred_id\n",
" real_label = real_id\n",
" plt.title(\n",
" f\"P: {predicted_label} ({pred_score:.2f})\\n\"\n",
" f\"E: {real_label} ({real_score:.2f})\"\n",
" )\n",
"\n",
" plt.tight_layout()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "F8hS785GzoGJ"
},
"source": [
"## Poprawne klasyfikacje"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "IUcJVbOlzm5z",
"outputId": "af278820-bf9a-4aa9-89af-e26b367b9f6f"
},
"outputs": [],
"source": [
"show_samples_rgb(\n",
" correct_indices, Samples(predicted, X_test, y_test, cifar_names), count=5\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PiyibL4yzpup"
},
"source": [
"## Błędne klasyfikacje"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "ECh_2RW6zgKB",
"outputId": "2a10e208-edfe-41a9-f89e-c401515d5932"
},
"outputs": [],
"source": [
"show_samples_rgb(\n",
" incorrect_indices, Samples(predicted, X_test, y_test, cifar_names), count=5\n",
")"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [
"F8hS785GzoGJ",
"PiyibL4yzpup"
],
"gpuType": "T4",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in New Issue View Git Blame Copy Permalink