diff --git a/colabs/image-normalization/image-normalization-demo.ipynb b/colabs/image-normalization/image-normalization-demo.ipynb new file mode 100644 index 00000000..0c08747d --- /dev/null +++ b/colabs/image-normalization/image-normalization-demo.ipynb @@ -0,0 +1,391 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "title" + }, + "source": [ + "# W&B Image Normalization Demo\n", + "\n", + "This notebook demonstrates how `wandb.Image` automatically normalizes different types of image data and how to control this behavior.\n", + "\n", + "## What you'll learn:\n", + "- How `wandb.Image` normalizes PyTorch tensors and NumPy arrays\n", + "- When normalization is applied vs when it's not\n", + "- How to avoid unwanted normalization\n", + "- Best practices for image logging" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "setup" + }, + "source": [ + "## Setup\n", + "\n", + "First, let's install the required dependencies and import the necessary libraries." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "install_deps" + }, + "outputs": [], + "source": [ + "# Install required packages\n!pip install --quiet wandb torch torchvision pillow matplotlib numpy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "imports" + }, + "outputs": [], + "source": [ + "import wandb\n", + "import torch\n", + "import numpy as np\n", + "from PIL import Image as PILImage\n", + "import matplotlib.pyplot as plt\n", + "\n", + "# Set up matplotlib for better visualization\n", + "plt.rcParams['figure.figsize'] = (12, 8)\n", + "plt.rcParams['font.size'] = 10" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "init_wandb" + }, + "source": [ + "## Initialize W&B\n\nLet's start a W&B run to log our examples.\n\n> **Note**: The previous cell imported all required libraries. If you see no output, that means the imports were successful!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "wandb_init" + }, + "outputs": [], + "source": [ + "# Initialize W&B run\n", + "run = wandb.init(\n", + " project=\"image-normalization-demo\",\n", + " name=\"normalization-examples\",\n", + " config={\n", + " \"description\": \"Demonstrating wandb.Image normalization behavior\"\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "overview" + }, + "source": [ + "## Understanding Image Normalization\n", + "\n", + "When you pass PyTorch tensors or NumPy arrays to `wandb.Image`, the pixel values are automatically normalized to the range [0, 255] unless you set `normalize=False`.\n", + "\n", + "**Normalization is applied to:**\n", + "- PyTorch tensors (format: `(channel, height, width)`)\n", + "- NumPy arrays (format: `(height, width, channel)`)\n", + "\n", + "**Normalization is NOT applied to:**\n", + "- PIL Images (passed as-is)\n", + "- File paths (loaded as-is)\n", + "\n", + "**Normalization algorithm:**\n", + "- [0, 1] range: values are multiplied by 255\n", + "- [-1, 1] range: values are rescaled using `255 * 0.5 * (data + 1)`\n", + "- Other ranges: values are clipped to [0, 255]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "example_1" + }, + "source": [ + "## Example 1: [0, 1] Range Data\n", + "\n", + "When your tensor/array values are in the [0, 1] range, `wandb.Image` will multiply all values by 255.\n", + "This example creates a 64x64 pixel image with three color channels (RGB) and random values for each pixel between 0 and 1. It then converts the image from a NumPy array to a PyTorch tensor, changing the format from (height, width, channels) to (channels, height, width) which is what PyTorch expects.\n", + "\n", + "The `wandb.Image(tensor_0_1)` function automatically:\n", + "1. **Detects** that your values are in the [0, 1] range\n", + "2. **Multiplies every value by 255** to convert to [0, 255] range\n", + "3. **Converts to uint8** (8-bit integers, which is standard for images)\n", + "\n", + "This ensures your image displays with the correct brightness and colors, since most image viewers expect values in the [0, 255] range." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "example_0_1_range" + }, + "outputs": [], + "source": [ + "# Create test data in [0, 1] range\n", + "data_0_1 = np.random.rand(64, 64, 3)\n", + "print(f\"Original data range: [{data_0_1.min():.3f}, {data_0_1.max():.3f}]\")\n", + "\n", + "# Convert to PyTorch tensor (channel, height, width format)\n", + "tensor_0_1 = torch.from_numpy(data_0_1).permute(2, 0, 1).float()\n", + "print(f\"Tensor shape: {tensor_0_1.shape}\")\n", + "print(f\"Tensor range: [{tensor_0_1.min():.3f}, {tensor_0_1.max():.3f}]\")\n", + "\n", + "# Visualize the original data\n", + "plt.figure(figsize=(8, 6))\n", + "plt.imshow(data_0_1)\n", + "plt.title(f'[0, 1] Range Data\\nValues will be multiplied by 255')\n", + "plt.colorbar()\n", + "plt.axis('off')\n", + "plt.show()\n", + "\n", + "# Log to W&B\n", + "wandb.log({\n", + " \"example_0_1_range\": wandb.Image(\n", + " tensor_0_1,\n", + " caption=\"[0, 1] range tensor - values will be multiplied by 255\"\n", + " )\n", + "})" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "example_2" + }, + "source": [ + "## Example 2: [-1, 1] Range Data\n\nThis example demonstrates how `wandb.Image` handles data in the [-1, 1] range, which is common in machine learning frameworks like PyTorch when using normalized data.\n\n**What this example shows:**\n- Creates a 64x64 pixel image with random values in the [-1, 1] range\n- Converts from NumPy array to PyTorch tensor with shape (3, 64, 64)\n- Shows how `wandb.Image` automatically normalizes this data to [0, 255] range\n- Demonstrates the visual effect of this normalization\n\n**Note on visual contrast:** When data in the [-1, 1] range is normalized to [0, 255], it increases the visual contrast between different pixel values. This is because the normalization process stretches the data across the full brightness range, making subtle differences more visible.\n\n**Expected warning:** You may see a deprecation warning about data normalization. This is expected when passing [-1, 1] range data and demonstrates the current normalization behavior. The warning indicates that this automatic normalization will change in future versions of wandb." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "example_neg1_1_range" + }, + "outputs": [], + "source": [ + "# Create test data in [-1, 1] range\n", + "data_neg1_1 = np.random.rand(64, 64, 3) * 2 - 1\n", + "print(f\"Original data range: [{data_neg1_1.min():.3f}, {data_neg1_1.max():.3f}]\")\n", + "\n", + "# Convert to PyTorch tensor\n", + "tensor_neg1_1 = torch.from_numpy(data_neg1_1).permute(2, 0, 1).float()\n", + "print(f\"Tensor shape: {tensor_neg1_1.shape}\")\n", + "print(f\"Tensor range: [{tensor_neg1_1.min():.3f}, {tensor_neg1_1.max():.3f}]\")\n", + "\n", + "# Visualize the original data\n", + "plt.figure(figsize=(8, 6))\n", + "plt.imshow(data_neg1_1, cmap='RdBu_r')\n", + "plt.title(f'[-1, 1] Range Data\\nValues will be rescaled: -1\u21920, 0\u2192127.5, 1\u2192255')\n", + "plt.colorbar()\n", + "plt.axis('off')\n", + "plt.show()\n", + "\n", + "# Log to W&B\n", + "wandb.log({\n", + " \"example_neg1_1_range\": wandb.Image(\n", + " tensor_neg1_1,\n", + " caption=\"[-1, 1] range tensor - values will be rescaled\"\n", + " )\n", + "})" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "example_3" + }, + "source": [ + "## Example 3: Avoiding Normalization with PIL Images\n", + "\n", + "To avoid normalization, you can convert your tensors to PIL Images before passing them to `wandb.Image`.\n", + "This example shows how to prevent automatic normalization by converting your PyTorch tensor to a PIL Image first. This is useful when you want to control exactly how your pixel values are processed.\n", + "\n", + "The process involves:\n", + "1. **Creating a tensor** with values in [0, 1] range\n", + "2. **Converting to NumPy array** and permuting dimensions back to (height, width, channels)\n", + "3. **Multiplying by 255** manually to convert to [0, 255] range\n", + "4. **Converting to uint8** for proper image format\n", + "5. **Creating a PIL Image** from the processed array\n", + "\n", + "When you pass a PIL Image to `wandb.Image`, it is passed through without any normalization, giving you complete control over the pixel values.\n", + "\n", + "**When to use PIL conversion vs normalize=False:**\n", + "\n", + "**Use PIL conversion when:**\n", + "- You want complete control over pixel values\n", + "- You need custom preprocessing (filters, brightness adjustments, etc.)\n", + "- You want to use PIL's image processing capabilities\n", + "- You're debugging and want to see exact values being logged\n", + "\n", + "**Use normalize=False when:**\n", + "- You want to see raw tensor values as they are\n", + "- Your data is already in the correct range (like [0, 255] integers)\n", + "- You're debugging normalization issues\n", + "- Quick testing without additional processing steps" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "example_pil_avoid_normalization" + }, + "outputs": [], + "source": [ + "# Create tensor with values in [0, 1] range\n", + "tensor_0_1 = torch.rand(3, 64, 64)\n", + "print(f\"Tensor range: [{tensor_0_1.min():.3f}, {tensor_0_1.max():.3f}]\")\n", + "\n", + "# Convert to PIL Image to avoid normalization\n", + "pil_image = PILImage.fromarray(\n", + " (tensor_0_1.permute(1, 2, 0).numpy() * 255).astype('uint8')\n", + ")\n", + "print(f\"PIL Image size: {pil_image.size}\")\n", + "print(f\"PIL Image mode: {pil_image.mode}\")\n", + "\n", + "# Visualize the PIL image\n", + "plt.figure(figsize=(8, 6))\n", + "plt.imshow(pil_image)\n", + "plt.title('PIL Image - No normalization applied')\n", + "plt.axis('off')\n", + "plt.show()\n", + "\n", + "# Log to W&B\n", + "wandb.log({\n", + " \"example_pil_no_normalization\": wandb.Image(\n", + " pil_image,\n", + " caption=\"PIL Image - no normalization applied\"\n", + " )\n", + "})" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "example_4" + }, + "source": [ + "## Example 4: Using normalize=False\n", + "\n", + "You can also disable normalization by setting `normalize=False`. Values will be clipped to [0, 255].\n", + "This example demonstrates how to disable automatic normalization using the `normalize=False` parameter. This is useful for debugging or when you want to see the raw values of your tensor.\n", + "\n", + "When `normalize=False` is set:\n", + "1. **No multiplication by 255** occurs\n", + "2. **Values are clipped** to the [0, 255] range (values below 0 become 0, values above 255 become 255)\n", + "3. **Values are converted to uint8** for image display\n", + "\n", + "This means that if your tensor has values in [0, 1] range, they will be treated as if they were already in [0, 255] range, which will make your image appear very dark since 0.5 becomes 0.5 out of 255 (almost black)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "example_normalize_false" + }, + "outputs": [], + "source": [ + "# Create tensor with values in [0, 1] range\n", + "tensor_0_1 = torch.rand(3, 64, 64)\n", + "print(f\"Tensor range: [{tensor_0_1.min():.3f}, {tensor_0_1.max():.3f}]\")\n", + "\n", + "# Disable normalization\n", + "wandb.log({\n", + " \"example_normalize_false\": wandb.Image(\n", + " tensor_0_1,\n", + " normalize=False,\n", + " caption=\"Normalization disabled - values will be clipped to [0, 255]\"\n", + " )\n", + "})\n", + "\n", + "# Also log with normal normalization for comparison\n", + "wandb.log({\n", + " \"example_normalize_true\": wandb.Image(\n", + " tensor_0_1,\n", + " normalize=True,\n", + " caption=\"Normalization enabled - values will be multiplied by 255\"\n", + " )\n", + "})\n", + "\n", + "print(\"Logged both normalized and non-normalized versions for comparison\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "best_practices" + }, + "source": [ + "## Best Practices\n", + "\n", + "Based on what we've learned, here are some best practices for working with `wandb.Image`:\n", + "\n", + "### 1. **For consistent results**: Pre-process your data to the expected [0, 255] range before logging\n", + "### 2. **To avoid normalization**: Convert tensors to PIL Images using `PILImage.fromarray()`\n", + "### 3. **For debugging**: Use `normalize=False` to see the raw values (they will be clipped to [0, 255])\n", + "### 4. **For precise control**: Use PIL Images when you need exact pixel values\n", + "\n", + "### Common Issues to Watch Out For:\n", + "- **Unexpected brightness**: If your tensor values are in [0, 1] range, they will be multiplied by 255, making the image much brighter\n", + "- **Data loss**: Values outside the [0, 255] range will be clipped, potentially losing information\n", + "- **Inconsistent behavior**: Different input types (tensor vs PIL vs file path) may produce different results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "finish" + }, + "outputs": [], + "source": [ + "# Finish the W&B run\n", + "wandb.finish()\n", + "print(\"\u2705 Demo completed! Check your W&B dashboard to see all the logged images.\")" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file