Skip to content

[DOCS-1016] Add colab notebook to demo image normalization #609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mdlinville wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[DOCS-1016] Add colab notebook to demo image normalization #609

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
391 changes: 391 additions & 0 deletions colabs/image-normalization/image-normalization-demo.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,391 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "title"
},
"source": [
"# W&B Image Normalization Demo\n",
"\n",
"This notebook demonstrates how `wandb.Image` automatically normalizes different types of image data and how to control this behavior.\n",
"\n",
"## What you'll learn:\n",
"- How `wandb.Image` normalizes PyTorch tensors and NumPy arrays\n",
"- When normalization is applied vs when it's not\n",
"- How to avoid unwanted normalization\n",
"- Best practices for image logging"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "setup"
},
"source": [
"## Setup\n",
"\n",
"First, let's install the required dependencies and import the necessary libraries."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "install_deps"
},
"outputs": [],
"source": [
"# Install required packages\n!pip install --quiet wandb torch torchvision pillow matplotlib numpy"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "imports"
},
"outputs": [],
"source": [
"import wandb\n",
"import torch\n",
"import numpy as np\n",
"from PIL import Image as PILImage\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Set up matplotlib for better visualization\n",
"plt.rcParams['figure.figsize'] = (12, 8)\n",
"plt.rcParams['font.size'] = 10"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "init_wandb"
},
"source": [
"## Initialize W&B\n\nLet's start a W&B run to log our examples.\n\n> **Note**: The previous cell imported all required libraries. If you see no output, that means the imports were successful!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wandb_init"
},
"outputs": [],
"source": [
"# Initialize W&B run\n",
"run = wandb.init(\n",
" project=\"image-normalization-demo\",\n",
" name=\"normalization-examples\",\n",
" config={\n",
" \"description\": \"Demonstrating wandb.Image normalization behavior\"\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "overview"
},
"source": [
"## Understanding Image Normalization\n",
"\n",
"When you pass PyTorch tensors or NumPy arrays to `wandb.Image`, the pixel values are automatically normalized to the range [0, 255] unless you set `normalize=False`.\n",
"\n",
"**Normalization is applied to:**\n",
"- PyTorch tensors (format: `(channel, height, width)`)\n",
"- NumPy arrays (format: `(height, width, channel)`)\n",
"\n",
"**Normalization is NOT applied to:**\n",
"- PIL Images (passed as-is)\n",
"- File paths (loaded as-is)\n",
"\n",
"**Normalization algorithm:**\n",
"- [0, 1] range: values are multiplied by 255\n",
"- [-1, 1] range: values are rescaled using `255 * 0.5 * (data + 1)`\n",
"- Other ranges: values are clipped to [0, 255]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "example_1"
},
"source": [
"## Example 1: [0, 1] Range Data\n",
"\n",
"When your tensor/array values are in the [0, 1] range, `wandb.Image` will multiply all values by 255.\n",
"This example creates a 64x64 pixel image with three color channels (RGB) and random values for each pixel between 0 and 1. It then converts the image from a NumPy array to a PyTorch tensor, changing the format from (height, width, channels) to (channels, height, width) which is what PyTorch expects.\n",
"\n",
"The `wandb.Image(tensor_0_1)` function automatically:\n",
"1. **Detects** that your values are in the [0, 1] range\n",
"2. **Multiplies every value by 255** to convert to [0, 255] range\n",
"3. **Converts to uint8** (8-bit integers, which is standard for images)\n",
"\n",
"This ensures your image displays with the correct brightness and colors, since most image viewers expect values in the [0, 255] range."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "example_0_1_range"
},
"outputs": [],
"source": [
"# Create test data in [0, 1] range\n",
"data_0_1 = np.random.rand(64, 64, 3)\n",
"print(f\"Original data range: [{data_0_1.min():.3f}, {data_0_1.max():.3f}]\")\n",
"\n",
"# Convert to PyTorch tensor (channel, height, width format)\n",
"tensor_0_1 = torch.from_numpy(data_0_1).permute(2, 0, 1).float()\n",
"print(f\"Tensor shape: {tensor_0_1.shape}\")\n",
"print(f\"Tensor range: [{tensor_0_1.min():.3f}, {tensor_0_1.max():.3f}]\")\n",
"\n",
"# Visualize the original data\n",
"plt.figure(figsize=(8, 6))\n",
"plt.imshow(data_0_1)\n",
"plt.title(f'[0, 1] Range Data\\nValues will be multiplied by 255')\n",
"plt.colorbar()\n",
"plt.axis('off')\n",
"plt.show()\n",
"\n",
"# Log to W&B\n",
"wandb.log({\n",
" \"example_0_1_range\": wandb.Image(\n",
" tensor_0_1,\n",
" caption=\"[0, 1] range tensor - values will be multiplied by 255\"\n",
" )\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "example_2"
},
"source": [
"## Example 2: [-1, 1] Range Data\n\nThis example demonstrates how `wandb.Image` handles data in the [-1, 1] range, which is common in machine learning frameworks like PyTorch when using normalized data.\n\n**What this example shows:**\n- Creates a 64x64 pixel image with random values in the [-1, 1] range\n- Converts from NumPy array to PyTorch tensor with shape (3, 64, 64)\n- Shows how `wandb.Image` automatically normalizes this data to [0, 255] range\n- Demonstrates the visual effect of this normalization\n\n**Note on visual contrast:** When data in the [-1, 1] range is normalized to [0, 255], it increases the visual contrast between different pixel values. This is because the normalization process stretches the data across the full brightness range, making subtle differences more visible.\n\n**Expected warning:** You may see a deprecation warning about data normalization. This is expected when passing [-1, 1] range data and demonstrates the current normalization behavior. The warning indicates that this automatic normalization will change in future versions of wandb."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "example_neg1_1_range"
},
"outputs": [],
"source": [
"# Create test data in [-1, 1] range\n",
"data_neg1_1 = np.random.rand(64, 64, 3) * 2 - 1\n",
"print(f\"Original data range: [{data_neg1_1.min():.3f}, {data_neg1_1.max():.3f}]\")\n",
"\n",
"# Convert to PyTorch tensor\n",
"tensor_neg1_1 = torch.from_numpy(data_neg1_1).permute(2, 0, 1).float()\n",
"print(f\"Tensor shape: {tensor_neg1_1.shape}\")\n",
"print(f\"Tensor range: [{tensor_neg1_1.min():.3f}, {tensor_neg1_1.max():.3f}]\")\n",
"\n",
"# Visualize the original data\n",
"plt.figure(figsize=(8, 6))\n",
"plt.imshow(data_neg1_1, cmap='RdBu_r')\n",
"plt.title(f'[-1, 1] Range Data\\nValues will be rescaled: -1\u21920, 0\u2192127.5, 1\u2192255')\n",
"plt.colorbar()\n",
"plt.axis('off')\n",
"plt.show()\n",
"\n",
"# Log to W&B\n",
"wandb.log({\n",
" \"example_neg1_1_range\": wandb.Image(\n",
" tensor_neg1_1,\n",
" caption=\"[-1, 1] range tensor - values will be rescaled\"\n",
" )\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "example_3"
},
"source": [
"## Example 3: Avoiding Normalization with PIL Images\n",
"\n",
"To avoid normalization, you can convert your tensors to PIL Images before passing them to `wandb.Image`.\n",
"This example shows how to prevent automatic normalization by converting your PyTorch tensor to a PIL Image first. This is useful when you want to control exactly how your pixel values are processed.\n",
"\n",
"The process involves:\n",
"1. **Creating a tensor** with values in [0, 1] range\n",
"2. **Converting to NumPy array** and permuting dimensions back to (height, width, channels)\n",
"3. **Multiplying by 255** manually to convert to [0, 255] range\n",
"4. **Converting to uint8** for proper image format\n",
"5. **Creating a PIL Image** from the processed array\n",
"\n",
"When you pass a PIL Image to `wandb.Image`, it is passed through without any normalization, giving you complete control over the pixel values.\n",
"\n",
"**When to use PIL conversion vs normalize=False:**\n",
"\n",
"**Use PIL conversion when:**\n",
"- You want complete control over pixel values\n",
"- You need custom preprocessing (filters, brightness adjustments, etc.)\n",
"- You want to use PIL's image processing capabilities\n",
"- You're debugging and want to see exact values being logged\n",
"\n",
"**Use normalize=False when:**\n",
"- You want to see raw tensor values as they are\n",
"- Your data is already in the correct range (like [0, 255] integers)\n",
"- You're debugging normalization issues\n",
"- Quick testing without additional processing steps"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "example_pil_avoid_normalization"
},
"outputs": [],
"source": [
"# Create tensor with values in [0, 1] range\n",
"tensor_0_1 = torch.rand(3, 64, 64)\n",
"print(f\"Tensor range: [{tensor_0_1.min():.3f}, {tensor_0_1.max():.3f}]\")\n",
"\n",
"# Convert to PIL Image to avoid normalization\n",
"pil_image = PILImage.fromarray(\n",
" (tensor_0_1.permute(1, 2, 0).numpy() * 255).astype('uint8')\n",
")\n",
"print(f\"PIL Image size: {pil_image.size}\")\n",
"print(f\"PIL Image mode: {pil_image.mode}\")\n",
"\n",
"# Visualize the PIL image\n",
"plt.figure(figsize=(8, 6))\n",
"plt.imshow(pil_image)\n",
"plt.title('PIL Image - No normalization applied')\n",
"plt.axis('off')\n",
"plt.show()\n",
"\n",
"# Log to W&B\n",
"wandb.log({\n",
" \"example_pil_no_normalization\": wandb.Image(\n",
" pil_image,\n",
" caption=\"PIL Image - no normalization applied\"\n",
" )\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "example_4"
},
"source": [
"## Example 4: Using normalize=False\n",
"\n",
"You can also disable normalization by setting `normalize=False`. Values will be clipped to [0, 255].\n",
"This example demonstrates how to disable automatic normalization using the `normalize=False` parameter. This is useful for debugging or when you want to see the raw values of your tensor.\n",
"\n",
"When `normalize=False` is set:\n",
"1. **No multiplication by 255** occurs\n",
"2. **Values are clipped** to the [0, 255] range (values below 0 become 0, values above 255 become 255)\n",
"3. **Values are converted to uint8** for image display\n",
"\n",
"This means that if your tensor has values in [0, 1] range, they will be treated as if they were already in [0, 255] range, which will make your image appear very dark since 0.5 becomes 0.5 out of 255 (almost black)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "example_normalize_false"
},
"outputs": [],
"source": [
"# Create tensor with values in [0, 1] range\n",
"tensor_0_1 = torch.rand(3, 64, 64)\n",
"print(f\"Tensor range: [{tensor_0_1.min():.3f}, {tensor_0_1.max():.3f}]\")\n",
"\n",
"# Disable normalization\n",
"wandb.log({\n",
" \"example_normalize_false\": wandb.Image(\n",
" tensor_0_1,\n",
" normalize=False,\n",
" caption=\"Normalization disabled - values will be clipped to [0, 255]\"\n",
" )\n",
"})\n",
"\n",
"# Also log with normal normalization for comparison\n",
"wandb.log({\n",
" \"example_normalize_true\": wandb.Image(\n",
" tensor_0_1,\n",
" normalize=True,\n",
" caption=\"Normalization enabled - values will be multiplied by 255\"\n",
" )\n",
"})\n",
"\n",
"print(\"Logged both normalized and non-normalized versions for comparison\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "best_practices"
},
"source": [
"## Best Practices\n",
"\n",
"Based on what we've learned, here are some best practices for working with `wandb.Image`:\n",
"\n",
"### 1. **For consistent results**: Pre-process your data to the expected [0, 255] range before logging\n",
"### 2. **To avoid normalization**: Convert tensors to PIL Images using `PILImage.fromarray()`\n",
"### 3. **For debugging**: Use `normalize=False` to see the raw values (they will be clipped to [0, 255])\n",
"### 4. **For precise control**: Use PIL Images when you need exact pixel values\n",
"\n",
"### Common Issues to Watch Out For:\n",
"- **Unexpected brightness**: If your tensor values are in [0, 1] range, they will be multiplied by 255, making the image much brighter\n",
"- **Data loss**: Values outside the [0, 255] range will be clipped, potentially losing information\n",
"- **Inconsistent behavior**: Different input types (tensor vs PIL vs file path) may produce different results"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "finish"
},
"outputs": [],
"source": [
"# Finish the W&B run\n",
"wandb.finish()\n",
"print(\"\u2705 Demo completed! Check your W&B dashboard to see all the logged images.\")"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}