Python Snippets

Parallel Image Processing with Multiprocessing

This snippet demonstrates how to use Python’s multiprocessing module to process multiple images in parallel, significantly speeding up batch operations like resizing or applying filters.

import os
from PIL import Image
from multiprocessing import Pool
from functools import partial

def process_image(image_path, output_dir, size=(128, 128)):
    """Resize an image and save it to the output directory."""
    try:
        with Image.open(image_path) as img:
            img = img.resize(size, Image.LANCZOS)
            output_path = os.path.join(output_dir, os.path.basename(image_path))
            img.save(output_path)
            print(f"Processed: {image_path}")
    except Exception as e:
        print(f"Error processing {image_path}: {e}")

def batch_process_images(input_dir, output_dir, num_processes=4):
    """Process all images in a directory in parallel."""
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    image_files = [
        os.path.join(input_dir, f) 
        for f in os.listdir(input_dir) 
        if f.lower().endswith(('.png', '.jpg', '.jpeg'))
    ]

    with Pool(num_processes) as pool:
        pool.map(partial(process_image, output_dir=output_dir), image_files)

if __name__ == "__main__":
    input_directory = "input_images"  # Directory containing source images
    output_directory = "output_images"  # Directory for resized images
    batch_process_images(input_directory, output_directory, num_processes=4)

Explanation

What It Does

• Processes images in parallel: Uses multiple CPU cores to resize a batch of images concurrently.
• Supports common formats: Handles .png, .jpg, and .jpeg files.
• Graceful error handling: Skips problematic files without crashing.

Why It’s Useful

• Performance: Parallel processing cuts processing time for large batches (e.g., 1000s of images).
• Automation: Eliminates manual work for repetitive tasks like creating thumbnails.
• Reusable: Adaptable for other image operations (e.g., filters, conversions).

How to Run

1. Install dependencies:

   pip install pillow
   

2. Place images in an input_images folder (or modify input_directory).
3. Run the script. Resized images will appear in output_images.

Customization

• Adjust size in process_image() to change output dimensions.
• Modify the file extensions in batch_process_images() to support other formats.
• Replace resize() with other PIL operations (e.g., convert('L') for grayscale).

This snippet is ideal for data scientists preprocessing datasets or developers building image-heavy applications.