Python Snippets

Parallel Image Downloader with Progress Bar

This snippet demonstrates how to download multiple images in parallel using Python’s concurrent.futures and tqdm for a progress bar. This is useful for batch downloading images from URLs while maintaining performance and visibility of progress.

import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
from tqdm import tqdm
import os

def download_image(url, save_path):
    """Download an image from a URL and save it to the specified path."""
    response = requests.get(url, stream=True)
    response.raise_for_status()
    with open(save_path, 'wb') as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)
    return save_path

def download_images_parallel(urls, output_dir="downloaded_images"):
    """Download multiple images in parallel with a progress bar."""
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)
    
    with ThreadPoolExecutor(max_workers=5) as executor:
        futures = []
        for i, url in enumerate(urls):
            save_path = os.path.join(output_dir, f"image_{i + 1}.jpg")
            futures.append(executor.submit(download_image, url, save_path))
        
        for future in tqdm(as_completed(futures), total=len(futures), desc="Downloading images"):
            future.result()  # Raise exceptions if any

if __name__ == "__main__":
    # Example usage
    image_urls = [
        "https://example.com/image1.jpg",
        "https://example.com/image2.jpg",
        "https://example.com/image3.jpg",
    ]
    download_images_parallel(image_urls)

Explanation

What It Does

1. Parallel Downloads: Uses ThreadPoolExecutor to download multiple images concurrently, improving speed.
2. Progress Tracking: Integrates tqdm to display a progress bar, showing the status of downloads.
3. Error Handling: The raise_for_status() method ensures failed downloads (e.g., 404 errors) are caught early.
4. File Management: Automatically creates an output directory if it doesn’t exist.

Why It’s Useful

• Efficiency: Unlike sequential downloads, this approach leverages concurrency for faster batch processing.
• User Feedback: The progress bar provides real-time feedback, useful for large batches.
• Robustness: Proper error handling ensures reliability.

How to Run

1. Install dependencies:

   pip install requests tqdm
   

2. Replace image_urls with your target URLs.
3. Run the script. Downloaded images will appear in ./downloaded_images/.

This snippet solves a common need (batch image downloads) while optimizing for speed and usability.