Modal

Overview

Modal is a Python-first serverless cloud computing platform designed for AI/ML workloads. Its core philosophy is "just write Python decorators, Modal handles all cloud infrastructure"—automatically managing containers, GPUs, caching, storage, and scheduling, billed by the second.

Compared to Baseten, which focuses on "model serving," Modal is more general-purpose: it can run training, batch processing, web scraping, data processing, and inference. Used in production by companies like Suno, Substack, and Ramp, it is one of the most popular AI cloud platforms among Python developers.

At its first Runtime conference on 2026-10-01, Modal shipped three major capabilities. VM Sandboxes reached general availability: setting the runtime to vm gives a full Linux VM with Docker, FUSE, and kernel features while keeping the existing Sandbox API, modal.Image, sub-second cold starts, and memory burst—early customers have already started over 20 million VMs. Modal Clusters became generally available, delivering a multi-node GPU cluster through a single decorator with up to 6.4 Tbps InfiniBand verbs between nodes and automatic PyTorch and NCCL configuration. Sidecars entered beta as an isolated container co-located with the main Sandbox, forming a low-latency trust boundary between trusted and untrusted code.

Key Features

• Python Decorator Deployment: Use @stub.function() to run Python functions on cloud GPUs
• Per-Second Billing + Sub-Second Cold Start: Cold start takes about 1-2 seconds, billed per second, no cost when idle
• Full GPU Model Support: Choose on demand from T4/L4/A10G/A100/H100
• Persistent Storage: Storage primitives like Volume, NetworkFS, and Dict
• Scheduled Tasks/Cron: Supports scheduled triggers and event-driven execution
• Webhooks/REST API: Expose Python functions directly as HTTP endpoints
• VM Sandboxes (GA, 2026-10-01): Setting the runtime to vm gives a full Linux VM with Docker, FUSE, and kernel features while keeping the existing Sandbox API, modal.Image, sub-second cold starts, and memory burst; early customers have started over 20 million VMs
• Modal Clusters (GA, 2026-10-01): A multi-node GPU cluster through a single decorator, with up to 6.4 Tbps InfiniBand verbs between nodes and automatic PyTorch and NCCL configuration
• Sidecars (Beta, 2026-10-01): An isolated container co-located with the main Sandbox, forming a low-latency trust boundary between trusted and untrusted code
• Usage and Access Controls (2026-09-17): Usage page adds network egress traffic breakdown by application; unauthenticated web endpoint controls can filter by authentication, audit, and block by environment; RBAC workspaces support user groups created manually or synced from IdP via SCIM; IdP metadata XML files can be uploaded when configuring SSO

Use Cases

• Rapid prototyping of AI inference APIs
• Large-scale data processing (ETL/features)
• Model training and fine-tuning
• Individual developers who want to use GPUs without managing infrastructure
• Scheduled batch processing jobs
• Multi-node GPU cluster training with automatic PyTorch and NCCL configuration
• Running trusted and untrusted code side by side with isolated sidecar containers

Pros

• Top-tier Python development experience
• Per-second billing + fast cold start
• Rich GPU resources including H100
• Great debugging experience
• Generous free tier ($30 per month)
• VM Sandboxes, Clusters, and Sidecars shipped at the 2026-10-01 Runtime conference
• LoRA Speedrun public leaderboard (2026-07-20) uses a free Modal sandbox for verification

Pricing

$30 free credit per month. CPU $0.000038/second, memory $0.00000667/GB·second, GPU by model: T4 $0.000164/second, A10G $0.000306/second, A100 40G $0.000583/second, H100 $0.001267/second. Billed per second, no charge when idle.

Summary

Modal is an AI cloud playground for Python developers—deploy with decorators, pay per second, access rich GPUs, and enjoy sub-second cold starts. With VM Sandboxes, Clusters, and Sidecars added at its 2026-10-01 Runtime conference, it now spans full Linux VMs, multi-node GPU clusters, and isolated sidecar containers. Ideal for AI prototyping, batch processing, and inference services for individuals and small teams.

Version History

• Annualised run rate passes USD 1 billion with a USD 750 million Series D (2026-10-14): On October 14, 2026, Modal announced that its annualised run rate passed USD 1 billion and that it raised a USD 750 million Series D. Modal provides serverless compute for AI and data workloads, starting containers and virtual machines on demand so teams do not manage clusters themselves.
• Runtime conference ships VM Sandboxes, Clusters, and Sidecars (2026-10-01): Modal released three things at its first Runtime conference. VM Sandboxes reached general availability: setting the runtime to vm gives a full Linux VM with Docker, FUSE, and kernel features while keeping the existing Sandbox API, modal.Image, sub-second cold starts, and memory burst, with early customers having started over 20 million VMs. Modal Clusters became generally available, giving a multi-node GPU cluster through a single decorator with up to 6.4 Tbps InfiniBand verbs between nodes and automatic PyTorch and NCCL configuration. Sidecars entered beta as an isolated container co-located with the main Sandbox, forming a low-latency trust boundary between trusted and untrusted code
• Modal releases multiple new features: The Usage page now adds network egress traffic breakdown by application; adds unauthenticated web endpoint control (can filter by authentication, audit (2026-09-17): Modal releases multiple new features: The Usage page now adds network egress traffic breakdown by app; adds unauthenticated web endpoint controls (can filter by authentication, audit, and block by environment); RBAC workspaces support user groups (can be created manually or synced from IdP via SCIM); supports uploading IdP metadata XML files when configuring SSO.
• LoRA Speedrun public leaderboard: fine-tuning Qwen2.5-1.5B to 61.1% GSM8K accuracy in 6 minutes 05 seconds (2026-07-20): The LoRA Speedrun project has launched a public leaderboard, competing on the fine-tuning runtime of Qwen2.5-1.5B on fixed hardware (single L40S). The current record is held by @Saivineeth147 at 6 minutes and 5 seconds, achieved using sequence packing and completion-only loss masking techniques, resulting in approximately 2x speedup compared to the baseline of 11 minutes and 57 seconds, with higher accuracy (61.1%). The project provides a free Modal sandbox for verification, and any submission must be confirmed through three independent reproductions.