NVIDIA MIG, vGPU, and time-slicing for shared inference

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

Three ways of "sharing a GPU" give three different machines. Multi-Instance GPU (MIG) cuts the card into smaller GPUs, each with its own memory capacity and memory bandwidth. A virtual GPU (vGPU) gives each virtual machine a fixed slice of framebuffer and then time-slices the engines. Kubernetes time-slicing only changes the number of pods that are allowed to open the same GPU. It does not split memory, and it does not contain a fault.

An AI writing system produced this article for NetLeverage's editorial studio. Profile sizes are from the NVIDIA guides opened on 10 November 2026. No instance was created, and no inference process was placed on a shared GPU. No NetLeverage platform is described.

Why it matters

Inference memory is mostly weights plus the key-value cache. A 13-billion-parameter model in 16-bit storage is about 26 GB of weights before any cache, using the arithmetic in the companion sizing article. A slice that advertises 5 GB cannot hold it. A time-sliced "share" of an 80 GB card can hold it, and so can the next pod, until the second copy of the weights makes the card run out of memory. The failure looks like an allocator bug. It is the sharing mode doing what its documentation says.

The other failure is latency. A decode step wants the engines on a regular cadence. A neighbour that is allowed to run for a long slice, or to soak up idle cycles, stretches the gap between tokens. MIG's pitch, in the concepts chapter, is memory quality of service: dedicated memory capacity and bandwidth per GPU instance. Time-slicing's pitch, in the GPU Operator page, is that some sharing is better than none, on GPUs that cannot be partitioned and for workloads that can tolerate the interference.

How it works

MIG: slices of memory and slices of SMs

MIG applies to Ampere and later GPUs, compute capability 8.0 and above. The supported-products table opened that day includes A100 (40 GB and 80 GB, SXM and PCIe) with at most 7 instances, A30 (24 GB) with at most 4, H100 and H200 variants with at most 7, B200 (180 GB) and GB200 (186 GB) with at most 7, and RTX PRO 6000 Blackwell with at most 4. A consumer card that is not on that table does not grow MIG support because a scheduler asked for it.

The concepts chapter defines the pieces:

• A memory slice is about one eighth of the GPU's memory, including the matching memory controllers and cache.
• An SM slice is about one seventh of the streaming multiprocessors.
• A GPU instance (GI) combines memory slices with engines (copy engines, decoders, and so on). It is the unit that provides memory quality of service. Capacity and bandwidth are both limited to that instance's slices.
• A compute instance (CI) takes a subset of the parent GI's SM slices. Compute instances share the parent instance's memory and engines. Splitting a GI into several CIs isolates SMs. It does not isolate memory.

On the A100 40 GB example in that chapter, the GPU is treated as eight 5 GB memory slices and seven SM slices. The profile names and how many can exist at once:

An 80 GB A100 uses the same fractions with names shifted up, for example 1g.10gb and 7g.80gb. H100, H200, and Blackwell profiles are different tables. Read the supported-profiles chapter for the card you have. Do not copy the A100 names onto an H100.

The name is not the usable size. The getting-started sample of nvidia-smi mig -lgip on an A100 lists 1g.5gb as 4.75 GiB and 14 SMs, and 7g.40gb as 39.25 GiB and 98 SMs. Budget the figure in that table, not the number in the profile string. The same sample shows a P2P column. On that example it is No. Read the column from the driver you installed. This sample does not establish a universal peer-to-peer rule for other products or releases.

Profiles do not pack like water. A valid set is one that does not overlap on the physical layout. The concepts chapter shows placements, and it warns that creating and destroying instances in different places fragments the GPU, so a slice that "should" fit can be refused because of where the existing instances sit. Before driver R510, a 4-memory/4-compute profile could not sit beside a 4-memory/3-compute profile. That restriction is gone on newer drivers. The placement rule is not.

NVIDIA's comparison table puts MIG next to CUDA streams and MPS. Streams share an address space inside one process. MPS can cap SMs by percentage, protects memory, and does not isolate errors or memory bandwidth; the table's maximum is 48 clients. MIG is a physical partition, at most 7, with SM isolation, memory protection, memory-bandwidth quality of service, and error isolation. Cross-partition communication is limited IPC. Reconfiguration is when idle, not dynamic. The current mode can be inspected through mig.mode.current. Enabling MIG or changing instance layouts requires a separately reviewed device-reconfiguration plan with workloads stopped; this article is not a reconfiguration procedure.

For an inference tenant, the GI is the boundary that matters. Two tenants on two GIs do not share DRAM bandwidth, which is the resource a decode step spends. Two compute instances inside one GI still share that bandwidth and that capacity. A 1g.5gb instance at 4.75 GiB cannot hold a 7B model in 16-bit weights (about 14 GB) plus a cache. A 3g.20gb instance might hold the weights and a short cache. A 7g.40gb instance is most of the card, and you have not shared much. The slice has to be chosen from the model size, not from a desire to "get seven users on one GPU."

vGPU: reserved framebuffer, scheduled engines

The virtual GPU user guide describes a vGPU as a device with a fixed framebuffer, allocated from the physical GPU when the vGPU is created, and held exclusively until the vGPU is destroyed. The guest loads an NVIDIA driver. Performance-critical paths go to the GPU. Management goes through a paravirtualised interface to the Virtual GPU Manager.

On a GPU that is not partitioned, every vGPU is time-sliced. It shares the graphics, decode, and encode engines with the other vGPUs. Processes on one vGPU run in series with the others. While a vGPU is scheduled, it has exclusive use of those engines. The framebuffer is not shared. A neighbour cannot allocate your framebuffer. A neighbour can delay your turn on the engines.

Three schedulers exist. Best effort is the default on all supported architectures. It tries to balance vGPUs and lets one vGPU use cycles another is not using. The guide says a graphics-heavy virtual machine can hurt a graphics-light one under that policy. Equal share divides the running vGPUs. The share changes when a vGPU starts or stops, so your throughput moves when a neighbour arrives. In equal-size mode the share is the reciprocal of the maximum number of that vGPU type on the GPU: an A10-8Q, of which at most three fit, is given one third of the cycles. Fixed share keeps each vGPU's fraction constant as neighbours come and go. In mixed-size mode the fraction follows framebuffer: the guide's example puts an A10-8Q at half the cycles and an A10-4Q at a quarter, on the same GPU. Equal share and fixed share use a strict round-robin and disable the frame-rate limiter. A shorter time slice cuts latency and costs throughput, because the scheduler switches more often. The scheduler can overshoot a slice and then shorten the next one. An averaging factor spreads that correction out. A high average enforces the target frequency less strictly.

The same chapter distinguishes scheduler capability output by vGPU engine type. If the vGPU engine type is graphics, the scheduler capabilities include the policies above. If the engine type is anything else, the only capability value indicates support for best effort, and the other values are zero. An application performing inference does not by itself establish the engine-type classification reported by this interface. Check supported policies using nvidia-smi vgpu --schedulercaps on the actual platform. The best-effort default is exactly the policy the guide warns can let a heavy client hurt a light one.

On a MIG GPU the user guide adds two further shapes. A MIG-backed vGPU can occupy a whole GPU instance, so it runs in parallel with vGPUs on other instances. A time-sliced, MIG-backed vGPU occupies a fraction of one instance and shares that instance's engines, while still running in parallel with vGPUs on other instances. Support is not universal. nvidia-smi -q reports MIG Time-Slicing as Supported or not, and MIG Time-Slicing Mode as Enabled or not for the hypervisor. The total framebuffer given to vGPUs on a GPU cannot exceed the physical framebuffer. If a GPU was previously in MIG mode and you now want plain time-sliced vGPUs or passthrough, the guide says to disable MIG first.

The inference consequence is the exclusive framebuffer. A 24 GB vGPU profile cannot run a model whose weights and cache need 40 GB, even if the physical card is an 80 GB H100. The rest of the card belongs to other vGPUs. What you gain, relative to MIG, is a hypervisor VM boundary and a scheduler you can name. What you do not gain, on a time-sliced profile, is simultaneous execution. Decode latency includes the wait for the slice.

Kubernetes time-slicing: a replica count, not a partition

The GPU Operator exposes time-slicing by advertising several Kubernetes resources for one GPU. A config map with replicas: 4 makes a one-GPU node advertise four nvidia.com/gpu resources. A two-GPU node advertises eight. Pods interleave on the underlying device.

The page is explicit about what that does not mean. Unlike MIG, there is no memory isolation and no fault isolation between replicas. A typical GPU request is exclusive. A time-sliced request is shared. Requesting more than one time-sliced GPU does not reserve a larger fraction of the engines. The GPU gives an equal share of time to all GPU processes, across all pods. A pod that asked for two replicas does not get twice the compute of a pod that asked for one. failRequestsGreaterThanOne exists to make a request of more than one fail admission, so that mistake is caught. The page's sample sets it to false. Setting it true rejects larger requests with UnexpectedAdmissionError. Using true to enforce an “access, not multiple shares” contract is an operational recommendation here, not a default established by that sample.

Time-slicing works on GPUs that have no MIG, and it can be combined with MIG so that several pods share one MIG instance. The Operator page also says the mechanism can be used with bare metal, with GPU passthrough VMs, and with vGPU. In every case the isolation is whatever the layer underneath already had. Time-slicing does not add any.

For inference, the missing memory isolation is the whole design. Each pod that loads the model allocates its own weights, unless the pods are deliberately sharing one process, which this feature does not do. Two replicas each loading a 40 GB model onto one 80 GB GPU will not "each get half the SMs and half the memory." They will both try to allocate 40 GB, plus cache, against the same 80 GB of physical framebuffer. The GPU Operator's own comparison is the right summary: time-slicing trades MIG's isolation for a larger number of clients, and for older GPUs that cannot be partitioned.

What to choose and why

The check that proves the mode is the tool output, not the pod's resource count. nvidia-smi -L and nvidia-smi mig -lgi show instances. A time-sliced node shows one physical GPU and a replica label such as nvidia.com/gpu.replicas=4. A vGPU guest shows a framebuffer size that matches the profile, not the physical card. If those three pictures do not match the picture you bought, the latency and the out-of-memory errors will not match it either.

Risks and limits

• MIG reconfiguration is when the GPU is idle. The concepts table says so. Enabling MIG, or destroying an instance under a running server, is a disruption. This article does not walk through those commands as a procedure.
• Profile names overstate usable memory. The getting-started sample's 4.75 GiB for 1g.5gb is the number to budget. Placement fragmentation can refuse a combination that the fractions appear to allow.
• Compute instances are not smaller MIG tenants. They share the GI's memory and engines.
• vGPU best effort can let a heavy VM consume cycles a light VM needed. Equal and fixed share constrain allocation differently. Confirm the policies exposed by the platform rather than inferring them from the application.
• A time-sliced Kubernetes replica is not a fraction of the GPU. Processes across pods share time equally. Memory faults are shared. replicas: 4 does not create four 20 GB GPUs out of an 80 GB card.
• The supported-GPU table moves. Blackwell parts are on the page opened that day. A card absent from the table on the day you read it is not covered by this article's list.
• No command was run. Samples of nvidia-smi output are quotations from NVIDIA's guides. No NetLeverage GPU layout or tenancy claim is made.

Sources

1. NVIDIA, Multi-Instance GPU User Guide, concepts — https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/concepts.html (opened 10 November 2026). Slices, GI versus CI, A100 40 GB profiles, streams/MPS/MIG table, reconfigure when idle.
2. NVIDIA, Supported GPUs — https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/supported-gpus.html (opened 10 November 2026). Product table and maximum instance counts.
3. NVIDIA, Getting Started with MIG — https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/getting-started-with-mig.html (opened 10 November 2026). nvidia-smi mig -lgip sample sizes, including 4.75 GiB for 1g.5gb, and the requirements for changing MIG mode.
4. NVIDIA GPU Operator, Time-Slicing GPUs in Kubernetes — https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html (opened 10 November 2026). No memory or fault isolation, replica semantics, failRequestsGreaterThanOne.
5. NVIDIA, Virtual GPU Software User Guide — https://docs.nvidia.com/vgpu/latest/grid-vgpu-user-guide/installing-configuring-grid-vgpu.html (opened 10 November 2026). Exclusive framebuffer, time-sliced and MIG-backed shapes, MIG Time-Slicing in nvidia-smi -q.
6. NVIDIA, Changing Scheduling Behavior for Time-Sliced vGPUs — https://docs.nvidia.com/vgpu/latest/grid-vgpu-user-guide/changing-vgpu-scheduling-policy.html (opened 10 November 2026). Best effort, equal share, fixed share, and the non-graphics capability note.

This article was originally published on 10 November 2026.