Australian cloud for business software — built and run in Sydney since 2003

NetLeverage
NetLeverage

LVM Thin Pool Almost Full: Check Data and Metadata Separately

Technical article
Troubleshooting

A thin pool has two separate capacity limits: space for data and space for the metadata that maps thin volumes to physical blocks. Check both on the pool itself. A guest filesystem showing free space, or a thin logical volume showing a large virtual size, does not establish that the shared pool can accept another write.1

Pool extension, metadata repair and snapshot removal require a separate approved procedure for the installed environment.

Why a healthy-looking filesystem can still fail

A conventional logical volume receives its storage from a volume group when allocated. A thin logical volume receives blocks from a thin pool as needed. Its advertised size can therefore exceed the physical capacity currently available. Several thin volumes can depend on the same pool, creating a shared failure boundary.1

The pool has a data component and a metadata component. The metadata records the block mappings used by the kernel's thin-provisioning implementation. Running out of either component matters. Adding filesystem capacity inside one thin volume does not by itself establish additional backing capacity for the pool.

Thin snapshots share unchanged blocks with their origins and with other snapshots. This makes initial snapshots efficient, but subsequent changes can consume additional pool space. Counting the virtual sizes of every snapshot as fully allocated overstates consumption; assuming snapshots are permanently free understates the risk.1

For an incident record, distinguish four quantities: filesystem free space, thin-volume virtual size, pool data and metadata utilisation, and unallocated extents in the volume group. Each answers a different question.

Read the pool rows, not just the volume rows

The upstream manual defines Data% as the proportion of the active thin pool's data component in use, and Meta% as the proportion of its metadata component in use. Both must remain below exhaustion. These percentages have different denominators; they cannot be added together.1

The following are unexecuted reporting examples taken from the documented field names. Replace VG/ThinPool with an already verified pool identity and confirm syntax in the installed manual before use:

lvs -o data_percent VG/ThinPool
lvs -o metadata_percent VG/ThinPool
lvs -o+seg_monitor VG/ThinPool

Use lvs -a VG with the verified volume-group name when the inspection needs the hidden _tdata and _tmeta component rows. Keep those component sizes distinct from the pool percentages above.1

Preserve the pool identity, timestamp, component sizes, percentages and relevant attributes with each sample. A thin-volume row and a thin-pool row do not provide interchangeable utilisation figures. A missing value on an inactive object is not evidence of zero usage.

These reports do not write application data, but they still need suitable administrative access and correct host context. Do not paste output containing unrelated customer volume names into public tickets. This article does not propose an automated discovery sweep across hosts.

Capacity is a rate as well as a percentage

Consider a hypothetical pool with modest data use but metadata approaching its limit. A filesystem cleanup aimed at reducing application bytes is not a demonstrated solution to the immediate metadata constraint. Conversely, comfortable metadata capacity does not protect a nearly full data component.

Record how quickly each component changes during representative activity. As an editorial planning aid, remaining capacity divided by an observed consumption rate can estimate a response window. It is not a guarantee: snapshot creation, overwrite patterns and bursts can change that rate. Use a range and retain the workload assumptions.

A percentage without a component size is also incomplete. The same percentage increase represents different absolute capacity on different pools. Record historical extensions so a falling percentage is not mistaken for reduced workload demand when the denominator actually grew.

The decision is whether there is enough time and backing capacity for an approved extension or workload intervention. A universal alert threshold cannot answer that for every write rate and operational response time.

Automatic extension has prerequisites

LVM can use dmeventd to monitor a thin pool and trigger policy-based extension. The volume group must have free extents. Monitoring must be active for that pool, and the effective autoextension policy must permit extension. A configuration entry alone does not demonstrate that all these conditions hold.1

The documented thin_pool_autoextend_threshold controls the utilisation threshold; setting it to 100 disables autoextension. thin_pool_autoextend_percent controls the extension increment as a percentage of current size. The upstream manual advises choosing the threshold with the write rate in mind so extension has time to react.1

Review the effective configuration, including applicable profiles, rather than relying on a copied global configuration file. Retain evidence of actual extension events and failures. The important operational question is whether the mechanism has both permission and available capacity to respond before exhaustion.

Automatic extension also consumes a shared reserve. If several pools depend on the same free extents, a reserve adequate for one pool may not cover simultaneous growth. That is a capacity-planning consideration, not evidence that the extension feature is defective.

A full pool is already an incident

For data exhaustion, LVM documents behaviour that either queues writes for a period or returns errors, depending on the configured full-pool policy. Queueing is not an unlimited safety buffer. The upstream manual warns that disabling the queue timeout can lead to resource exhaustion and hangs. Changing this policy does not create storage.1

Metadata exhaustion, or a failed metadata operation, can produce I/O errors and inconsistent pool metadata. Treat this as a recovery assessment, not an ordinary capacity alert. Record the first pool and filesystem errors before taking corrective action.1

Do not assume that extending capacity after errors makes every dependent filesystem and application healthy. The storage owner must assess the pool state, and application owners must assess the consequences of failed writes. A database may need its own recovery and consistency checks under its supported procedure.

The incident record should connect the sequence: which component reached its limit, whether monitoring attempted extension, why that attempt failed or arrived too late, and which consumers reported errors. This supports a durable correction rather than repeated emergency expansion.

Choose the intervention at the right layer

If the pool has headroom and is growing predictably, plan capacity and validate alerting. If growth is rapid, reduce avoidable demand through an approved application action while the storage owner evaluates capacity. If I/O errors or metadata faults are present, involve the recovery owner before normal administration resumes.

Extending a thin volume's virtual size, extending the pool's data component and extending its metadata component are distinct operations. Removing snapshots is another distinct action with retention and recovery consequences. None should be selected merely because its command is familiar.

This article deliberately omits extension, repair and removal commands. The required choices depend on layout, installed versions, free extents, recovery requirements and current pool health. A generic recipe cannot safely resolve those conditions.

After an approved change, verify both utilisation measures, monitoring state, available reserve and dependent application behaviour. Preserve before-and-after evidence under comparable load; a single low percentage is not a complete acceptance test.

Learning notes

  • Filesystem free space and thin-pool backing capacity are different measurements.
  • Data and metadata have separate limits and need separate alerts.
  • Automatic extension needs working monitoring, a usable policy and free volume-group extents.
  • Once writes have failed, capacity recovery and application recovery need separate verification.

Acceptance checklist

  • [ ] Pool identity, data usage and metadata usage are recorded separately.
  • [ ] Volume-group reserve, monitoring and extension policy are verified.
  • [ ] The approved correction addresses the component approaching exhaustion.
  • [ ] Any prior write failures receive application-integrity checks as well as capacity verification.

References

  1. lvmthin(7) - Linux manual page. Accessed 2026-11-01.

Related articles