Australian cloud for business software — built and run in Sydney since 2003

+61 2 8282 5656
support@netleverage.com
Talk to an engineer
Talk to an engineer
NetLeverage
NetLeverage

Restoring files to another system: UID and GID mapping, numeric ownership and NFSv4 ID mapping

Technical article
Backup & Recovery

Originally published:

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

Restoring files to a different identity environment can change who owns them. Even the original server can have changed users or reused identifiers since the backup. Files come back owned by the wrong person, by a bare number with no name, or by "nobody". Applications then cannot write to their own data, and users cannot open their own files.

The cause is simple once seen. Linux files do not store their owner's name. They store a number, and each system decides for itself which name that number belongs to. Copy tools and network file systems then make their own choices about whether to carry names or numbers across.

This guide sets out those rules from the primary documentation and suggests how to plan ownership for a restore to another system.

Why it matters

Ownership controls access. A database server that cannot write to its own data folder will not start. A shared folder whose files now belong to a different user may be readable by the wrong person. A web application's uploads owned by a numeric ID that no longer exists can stop the application saving files.

The problem is easy to miss in a test restore made to the same server, because the numbers line up. It appears in a real disaster, when the replacement system was built differently.

How it works

Files store numbers, not names

The inode(7) manual page describes the owner field: it "records the user ID of the owner of the file." For the group: "The inode records the ID of the group owner of the file."

The credentials(7) page confirms these are numbers: "These IDs are integers, respectively represented using the types uid_t and gid_t".

Our reading: the name shown by a file listing is looked up at the time of display, from the system's user and group databases (local files or a directory service). If user number 1001 is alice on the old server and backup on the new one, the same restored file appears to belong to backup.

Two ways to carry ownership across

Any copy or restore tool that moves files between systems has to choose:

  • By name: look up the owner's name on the source, then find the number that name has on the destination.
  • By number: keep the number exactly, whatever name it has at the destination.

Neither is always right. Matching by name is right when the same people exist on both systems with different numbers. Matching by number can preserve an original system image or data awaiting restoration of the original identity database. It is safe only when those numbers will identify the intended people and services before access is allowed; using the same directory provider alone is not proof.

rsync: names by default, numbers on request

The rsync manual explains the default: "By default rsync will use the username and groupname to determine what ownership to give files." It then names the exceptions:

  • "The special uid 0 and the special group 0 are never mapped via user/group names even if the --numeric-ids option is not specified."
  • "If a user or group has no name on the source system or it has no match on the destination system, then the numeric ID from the source system is used instead."

Our reading: the default is a mix. Users found on both systems are matched by name. Everyone else keeps their number silently, which may now belong to someone else on the destination.

With --numeric-ids, "rsync will transfer numeric group and user IDs rather than using user and group names and mapping them at both ends."

Ownership is only copied at all when asked, and only with enough rights. The --owner option sets the owner to match the source "but only if the receiving rsync is being run as the super-user". The manual continues: "Without this option, the owner of new and/or transferred files are set to the invoking user on the receiving side." For groups, a receiver that is not the super-user preserves "only groups that the invoking user on the receiving side is a member of".

rsync can also translate deliberately. --usermap and --groupmap let you "specify users and groups that should be mapped to other values by the receiving side". The manual notes that "For the --usermap option to work, the receiver will need to be running as a super-user". The simpler --chown "forces all files to be owned by USER with group GROUP."

Our reading: a forced single owner suits a handful of files handed to one person. It is the wrong choice for a whole server, where it erases the ownership the restore was meant to bring back.

GNU tar: both stored, names tried first

GNU tar records both. The manual's Handling File Attributes page says: "When writing an archive, tar writes the user ID and user name separately. If it can't find a user name (because the user ID is not in /etc/passwd ), then it does not write one. When restoring, it tries to look the name (if one was written) up in /etc/passwd. If it fails, then it uses the user ID stored in the archive instead."

It adds: "The numeric ids are always saved into tar archives."

Who extracts the archive matters. --same-owner creates files "with the same ownership they have in the archive. This is the default behavior for the superuser". For everyone else, --no-same-owner "is the default behavior for ordinary users", so files belong to whoever extracted them.

The manual names one consequence of giving files away: "the suid or sgid attributes of files are easily and silently lost when files are given away."

The --numeric-owner option "forces extraction using the numeric ids from the archive, ignoring the names." The manual gives the classic case: "It is otherwise impossible to extract files with the right ownerships if the password file in use during the extraction does not match the one belonging to the file system(s) being extracted." Its example is restoring "after a major crash" from a system "with no password file", or putting "your disk into another machine to do the restore."

GNU tar also has --owner-map and --group-map options to translate names, but each option's entry in the Option Summary states: "This option does not affect extraction from archives." Our reading: in GNU tar, those maps shape the archive when it is made; they are not a restore-time translation tool.

NFSv4: ownership attributes and request credentials

NFS adds another layer. The nfsidmap(8) page explains: "The NFSv4 protocol represents the local system's UID and GID values on the wire as strings of the form user@domain". The page calls this translation ID mapping, and says the system "derives the user part of the string by performing a password or group lookup."

The domain part matters as much as the user. By default it "is the system's DNS domain name". It can be set in the configuration file, and "When the domain is not specified in /etc/idmapd.conf the local DNS server will be queried for the _nfsv4idmapdomain text record."

The idmapd.conf(5) page defines the domain: "An NFSv4 domain is a namespace with a unique username<->UID and groupname<->GID mapping." When a name cannot be mapped, the system uses the "Nobody-User", described as the "Local user name to be used when a mapping cannot be completed".

Our reading: if a restored NFS server and its clients disagree about the domain, or a user exists on one side but not the other, files show as owned by nobody on the clients even though the numbers on disk are correct.

nfsidmap(8) warns about caching too: an incorrect cached key "can result in file and directory ownership reverting to 'nobody' on NFSv4 mount points." The default cache time is "600 seconds". Our reading: after fixing a mapping problem on a restored server, clients may show the old result until their cache expires or is cleared.

NFS file ownership representation and authentication of requests are separate layers. NFSv4 can use AUTH_SYS as well as name-based ownership attributes; these are not mutually exclusive protocol versions. The nfs(5) page says that "AUTH_SYS, relies on sending local UID and GID numbers to identify users making NFS requests." Our reading: with that mode, the numbers on the client must match the numbers on the server, whatever the names are. A restore that changed numbers to match names can break access for NFS clients that still use the old numbers.

A worked example

This illustration applies the documented rules above to an imagined case; it was not run.

An old file server had two users: alice with user number 1001 and bob with 1002. A contractor account, removed last year, had 1003, and some of its files remain. The replacement server was built fresh. On it, bob was created first and received 1001, then alice received 1002. No 1003 exists.

  • Copied with rsync ownership and group preservation enabled, using default name mapping, as the super-user: Alice's and Bob's files are matched by name and come out right. The contractor's files have no name on the source, so they keep 1003. If a new user is later given 1003, that person silently owns the old files.
  • Copied with numeric IDs: Alice's files now show as Bob's, and Bob's as Alice's. Depending on the restored permissions and ACLs, each may gain access intended for the other.
  • Extracted from a GNU tar archive as the super-user: names are looked up first, so Alice and Bob come out right, and the contractor's files keep 1003, as with rsync.
  • Extracted by an ordinary user: every file belongs to that user.

The lesson for planning is that the correct option depends on the target's identities, which is why they come first.

A planning checklist

These are our recommendations, drawn from the sources above.

  1. Keep a record of identities with every backup. Save the server's user and group lists (names and numbers, not password hashes) or note which directory service provided them.
  2. Establish the identity mapping before access. Restore the correct directory or local identity records, or keep numeric data isolated until that is possible. Verify that a matching name represents the same intended identity and reserve orphaned IDs against reuse.
  3. Decide name or number deliberately. Rebuilding the same server with the same identities: numbers. Moving data to a different system where the same people have different numbers: names, with a check that every name exists.
  4. Use enough rights. Run the restore as the super-user where ownership matters; otherwise both rsync and GNU tar give the files to the person running the restore.
  5. Watch for set-user-ID and set-group-ID bits on programs after any ownership change.
  6. For NFS, match the domain on server and clients, check that every user resolves on both sides, and allow for cached mappings.
  7. Check a sample. Compare numeric and named ownership of a sample of folders against the identity record before handing the system to users.
  8. Avoid a blanket owner change on a whole restored server. It hides the problem and removes the information needed to fix it.

Risks and limits

  • Names and numbers can disagree between systems; the same file can appear owned by someone else.
  • rsync's default mixes names and numbers, falling back to numbers silently.
  • Without super-user rights, restored files belong to the person running the restore.
  • GNU tar's owner maps apply at archive creation, not extraction.
  • NFSv4 needs matching domains and users; otherwise ownership shows as nobody.
  • Nothing here was run. Behaviour is quoted from the documentation for planning; rehearse on a non-production system.

Sources

This article was originally published on .