The Command-Line Really Does Matter, Part 4

09:11 reading time

In Part 3 of this series, we had a look at the find command and the set builtin, two tools that have a lot to offer and come with some lengthy manuals.

Commands dealing with file permissions have short manuals. They shouldn’t!

File modes

Unix file permissions seem to be another thing about the command-line that annoy people. You have these commands, chmod, chown, chgrp, etc, and while the latter two seem reasonable enough, the first can accept numbers or letters as arguments; neither technique is all that intuitive. When Terry asked us during a lightning talk, “do you use numbers or letters?”, most responded, to his astonishment, with “numbers.”

I like both methods for different reasons. But let’s back up first. The most basic file permissions dictate if a file is readable, writeable, or executable (or any combination of them). A directory is a type of a file, but it behaves a little differently than what you might expect depending on its readable, writeable, and executable status. Let’s see what happens when we make a directory non-executable:

~/stuff$ mkdir haha
~/stuff$ touch haha/foo haha/bar haha/baz
~/stuff$ echo "hi" >> haha/foo
~/stuff$ /bin/ls haha
bar    baz    foo
~/stuff$ chmod a-x haha
~/stuff$ /bin/ls haha
bar    baz    foo
~/stuff$ cd haha
-bash: cd: haha: Permission denied
~/stuff$ cat haha/foo
cat: haha/foo: Permission denied

So when a directory is readable but can’t be executed, you can see what’s in it, but good luck getting at what’s there.

Let’s try the opposite: make haha executable but not readable:

~/stuff$ chmod a+x,a-r haha
~/stuff$ ls haha
ls: cannot open directory haha: Permission denied
~/stuff$ cat haha/foo

When a directory is executable but not readable, you can access what’s in the directory, but you can’t explicitly list its contents.

Previously, we explored how files that are executable can be “run” whether they are binaries or interpreted scripts. However, if they’re not readable, you can’t execute them, because they can’t be read for execution!

If we examine the arguments to the chmod command above, I used the incantations a+x and a-r. This means, “all users can execute”, and “all users cannot read.” Read, write, and execute options are broken down by user, group, and other—that is, the person who made the file, the designated group of users that the owner may have assigned to it, and all other users on the system.

If we look at the output of an ls -l command (list files in “long” format), the first portion of each line tells us the permissions of each file:

~/stuff$ ls -l
-rw-r--r-- 1 ian staff   0 Apr 20 22:56 fobar
-rw-r--r-- 1 ian staff   0 Apr 27 23:33 foo
-rw-r--r-- 1 ian staff   0 Apr 20 22:56 foobar
-rw-r--r-- 1 ian staff   0 Apr 20 22:56 fooxbar
drwxr-xr-x 5 ian staff 170 Apr 28 00:11 haha

Therefore, all files in my “stuff” folder have the permissions “user (or owner) can read and write, group members can read, and all other users can read.”

The final entry shown here is for the directory “haha” that we used in the example above. Notice how it has a “d” to indicate that it is a directory, and not a traditional file.

Another aside…

The owner, in this case, is “ian”, and the group is “staff.” The “haha” folder has a “5” following its permissions details, indicating that the directory references five inodes—or, references to files. Recall from above that we created the files “foo”, “bar”, and “baz.” There are three actual files in that directory, plus the special “.” and “..” names that refer to the current directory and the parent directory, respectively.

A traditional file will usually have a “1” following the permissions listing. When you see a value higher than that, it means that multiple files reference the same inode, accomplished using a “hard link.” Directories cannot have hard links, as it would cause the filesystem to morph from a tree to a graph-like data structure.

The Numbers

Remember that permissions can be specified both using letters and by using numbers?

This is how the numbers work:

  • 4 means read.
  • 2 means write.
  • 1 means execute.

Add any combination of those together, and you get an access mode. For example, let’s say we wanted a file that was readable and writeable, but not executable. Its mode would be 6. But for what users? The owner? The group? Everyone else? Let’s say we wanted the owner to have read and write access to the file, and no one else can do anything with it at all. That would give the file mode 600. Maybe the group should be able to read it, but not write to it. That would give it 640. Remember: the ordering is user, then group, then others.

This seemingly bizarre way of indicating modes is actually pretty cool. The numbers here are bitmasks in octal. They’re OR'ed together to determine mode.

For example, consider the following binary string:

1 0 0

As a decimal (or octal) value, we would read this as “4.” Now consider this next binary value:

0 0 1

In decimal and octal, we would read this as “1.”

When we perform a boolean OR operation on these values, we get a “5”:

1 0 0
0 0 1
1 0 1

(The binary value 101 in decimal and octal is “5.”)

Feeling excited? Check out K-Maps.

Every “permission” in a unix file mode is made up of bit flags. Octal is base-8, and you’ll notice that no read/write/execute combination exceeds the value of “7.” Occasionally you may see file modes with a “leading zero”. This is not necessarily an indicator of octal notation. The leading zero in mode “0644” is actually part of the file mode. It may also have the values of a 4, a 2, or a 1, or some combination, just like the read/write/execute modes.

New rules:

  • 4 in this leading position means “set user ID”
  • 2 in this leading position means “set group ID”
  • 1 in this leading position means “make sticky”

A file with mode 5755 can be read and executed by everyone, but cannot be deleted by anyone other than the superuser and the user that owns the file. Additionally, when the file is executed by anyone, it is executed as if the user that owns the file was running it herself. That is, it is executed as that user, granting the executable portions of that file the same level of access to the system that the owner of that file has, regardless of who is actually running the file.

So: sticky mode means “harder to delete it” (i.e., users with write access to a directory cannot delete a sticky file unless they own it), and SUID / SGID modes mean “if the file is executable, execute it as the user that owns the file, or as the group that is associated with the file, respectively.”

In the output from the ls -l command, set-UID and set-GID modes are shown with the “s” symbol. Sticky mode is shown with the “t” symbol. These symbols are substituted where the “x” is usually placed, when they are set.

To even further complicate things, some file systems have extended attributes and additional access control options that are added to the inode’s metadata and are indicated with “@” and “+” symbols, respectively. These can vary from system to system, and our heads already hurt, so let’s not go there today. The thing to take away right now is that if your file access isn’t working as you’d expect, make sure you also don’t have “@”, “+”, or “.” symbols in your ls -l output on the file. You may have some additional settings to change.

A final note is in order regarding umask. This command is generally available as a shell builtin, but is also available as a stand-alone executable. Use it to set the default mode when creating new files. The numeric values umask uses are flipped on their head. Think of subtraction from 777 when analyzing these values.

You might find umask in a shell startup configuration script, like ~/.profile. On my system, the default umask is 022. Thus, the directories I create are mode 755.

Special thanks to my colleagues Lauren Gold, Jonathan Stern, and Nat Williams, for their feedback, careful review, and editing of this series.


Ian Melnick
Senior Software Engineer