89

Linux determines a file's type via code in the file's header. This process doesn't depend on file extensions to know which software to use for opening the file.

(That's what I remember from my education. Please correct me in case I'm wrong!)

Working a bit with Ubuntu systems recently: I see a lot of files on the systems which have extensions like .sh, .txt, .o, .c.

Now I'm wondering: Is the purpose of these extensions to merely help people understand what sort of file they happen to be looking at? Or do they have some purpose for the operating system also?

10
  • 5
    If you don't get a good response here, remember there is also unix.stackexchange.com
    – mchid
    Jul 27, 2016 at 6:51
  • 1
    Related, almost duplicate: askubuntu.com/questions/390015/… Jul 27, 2016 at 7:01
  • 1
    related: unix.stackexchange.com/questions/207276/…
    – chaos
    Jul 27, 2016 at 10:33
  • 6
    In Windows they do, in Linux/Unix they mostly don't. The main exception are the compression-programs - gzip, bzip2, xz - and so on. These programs uses suffixes to separate the compressed version of a file from the uncompressed one they replace. Compression-programs will often complain about incorrect suffix, even though the file actually is a compressed file of the type it should handle. Jul 27, 2016 at 10:34
  • 6
    I think part of the problem with this question is that "the operating-system" is not a well-defined concept. What is part of the operating system, and what is an application on top of it? Not many parts of the OS (whichever OS we're talking about) care what type a file is - they just do what they're told. So distinctions about how they know are irrelevant; they do neither. Applcations, on the other hand, may well do one or both things.
    – IMSoP
    Jul 27, 2016 at 17:23

7 Answers 7

80

There is no 100% black or white answer here.

Usually Linux does not rely on file names (and file extensions i.e. the part of the file name after the normally last period) and instead determines the file type by examining the first few bytes of its content and comparing that to a list of known magic numbers.

For example all Bitmap image files (usually with name extension .bmp) must start with the letters BM in their first two bytes. Scripts in most scripting languages like Bash, Python, Perl, AWK, etc. (basically everything that treats lines starting with # as comment) may contain a shebang like #!/bin/bash as first line. This special comment tells the system with which application to open the file.

So normally the operating system relies on the file content and not its name to determine the file type, but stating that file extensions are never needed on Linux is only half of the truth.


Applications may of course implement their file checks however they want, which includes verifying the file name and extension. An example is the Eye of Gnome (eog, standard picture viewer) which determines the image format by the file extension and throws an error if it does not match the content. Whether this is a bug or a feature can be discussed...

However, even some parts of the operating system rely on file name extensions, e.g. when parsing your software sources files in /etc/apt/sources.list.d/ - only files with the *.list extension get parsed all others are ignored. It's maybe not mainly used to determine the file type here but rather to enable/disable parsing of some files, but it's still a file extension that affects how the system treats a file.

And of course the human user profits most from file extensions as that makes the type of a file obvious and also allows multiple files with the same base name and different extensions like site.html, site.php, site.js, site.css etc. The disadvantage is of course that file extension and the actual file type/content do not necessarily have to match.

Additionally it's needed for cross-platform interoperability, as e.g. Windows will not know what to do with a readme file, but only a readme.txt.

4
  • You slightly contradict yourself here: if the standard image viewer requires a filename ending .bmp, what part of the OS are you saying relies on the file content starting "BM"? AFAIK, the only "magic numbers the kernel cares about are executable types, including the special case of #!. Everything else is up to some application's decision.
    – IMSoP
    Jul 27, 2016 at 18:20
  • @IMSoP I don't know the exact implementation of eog and I don't know why they care about the file name at all. This is a bug in my opinion. And of course if the file is named "bmp" but its content format does not match, there will be an error as well, of course. Of course each application decides how to verify files, but in general Linux applications should not rely on the name. Btw, you can use the file commend to examine file types by their content.
    – Byte Commander
    Jul 27, 2016 at 19:12
  • 2
    The sentence I am challenging is this: "Linux ... determines the file type by examining the first few bytes". What definition of "Linux" are you using in that sentence? The existence of the file utility doesn't really prove anything; it's a useful tool, that could exist on any OS. What fundamental part of the OS makes running file any more "correct" than globbing the file name?
    – IMSoP
    Jul 27, 2016 at 21:51
  • Note that files without an extension can be associated with a program.
    – isanae
    Jul 28, 2016 at 2:09
47

Linux determines the type of a file via a code in the file header. It doesn't depend on file extensions for to know with software is to use for opening the file.

That's what I remember from my education. Please correct me in case I'm wrong!

  • correctly remembered.

Are these extensions are meant only for humans?

  • Yes, with a but.

When you interact with other operating systems that do depend on extensions being what they are it is the smarter idea to use those.

In Windows, opening software is attached to the extensions.

Opening a text file named "file" is harder in Windows than opening the same file named "file.txt" (you will need to switch the file open dialog from *.txt to *.* every time). The same goes for TAB and semi-colon separated text files. The same goes for importing and exporting e-mails (.mbox extension).

In particular when you code software. Opening a file named "software1" that is an HTML file and "software2" that is a JavaScript file becomes more difficult compared to "software.html" and "software.js".


If there is a system in place in Linux where file extensions are important, I would call that a bug. When software depends on file extensions, that is exploitable. We use an interpreter directive to identify what a file is ("the first two bytes in a file can be the characters "#!", which constitute a magic number (hexadecimal 23 and 21, the ASCII values of "#" and "!") often referred to as shebang,").

The most famous problem with file extensions was LOVE-LETTER-FOR-YOU.TXT.vbs on Windows. This is a visual basic script being shown in file explorer as a text file.

In Ubuntu when you start a file from Nautilus you get a warning what it is going to do. Executing a script from Nautilus where it wants to start some software where it is supposed to open gEdit is obvious a problem and we get a warning about it.

In command line when you execute something, you can visually see what the extension is. If it ends on .vbs I would start to become suspicious (not that .vbs is executable on Linux. At least not without some more effort ;) ).

21
  • 37
    I completely don't get what you wanted to say in your last sentence. First, it is a problem of hiding the extension rather than having it, second the exploit would work the same in Linux - you name a binary file readme.txt and make it executable. If user executed it, it does not open the editor, but runs the code. In this respect making extensions matter (but not hiding them) is more secure and easier to explain for non-savvy users. There are other differences (most notably not executing files from the current directory), but they have nothing to do with extensions.
    – techraf
    Jul 27, 2016 at 8:07
  • 5
    @techraf Actually the file manager will probably try to open the readme.txt file with a text editor. I just tried with dolphin in KDE, creating a shell script adding executable permission, saving it as .txt and clicking on it will make it open in Kate. If I rename it to .sh then clicking on it runs it.
    – Bakuriu
    Jul 27, 2016 at 11:22
  • 9
    linux: since make is build around rules that depend on the file extension, wouldn't this make (no pun intended) the extensions meant for more than just humans?
    – bolov
    Jul 27, 2016 at 13:08
  • 21
    This is a monumentally wrong answer. Some parts of Linux use magic numbers to determine file types. Executing files at the command line. But other huge parts of the system use file extensions to know what to look at, whether those be the dynamic linker (which wants .so files), modprobe, build systems, plugins, libraries for python, ruby, etc. Many files don't have magic numbers, file is heuristic-based, not definite. Jul 29, 2016 at 0:08
  • 4
    "Linux determines the type of a file via a code in the file header" "correct" WTF? What "code in the file header" ? There is no such code, and there is no such a generic "file header" in Linux.
    – leonbloy
    Jul 30, 2016 at 18:37
24

As mentioned by others, in Linux an interpreter directive method is used (storing some metadata in a file as a header or magic number so the correct interpreter can be told to read it) rather than the filename extension association method used by Windows.

This means you can create a file with almost any name you like... with a few exceptions

However

I would like to add a word of caution.

If you have some files on your system from a system that uses filename association, the files may not have those magic numbers or headers. Filename extensions are used to identify these files by applications that are able to read them, and you may experience some unexpected effects if you rename such files. For example:

If you rename a file My Novel.doc to My-Novel, Libreoffice will still be able to open it, but it will open as 'Untitled' and you will have to name it again in order to save it (Libreoffice adds an extension by default, so you would then have two files My-Novel and My-Novel.odt, which could be annoying)

More seriously, if you rename a file My Spreadsheet.xlsx to My-Spreadsheet, then try to open it with xdg-open My-Spreadsheet you will get this (because it's actually a compressed file):

And if you rename a file My Spreadsheet.xls to My-Spreadsheet, when you xdg-open My-Spreadsheet you get an error saying

error opening location: No application is registered as handling this file

(Although in both these cases it works OK if you do soffice My-Spreadsheet)

If you then rename the extensionless file to My-Spreadsheet.ods with mv and try to open it you will get this:

(repair fails)

And you will have to put the original extension back on to open the file correctly (you can then convert the format if you wish)

TL;DR:

If you have non-native files with name extensions, don't remove the extensions assuming everything will be OK!

5
  • 4
    A new-style MS Office document (docx, xlsx, pptx etc) without file extension opens in the archive manager because those file types are actually just ordinary ZIP compressed files which contain all the XML documents and media files necessary to define the document content. The file format of a ZIP compressed directory is pretty common nowadays btw.
    – Byte Commander
    Jul 27, 2016 at 8:21
  • 1
    Already many great answers, but just one more specific to libreoffice that I've noticed. You create a file of comma separated values (CSV) and save it as "test.csv", a window will open asking what type of separator are you using (i.e., libreoffice Calc). If you rename this file to "test.cs", for example, then libreoffice's Writer opens it. So, besides the ZIP example above, it does seem like libreoffice does make use of the file extension.
    – Ray
    Jul 27, 2016 at 8:31
  • 3
    The linux filesystem doesn't do anything regarding file types. That is all down to the programs running on top of it. Jul 27, 2016 at 15:29
  • 1
    @PeterGreen Yes, but the fact that the programs do assign it significance means it's not "just for humans" the way, e.g., classic MacOS had it [there were four-byte "file type" and "creator app" fields that weren't part of the file name, so the OS and applications had all the information they needed without looking at file extensions]
    – Random832
    Jul 27, 2016 at 15:36
  • 5
    @PeterGreen The Windows filesystem doesn't do anything regarding file types either. The graphical shell (Windows Explorer) uses file extension to choose an action for double-click, but technically that's just a program running on top of the OS, just as Nautilus is. It would be perfectly possible to write a Linux file manager with that behaviour, or a Windows one which examined the file contents.
    – IMSoP
    Jul 27, 2016 at 17:19
24

I'd like to take a different approach to this from other answers, and challenge the notion that "Linux" or "Windows" have anything to do with this (bear with me).

The concept of a file extension can be simply expressed as "a convention for identifying the type of a file based on part of its name". The other common conventions for identifying the type of a file are comparing its contents against a database of known signatures (the "magic number" approach), and storing it as an extra attribute on the file system (the approach used in the original MacOS).

Since every file on a Windows or a Linux system has both a name and contents, processes which want to know the file type can use either the "extension" or the "magic number" approaches as they see fit. The metadata approach is not generally available, as there is no standard place for this attribute on most file systems.

On Windows, there is a strong tradition of using the file extension as the primary means of identifying a file; most visibly, the graphical file browser (File Manager on Windows 3.1 and Explorer on modern Windows) uses it when you double-click on a file to determine which application to launch. On Linux (and, more generally, Unix-based systems), there is more tradition for inspecting the contents; most notably, the kernel looks at the beginning of a file executed directly to determine how to run it; script files can indicate an interpreter to use by starting with #! followed by the path to the interpreter.

These traditions influence UI design of programs written for each system, but there are plenty of exceptions, because each approach has pros and cons in different situations. Reasons to use file extensions rather than examining contents include:

  • examining file contents is fairly costly compared to examining file names; so for instance "find all files named *.conf" will be a lot quicker than "find all files whose first line matches this signature"
  • file contents can be ambiguous; many file formats are actually just text files treated in a special way, many others are specially-structured zip files, and defining accurate signatures for these can be tricky
  • a file can genuinely be valid as more than one type; an HTML file may also be valid XML, a zip file and a GIF concatenated together remain valid for both formats
  • magic number matching might lead to false positives; a file format that has no header might happen to begin with the bytes "GIF89a" and be misidentified as a GIF image
  • renaming a file can be a convenient way to mark it as "disabled"; e.g. changing "foo.conf" to "foo.conf~" to indicate a backup is easier than editing the file to comment out all of its directives, and more convenient than moving it out of an autoloaded directory; similarly, renaming a .php file to .txt will tell Apache to serve its source as plain text, rather than passing it to the PHP engine

Examples of Linux programs which use file names by default (but may have other modes):

  • gzip and gunzip have special handling of any file ending ".gz"
  • gcc will handle ".c" files as C, and ".cc" or ".C" as C++
3
  • Windows also has a strong tradition of hiding the extension if it's "well known" and even DOS allowed a command to omit .COM, .BAT, and .EXE, automatically searching for those to determine what actual program to execute. There is no such tradition in *nix. Jul 28, 2016 at 21:59
  • This is a much better answer but has one factual error... a script cannot be made executable by placing #! at the beginning. Any file with its executable bit(s) set can executed in one of several ways. #!/bin/bash and similar signatures just specify which interpreter to use. If no such signature is supplied, the default shell interpreter is assumed. A file containing nothing but the two words 'Hello World', but with its execution bit set, will attempt to find a 'Hello' command when run. Aug 2, 2016 at 21:33
  • 2
    @DocSalvager Good catch, that was clumsy wording as much as anything. I've reworded it a bit to make clear that the shebang doesn't make the script executable, it just changes how it is executed.
    – IMSoP
    Aug 2, 2016 at 22:08
16

Actually, some technologies do rely on file extensions, so if you use those technologies in Ubuntu, you'll have to rely on extensions too. A few examples:

  • gcc uses extensions to distinguish between C an C++ files. Without the extension it's pretty much impossible to differentiate them (imagine a C++ file with no classes).
  • many files (docx, jar, apk) are just particularly structured ZIP archives. While you can usually infer the type from the content, it may not always be possible (e.g. Java Manifest is optional in jar files).

Not using file extensions in such cases will only be possible with hacky workarounds and is likely to be very error-prone.

4
  • Good on you for mentioning programming, but you got most of the details wrong. gcc is the front-end for C files, for C++ files you need either the g++ front-end or a command-line switch to specify language. More important is the make program that decides whether to use gcc or g++ to build a particular file -- and make is completely dependent on filename patterns (mostly extensions) for its rule-matching.
    – Ben Voigt
    Jul 29, 2016 at 18:50
  • @BenVoigt When compiling a file with a .cc extension with gcc, it really will be compiled as C++, and this is documented in man gcc: "For any given input file, the file name suffix determines what kind of compilation is done:" followed by a list of extensions and how they are treated.
    – hvd
    Jul 30, 2016 at 10:53
  • 1
    @hvd Then maybe it's the default set of libraries that goes horribly wrong if you don't use the right frontend. Anyway make is the prime example because everything it does is based on file extension.
    – Ben Voigt
    Jul 30, 2016 at 13:28
  • 2
    @BenVoigt make is a good example too, but gcc relies just as heavily on filenames. Here's an example clearer than .c vs .cc: For C, gcc uses suffixes to tell if its first step is to preprocess (.c), compile (.i), assemble (.s), or link (.o). Here, I use -E, -S and -c to tell gcc where to stop, but it uses filenames to know where to start. gcc something.cc won't link to the right libraries for C++ but it will treat the file as C++, which is why many users are confused by the error messages they get when making that mistake. Jan 24, 2017 at 13:56
6

Your first assumption is correct: the extensions on Linux do not matter and only are useful for humans( and other non-Unix-like OS that care about extensions ). The type of a file is determined by first 32 bits of data in the file , which is known as magic number This is why shell scripts need #! line - to tell operating system what interpreter to call. Without it , the shell script is just text file.

As far as file managers go, they do want to know extensions of some files, such as .desktop files , which basically same as Window's version of shortcuts but with more capabilities. But as far as OS is concerned, it needs to know what's in the file, not what's in its name

12
  • 4
    This isn't quite true. There are programs that expect a specific extension. The most commonly used example is probably gunzip which won't decompress a file if it isn't called foo.gz.
    – terdon
    Jul 27, 2016 at 9:27
  • 8
    For the most part they don't, no. Your first sentence, however, claims that they are never used and only matter to humans. That isn't entirely true. gunzip is one example, eog is another. Also, many tools won't autocomplete names without the right extension. All I'm saying is that it's a bit more complicated than "extensions are always irrelevant".
    – terdon
    Jul 27, 2016 at 9:40
  • 1
    1 small issue: OP asked about the operating system. 'gunzip' and 'eog' are not the operating system but decided to create their own restrictions (in case of gunzip) or method (eog). "mime types" though.
    – Rinzwind
    Jul 27, 2016 at 17:49
  • 2
    @Serg Sure, you can define OS narrowly, and get a trivial answer to the question. It's not a particularly helpful answer, though, because the vast majority of what a user does with a computer involves software you've excluded. Note that the question contrasted "only for humans" against "the operating-system"; I don't think they meant "the kernel".
    – IMSoP
    Jul 29, 2016 at 9:06
  • 2
    @Serg "Clearly operating system isn't a very debatable topic" Uh, citation needed. Canonical calls Ubuntu an "operating system." You say an operating system is "composed out of kernel and several basic services," but you do not state which "basic services" you think those are. I'd be particularly interested to know which parts of systemd you consider to be operating system components and which you do not. This answer implicitly uses a notion of "operating system" that is vague and not what most people mean by that term, especially in *nix circles. Jan 25, 2017 at 17:07
5

This is a too big for a comment answer.

Keep in mind that even "extension" has a lot if different meanings.

What your talking about seems to be the 3 letters after the . DOS made the 8.3 format really popular and windows uses the .3 part to this day.

Linux has a lot of files like .conf or .list or .d or .c that have meaning, but are not really extensions in the 8.3 sense. For example Apache looks at /etc/apache2/sites-enabled/website.conf for it's configuration directive. While the system uses MIME Types and content headers and what not to determine it's a text file, Apache (by default) still isn't going to load it without it ending in .conf.

.c is another great one. Yep it's a text file, but gcc depends on main.c becoming main.o and finally main (after linking). At no time does the system use the .c, .o or no extension to have any meaning as far as content, but the stuff after the . does have some meaning. You would probably setup your SCM to ignore main.o and main.

Point being is this: Extensions are not used the way they are in windows. The kernel will not execute a .txt file because you remove the .txt part of the name. It is also very happy to execute a .txt file if the execute permission is set. That being said, they do have meaning, and are still used on a "computer level" for many things.

6
  • 2
    Windows is also not bound to the x.3 naming scheme any more, you have got longer extensions there as well like .doxc, .torrent, .part, etc. It's just that many file formats and extensions were already defined back in the time when 8.3 naming was still a thing and later formats mostly simply adapted the convention of using up to 3 letters.
    – Byte Commander
    Jul 27, 2016 at 10:07
  • 1
    I don't see how ".conf", ".c", etc, are "a different meaning" from "the 8.3 sense". The concept of a file extension can be simply expressed as "a convention for identifying the type of a file based on part of its name". Not even DOS / Win3.1 required the correct extension (you could call a Word document "STUPIDN.AME" and open it with Ctrl-O in WinWord). It's just that some systems (e.g. double-click on Windows, gzip, your Makefile, etc) may be written to use this convention to make assumptions about the correct action to take on each file.
    – IMSoP
    Jul 27, 2016 at 16:42
  • @ByteCommander That's true, but the extension still determines the app used. I am not sure how to edit the answer to reflect that.
    – coteyr
    Jul 27, 2016 at 20:16
  • 2
    @coteyr Again, it all depends what we mean by "the OS". The File Manager will certainly look up a registry key for "AME", and will tell me that "foo.txt" is a text file. But running dir at a command prompt will tell me no such thing; it simply won't care. Executing files is certainly an exception, on both OSes; if the question was limited to those, the answer would be that DOS/Windows only care about the name, and Unix/Linux only care about the execute permission and the first bytes of the file. Outside of that, there is always some application choosing a convention to follow.
    – IMSoP
    Jul 27, 2016 at 21:07
  • 2
    @coteyr You forgot *.scr (screen saver binary) in Windows 3.1 and up. That said, the file extension even in DOS/Windows systems even for executables is still just a convenience. The specifics depend very much on where you draw the line of "operating system", but you can always load a binary into memory and jump into it yourself, doing the work that one normally asks of the OS to do. In MS-DOS, if you look through command.com, I'm pretty sure there's a list like EXE COM that you can edit such that it looks for other extensions if none is specified (not saying it'd be a good idea, mind you).
    – user
    Jul 30, 2016 at 11:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .