29 KiB
Introduction
During the writing of the linux-insides book I have received many emails with questions related to the linker script and linker-related subjects. So I've decided to write this to cover some aspects of the linker and the linking of object files.
If we open page the Linker page on wikipidia, we will see following definition:
In computer science, a linker or link editor is a computer program that takes one or more object files generated by a compiler and combines them into a single executable file, library file, or another object file.
If you've written at least one program on C in your life, you will have seen files with the *.o extension. These files are object files. Object files are blocks of machine code and data with placeholder addresses that reference data and functions in other object files or libraries, as well as a list of its own functions and data. The main purpose of the linker is collect/handle the code and data of each object file, turning it into the the final executable file or library. In this post we will try to go through all aspects of this process. Let's start.
Linking process
Let's create simple project with the following structure:
*-linkers
*--main.c
*--lib.c
*--lib.h
And write there our example factorial program. Our main.c source code file contains:
#include <stdio.h>
#include "lib.h"
int main(int argc, char **argv) {
printf("factorial of 5 is: %d\n", factorial(5));
return 0;
}
The lib.c contains:
int factorial(int base) {
int res = 1, i = 1;
if (base == 0) {
return 1;
}
while (i <= base) {
res *= i;
i++;
}
return res;
}
And the lib.h:
#ifndef LIB_H
#define LIB_H
int factorial(int base);
#endif
Now let's compile only the main.c source code file with:
$ gcc -c main.c
If we look inside the outputted object file with the nm util, we will see the
following output:
$ nm -A main.o
main.o: U factorial
main.o:0000000000000000 T main
main.o: U printf
The nm util allows us to see the list of symbols from the given object file. It consists of three columns: the first is the name of the given object file and the address of any resolved symbols. The second column contains a character that represents the status of the given symbol. In this case the U means undefined and the T denotes that the symbols are placed in the .text section of the object. The nm utility shows us here that we have three symbols in the main.c source code file:
factorial- the factorial function defined in thelib.csource code file. It is marked asundefinedhere because we compiled only themain.csource code file, and it does not know anything about code from thelib.cfile for now;main- the main function;printf- the function from the glibc library.main.cdoes not know anything about it for now either.
What can we understand from the output of nm so far? The main.o object file contains the local symbol main at address 0000000000000000 (it will be filled with correct address after is is linked), and two unresolved symbols. We can see all of this information in the disassembly output of the main.o object file:
$ objdump -S main.o
main.o: file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <main>:
0: 55 push %rbp
1: 48 89 e5 mov %rsp,%rbp
4: 48 83 ec 10 sub $0x10,%rsp
8: 89 7d fc mov %edi,-0x4(%rbp)
b: 48 89 75 f0 mov %rsi,-0x10(%rbp)
f: bf 05 00 00 00 mov $0x5,%edi
14: e8 00 00 00 00 callq 19 <main+0x19>
19: 89 c6 mov %eax,%esi
1b: bf 00 00 00 00 mov $0x0,%edi
20: b8 00 00 00 00 mov $0x0,%eax
25: e8 00 00 00 00 callq 2a <main+0x2a>
2a: b8 00 00 00 00 mov $0x0,%eax
2f: c9 leaveq
30: c3 retq
Here we are interested only in the two callq operations. The two callq operations contain linker stubs, or the function name and offset from it to the next instruction. These stubs will be updated to the real addresses of the functions. We can see these functions' names with in the following objdump output:
$ objdump -S -r main.o
...
14: e8 00 00 00 00 callq 19 <main+0x19>
15: R_X86_64_PC32 factorial-0x4
19: 89 c6 mov %eax,%esi
...
25: e8 00 00 00 00 callq 2a <main+0x2a>
26: R_X86_64_PC32 printf-0x4
2a: b8 00 00 00 00 mov $0x0,%eax
...
The -r or --reloc flags of the objdump util print the relocation entries of the file. Now let's look in more detail at the relocation process.
Relocation
Relocation is the process of connecting symbolic references with symbolic definitions. Let's look at the previous snippet from the objdump output:
14: e8 00 00 00 00 callq 19 <main+0x19>
15: R_X86_64_PC32 factorial-0x4
19: 89 c6 mov %eax,%esi
Note e8 00 00 00 00 on the first line. The e8 is the opcode of the call instruction with a relative offset. So the e8 00 00 00 00 contains a one-byte operation code followed by a four-byte address. Note that the 00 00 00 00 is 4-bytes, but why only 4-bytes if an address can be 8-bytes in the x86_64. Actually we compiled the main.c source code file with the -mcmodel=small. From the gcc man:
-mcmodel=small
Generate code for the small code model: the program and its symbols must be linked in the lower 2 GB of the address space. Pointers are 64 bits. Programs can be statically or dynamically linked. This is the default code model.
Of course we didn't pass this option to the gcc when we compiled the main.c, but it is default. We know that our program will be linked in the lower 2 GB of the address space from the quoute from gcc manual. In this way 4-bytes enough for this. So we have opcode of the call instruction and unknown address. When we compile main.c with all dependencies to the executable file and will look on the call of the factorial we will see:
$ gcc main.c lib.c -o factorial | objdump -S factorial | grep factorial
factorial: file format elf64-x86-64
...
...
0000000000400506 <main>:
40051a: e8 18 00 00 00 callq 400537 <factorial>
...
...
0000000000400537 <factorial>:
400550: 75 07 jne 400559 <factorial+0x22>
400557: eb 1b jmp 400574 <factorial+0x3d>
400559: eb 0e jmp 400569 <factorial+0x32>
40056f: 7e ea jle 40055b <factorial+0x24>
...
...
As we can see in the previous output, the address of the main function is 0x0000000000400506. Why it does not starts from the 0x0? You already can know that standard C program is linked with the glibc C standard library if the -nostdlib was not passed to the gcc. The compiled code for a program includes constructors functions to initialize data in the program when the program is started. These functions need to be called before the program is started or in another words before the main function is called. To make the initialization and termination functions work, the compiler must output something in the assembler code to cause those functions to be called at the appropriate time. Execution of this program will starts from the code that placed in the special section which is called .init. We can see it in the beginning of the objdump output:
objdump -S factorial | less
factorial: file format elf64-x86-64
Disassembly of section .init:
00000000004003a8 <_init>:
4003a8: 48 83 ec 08 sub $0x8,%rsp
4003ac: 48 8b 05 a5 05 20 00 mov 0x2005a5(%rip),%rax # 600958 <_DYNAMIC+0x1d0>
Not that it starts at the 0x00000000004003a8 address relative to the glibc code. We can check it also in the resulted ELF:
$ readelf -d factorial | grep \(INIT\)
0x000000000000000c (INIT) 0x4003a8
So, the address of the main function is the 0000000000400506 and it is offset from the .init section. As we can see from the output, the address of the factorial function is 0x0000000000400537 and binary code for the call of the factorial function now is e8 18 00 00 00. We already knwo that e8 is opcode for the call instruction, the next 18 00 00 00 (note that address represented as little endian for the x86_64, in other words it is 00 00 00 18) is the offset from the callq to the factorial function:
>>> hex(0x40051a + 0x18 + 0x5) == hex(0x400537)
True
So we add 0x18 and 0x5 to the address of the call instruction. The offset is measured from the address of the following instruction. Our call instruction is 5-bytes size - e8 18 00 00 00 and the 0x18 is the offset from the next after call instruction to the factorial function. A compiler generally creates each object file with the program addresses starting at zero. But if a program is created from multiple object files, all of they will be overlapped. Just now we saw a process which called - relocation. This process assigns load addresses to the various parts of the program, adjusting the code and data in the program to reflect the assigned addresses.
Ok, now we know a little about linkers and relocation. Time to link our object files and to know more about linkers.
GNU linker
As you can understand from the title, I will use GNU linker or just ld in this post. Of course we can use gcc to link our factorial project:
$ gcc main.c lib.o -o factorial
and after it we will get executable file - factorial as a result:
./factorial
factorial of 5 is: 120
But gcc does not link object files. Instead it uses collect2 which is just wrapper for the GNU ld linker:
~$ /usr/lib/gcc/x86_64-linux-gnu/4.9/collect2 --version
collect2 version 4.9.3
/usr/bin/ld --version
GNU ld (GNU Binutils for Debian) 2.25
...
...
...
Ok, we can use gcc and it will produce executable file of our program for us. But let's look how to use GNU ld linker for the same purpose. First of all let's try to link these object files with the following example:
ld main.o lib.o -o factorial
Try to do it and you will get following error:
$ ld main.o lib.o -o factorial
ld: warning: cannot find entry symbol _start; defaulting to 00000000004000b0
main.o: In function `main':
main.c:(.text+0x26): undefined reference to `printf'
Here we can see two problems:
- Linker can't find
_startsymbol; - Linker does not know anything about
printffunction.
First of all let's try to understand what is this _start entry symbol that appears to be required for our program to run? When I've started to learn programming I have learned that main function is the entry point of the program. I think you learned this too :) But actually it is not entry point, there is _start instead. The _start symbol defined in the crt1.o object file. We can find it with the:
$ objdump -S /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o: file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <_start>:
0: 31 ed xor %ebp,%ebp
2: 49 89 d1 mov %rdx,%r9
...
...
...
and we pass this object file to the ld command as first argumet (see above). Now let's try to link it and will look on result:
ld /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o \
main.o lib.o -o factorial
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o: In function `_start':
/tmp/buildd/glibc-2.19/csu/../sysdeps/x86_64/start.S:115: undefined reference to `__libc_csu_fini'
/tmp/buildd/glibc-2.19/csu/../sysdeps/x86_64/start.S:116: undefined reference to `__libc_csu_init'
/tmp/buildd/glibc-2.19/csu/../sysdeps/x86_64/start.S:122: undefined reference to `__libc_start_main'
main.o: In function `main':
main.c:(.text+0x26): undefined reference to `printf'
Unfortunately we will see even more errors. We can see here old error about undefined printf and yet another three undefined references:
__libc_csu_fini__libc_csu_init__libc_start_main
The _start symbol defined in the sysdeps/x86_64/start.S assembly file in the glibc source code. We can find following assembly code lines there:
mov $__libc_csu_fini, %R8_LP
mov $__libc_csu_init, %RCX_LP
...
call __libc_start_main
Here we pass address of the entry point to the .init and .fini section that contain code that starts to execute when program runned and the code that executes when program terminates. And in the end we see the call of the main function from our program. These three symbols defined in the csu/elf-init.c source code file. The following two object files:
crtn.o;crtn.i.
Defines the function prologs/epilogs for the .init and .fini sections (with the _init and _fini symbols respectively).
The crtn.o object file contains these .init and .fini sections:
$ objdump -S /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crtn.o
0000000000000000 <.init>:
0: 48 83 c4 08 add $0x8,%rsp
4: c3 retq
Disassembly of section .fini:
0000000000000000 <.fini>:
0: 48 83 c4 08 add $0x8,%rsp
4: c3 retq
And the crti.o contains _init and _fini symbols. Let's try to link again with these two object files:
$ ld \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crti.o \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crtn.o main.o lib.o \
-o factorial
And anyway we will get the same errors. Now we need to pass -lc option to the ld. This option will search the standard library in the paths that are pointed in the $LD_LIBRARY_PATH enviroment variable. Let's try to link again wit the -lc option:
$ ld \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crti.o \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crtn.o main.o lib.o -lc \
-o factorial
Finally we will get executable file, but if we will try to run it, we will get strange result:
$ ./factorial
bash: ./factorial: No such file or directory
What's the problem here? Let's look on the executable file with the readelf util:
$ readelf -l factorial
Elf file type is EXEC (Executable file)
Entry point 0x4003c0
There are 7 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040
0x0000000000000188 0x0000000000000188 R E 8
INTERP 0x00000000000001c8 0x00000000004001c8 0x00000000004001c8
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x0000000000000610 0x0000000000000610 R E 200000
LOAD 0x0000000000000610 0x0000000000600610 0x0000000000600610
0x00000000000001cc 0x00000000000001cc RW 200000
DYNAMIC 0x0000000000000610 0x0000000000600610 0x0000000000600610
0x0000000000000190 0x0000000000000190 RW 8
NOTE 0x00000000000001e4 0x00000000004001e4 0x00000000004001e4
0x0000000000000020 0x0000000000000020 R 4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 10
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame
03 .dynamic .got .got.plt .data
04 .dynamic
05 .note.ABI-tag
06
Note on the strange line:
INTERP 0x00000000000001c8 0x00000000004001c8 0x00000000004001c8
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
The .interp section in the elf file holds the path name of a program interpreter or in another words the .interp section simply contains an ascii string that is the name of the dynamic linker. The dynamic linker is the part of an Linux that loads and links shared libraries needed by an executable when it is executed, by copying the content of libraries from disk to RAM. As we can see in the output of the readelf command it placed in the /lib64/ld-linux-x86-64.so.2 for the x86_64. Now let's add pass -dynamic-linker option with the path of the ld-linux-x86-64.so.2 to the ld and will see on the result:
$ gcc -c main.c lib.c
$ ld \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crti.o \
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crtn.o main.o lib.o \
-dynamic-linker /lib64/ld-linux-x86-64.so.2 \
-lc -o factorial
Now we can run it as normal executable file:
$ ./factorial
factorial of 5 is: 120
It works! With the first line we compile the main.c and the lib.c source code files to the object files. We will get the main.o and the lib.o after execution of the gcc:
$ file lib.o main.o
lib.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
main.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
and after this we link object files of the our program with the needed system object files and libraries. We just saw simple example how to compile and link C program with the gcc compiler and GNU ld linker. In this example we have used a couple command line options of the GNU linker, but it supports much more command line options than -o, -dynamic-linker and etc. Moreover GNU ld has own language that allows to control of the linking process. In the next two paragraps we will look it.
Useful command line options of the GNU linker
As I already wrote and as you can see in the manual of the GNU linker, it has big set of the command line options. We've seen a couple of options in this post: -o <output> - that tells ld to produce an output file called output as the result of linking, -l<name> that adds the archive or object file specified by the name, -dynamic-linker that specifies the name of the dynamic linker. Of course the ld supports much more command line options, let's look on some of it.
The first useful command line option is @file. In this case the file specifies filename where command line options will be read. For example we can create file with the name linker.ld, put there our command line arguments from the previous example and execute it with:
$ ld @linker.ld
The next command line option is -b or --format. This command line option specifies format of the input object files ELF, DJGPP/COFF and etc. There is command line option for the same purpose but for the output file: --oformat=output-format.
The next command line option is --defsym. Full format of this command line option is the --defsym=symbol=expression. It allows to create global symbol in the output file containing the absolute address given by expression. We can find following case when this command line option can be useful. For example let's look in the Linux kernel source code and more precisely in the Makefile that related to the kernel decompression for ARM architecture - arch/arm/boot/compressed/Makefile. We can find following definition there:
LDFLAGS_vmlinux = --defsym _kernel_bss_size=$(KBSS_SZ)
As we already know, it defines the _kernel_bss_size symbol with the size of the .bss section in the output file. This symbol will be used in the first assembly file that will be executed during kernel decompressing:
ldr r5, =_kernel_bss_size
The next command line options is the -shared that allows us to create shared library. The -M or -map <filename> command line option prints the linking map with the information about symbols. In our case:
$ ld -M @linker.ld
...
...
...
.text 0x00000000004003c0 0x112
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
.text 0x00000000004003c0 0x2a /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/crt1.o
...
...
...
.text 0x00000000004003ea 0x31 main.o
0x00000000004003ea main
.text 0x000000000040041b 0x3f lib.o
0x000000000040041b factorial
Of course the GNU linker support standard command line options: --help and --version that print common help of the usage of the ld and its version. That's all about command line options of the GNU linker. Of course it is not full set of the command line options support by the ld util. Full description you can find in the manual of this util.
Control Language linker
As I wrote previously, the ld has support of the own language. It accepts Linker Command Language files written in a superset of AT&T's Link Editor Command Language syntax, to provide explicit and total control over the linking process. Let's look on its details.
With the linker language we can control:
- input files;
- output files;
- file formats
- addresses of sections;
- and etc.
Usually commands written on linker control language placed in a file that called - linker script. We can pass it to the ld with the -T command line option. The main command in the each linker script is the SECTIONS. Each linker script must contain this command and it determines the map of the output file. The special variable - . contains current position of the output. Let's write simple assembly program and will look how we can use linker script to control linking of this program. For example it will be hello world:
section .data
msg db "hello, world!",`\n`
section .text
global _start
_start:
mov rax, 1
mov rdi, 1
mov rsi, msg
mov rdx, 14
syscall
mov rax, 60
mov rdi, 0
syscall
We can compile and link it with the:
$ nasm -f elf64 -o hello.o hello.asm
$ ld -o hello hello.o
Our program consists from tw sections: .text - contains code of the program and .data - contains initialized variables. Let's write simple linker script and try to link our hello.asm assembly file with it. Our script is:
/*
* Linker script for the factorial
*/
OUTPUT(hello)
OUTPUT_FORMAT("elf64-x86-64")
INPUT(hello.o)
SECTIONS
{
. = 0x200000;
.text : {
*(.text)
}
. = 0x400000;
.data : {
*(.data)
}
}
On the first three lines you can see comment that written with C style. After it the OUTPUT and the OUTPUT_FORMAT command specifies name of the our executable file and its format. The next command - is INPUT specfies input file to the ld linker. After all of this command we can see main SECTIONS command, as I already wrote each linker script must contain definition of this command. The SECTIONS command represents set and order of the sections which are will be in the output file. At the beginning of the SECTIONS command we can see following line . = 0x200000. I already wrote above that . command points to the current position of the output. This line says that the code should be loaded at address 0x200000 and the line . = 0x400000 says that data section should be loaded at address 0x400000. The second line after the . = 0x200000 defines .text section as an output section. We can see *(.text) expression inside it. The * symbol is wildcard that matches any file name. In another words the *(.text) expression says all .text input sections in all input files. We can rewrite it as hello.o(.text) for our example. After the following location counter . = 0x400000, we can see definition of the data section.
We can compile and link it with the:
$ nasm -f elf64 -o hello.o hello.S && ld -T linker.script && ./hello
hello, world!
If we will look inside with the objdump util, we will see that .text section starts from the 0x200000 and the .data sections starts from the 0x400000 address:
$ objdump -D hello
Disassembly of section .text:
0000000000200000 <_start>:
200000: b8 01 00 00 00 mov $0x1,%eax
...
Disassembly of section .data:
0000000000400000 <msg>:
400000: 68 65 6c 6c 6f pushq $0x6f6c6c65
...
Except of those comands that we have already seen, there are a few other linker scripts commands. The first is the ASSERT(exp, message) that ensures that given expression is not zero. If it is zero, then exit the linker with an error code and print given error message. If you've read about Linux kernel booting process in the linux-insides book, you may know that setup header of the Linux kernel has offset - 0x1f1. In the linker script of the Linux kernel we can find check for this:
. = ASSERT(hdr == 0x1f1, "The setup header has the wrong offset!");
The next INCLUDE filename command allows to include external linker script symbols to the current. In a linker script we can assign a value to a symbol. The ld support a couple of assignment operators:
- symbol = expression ;
- symbol += expression ;
- symbol -= expression ;
- symbol *= expression ;
- symbol /= expression ;
- symbol <<= expression ;
- symbol >>= expression ;
- symbol &= expression ;
- symbol |= expression ;
As you can note all operators are C assignment operators. For example we can use it in our linker script as:
START_ADDRESS = 0x200000;
DATA_OFFSET = 0x200000;
SECTIONS
{
. = START_ADDRESS;
.text : {
*(.text)
}
. = START_ADDRESS + DATA_OFFSET;
.data : {
*(.data)
}
}
As you already may noted the syntax for expressions in the linker script language is identical to that of C expressions. Besides this the control language of the linking supports following builtin functions:
ABSOLUTE- returns absolute value of the given expression;ADDR- takes the section and returns its address;ALIGN- returns the value of the location counter (.operator) that aligned by the boundary of the next expression after the given expression;DEFINED- returns1if the given symbol placed in the global symbol table and0in other way;MAXandMIN- return maximum and minimum of the two given expressions;NEXT- returns the next unallocated address that is a multiple of the give expression;SIZEOF- returns the size in bytes of the given named section.
That's all.
Conclusion
This is the end of the post about linkers. We knew many things about linkers in this post, such things like what is it linker and why we need in it, how to use it and etc..
If you will have any questions or suggestions write me email or ping me in twitter.
Please note that English is not my first language, And I am really sorry for any inconvenience. If you will find any mistakes please send let me know via emal or send a PR.