just-in-time compiler (JIT)
A just-in-time (JIT) compiler is a program that turns bytecode into instructions that can be sent directly to a computer's processor (CPU). Typically, compilers are key in deciding the speed of an application for developers and end users. Just-in-time compilers can be used for performance optimization to improve application runtime.
As opposed to other compiler types, the main defining characteristic of a JIT compiler is that a JIT compiler runs after a program starts and compiles code. A common way to say this is that a JIT compiler compiles code on the fly, or in other words, just in time. The compiled code, or bytecode sequences, are turned into a faster, more readable machine language, or native code. Translating source code into a language that a host CPU natively understands typically means a faster, more easily readable instruction set.
JIT compilers contrast different compiler types such as a traditional compiler, which will compile all code to a machine language before a program starts to run. Newer programs will make use of JIT compilers, which generate code while the program is running. Two common uses of JIT compilers include Java Virtual Machine (JVM) which is used in Java, as well as CLR (Common Language Runtime) which is used in C#.
For example, in the Java programming language and environment, a just-in-time (JIT) compiler turns Java bytecode -- a program that contains instructions that must be interpreted -- into instructions that can be sent directly to the processor. JIT compilers are utilized commonly in a Java Runtime Environment, as they are used to optimize performance in java-based applications.
How does just-in-time compilation work?
Bytecode is an important feature in applications that help in cross-platform execution. Additionally, bytecode must be compiled and translated to a language a CPU can properly understand. However, how that bytecode is translated into a native language may have a large impact on the speed and performance of an application. To improve performance, JIT compilers will, at runtime, compile suitable bytecode sequences into machine code. The code is typically sent to a processor, where the code instructions are then carried out. When the same block of bytecode is needed again, the previously created object code will be used. Code that looks like it can be re-optimized is called "hot." Code can be monitored, and hot code paths can be created to optimize code, as opposed to having the same sequence of code be interpreted multiple times -- which may occur in other compiler types. With less chance of code being interpreted multiple times, there is less overhead, meaning faster execution speeds. This is why most implementations of JVM use JIT compilers.
A JIT compiler can also make relatively simple optimizations when compiling bytecode into a native machine language. As an example, a JIT compiler can get rid of common sub-expressions, reduce memory access in register allocations, and perform data-analysis and register operations by translating from stack operations.
However, because of the time it takes to load and compile bytecode, there is a startup delay in the initial execution of an application. To help anticipate startup times, a good rule of thumb to follow is that the more JIT compilers are used to optimize a system, the longer the initial startup delay will be. There are a few ways to decrease initial startup delays, such as separating startup modes. For example, including both a client and server mode could allow minimal compilation and optimization when in one mode versus the other, meaning that the chosen mode will have a faster startup time. Another way is to combine JIT compilers with either AOT (ahead-of-time) compilers or interpreters.
Phases of just-in-time compilation
As mentioned, a JIT compiler will compile suitable bytecode sequences into machine code, and then the code is sent to a processor where the code instructions are carried out.
JIT compilation can also be separated by different levels of optimization: cold, warm, hot, very hot and scorching. Each optimization level is set to provide a certain level of performance. The initial, or default, level of optimization is called warm, while code that looks like it can be further re-optimized is called hot. This level increases upwards until scorching, with each level being of higher importance pertaining to performance that can be re-optimized. Through sampling, a JIT compiler can determine which methods appear more often at the top of a stack. The optimization level can also be downgraded to cold, to further improve startup time.
Just-in-time compilers vs. interpreters vs. ahead-of-time compilers
In general, interpreters and compilers can both be used to translate machine language. An interpreter will commonly perform tasks such as Parsing, type checking and lexing, and does the same kind of work as a compiler. Interpreters have a fast startup time, don't have to go through a compilation step and will translate code one line at a time, on the fly. However, an interpreter may be slower than a compiler in a case where an application runs the same code multiple times -- since the same translation must happen as many times as the code is repeated.
A typical compiler does not translate on the fly, as it translates compiled code before the application begins running. Because of this, its startup time is a bit longer, but any code loops that appear will run faster -- since the code translation does not need to be repeated multiple times. Compilers have more time to look at the code and make any optimizations as well.
JIT compilers combine both principals to get the best of both worlds.
Ahead-of-time, or AOT compilers, compile code into a native machine language similar to a normal compiler; however, AOT will transform bytecode of a virtual machine into machine code.
Advantages of just-in-time compilation
Advantages of JIT compilation include:
- JIT compilers need less memory usage.
- JIT compilers run after a program starts.
- Code optimization can be done while the code is running.
- Any page faults can be reduced.
- Code that is used together will be localized on the same page.
- Can utilize different levels of optimization.
Disadvantages of just-in-time compilation
Disadvantages of JIT compilation include:
- Startup time can take a noticeable amount of time.
- Heavy usage of cache memory.
- Increases the level of complexity in a Java program.
What do just-in-time compilers do in Java?
In the past, most programs written in any language have had to be recompiled, and sometimes rewritten, for each computer platform. One of the biggest advantages of Java is that programs only have to be written and compiled once. The Java on any platform will interpret the compiled bytecode into instructions understandable by the particular processor. However, the virtual machine handles one bytecode instruction at a time. Using the Java just-in-time compiler -- really a second compiler -- at the particular system platform compiles the bytecode into the particular system code. This is as though the program had been compiled initially on that platform. Once the code has been recompiled by the JIT compiler, it will usually run more quickly in a computer.
