26

I am trying to analyze the QEMU source code.

I know its huge and till date there is no official documentation for it.

My main areas of concern are the Instruction cache management and TCG operation.

Any pointers to them would be helpful ?

Improve this question
3

2 Answers 2

Reset to default
48

I know full answer would be much longer, but for start I just want to bring to your attention this diagram: (now, it would be useful for you to play with gdb running QEMU, set breakpoints in functions you see in the diagram, follow code execution, etc.)

qemu source code flow design implementation diagram

Improve this answer
6
  • Thanks, VividD, i had to eventually take the hard route to find out, as did not saw your post earlier. My mistake, that i did not share the findings here. Almost most of the things you have covered here. Would like to point that tb_find_fast and tb_find_slow both search in the cache, only difference i could found was that tb_find_fast uses hash func value to find the index of TB. If the TB found i snot valid it does a sequential search using tb_find_slow.
    – Dexter
    Commented Jan 23, 2014 at 13:47
  • 4
    your illustration is very well done, what software was used here?
    – Peter Teoh
    Commented Mar 18, 2014 at 16:26
  • Seconding what Peter said. Did you generate that image from sources using some tool?
    – Martin
    Commented Jul 18, 2014 at 14:25
  • 4
    @Phil - No, a source code analysis tool was not used. Only plain drawing program, LibreOffice Draw. The diagram is somewhat informal, but I found it conveys execution flow well, helps locate main source code files etc...
    – VividD
    Commented Jul 18, 2014 at 16:52
  • @Dexter @VividD I am trying to understand how the TB size can be manipulated ... Am I right to run ddd qemu-system-arm and place breakpoints in the code say in gen_intermediate_code()? Then I run the program say, -M versatilepb -kernel ... Doing this however hangs the program because of a glib error that fails to allocate a huge amount of memory. Could someone point me in the right direction?
    – Arjun
    Commented Jul 19, 2016 at 13:25
1

Yes, the QEMU code flow has changed a good bit. I'm not going to do a libreoffice presentation but here's a couple TCG stack traces of the QEMU 5.1 MTTCG code. The first is the TCG Front-end (FE) taking the guest code and converting it to internal intermediate code in a Translation Block (TB). The TB has a max size of 512 instructions.

#0  0x00005555559ce81f in disas_insn (s=0x7fffe933d450, cpu=0x555556b2d990) at /opt/distros/qemu-5.1.0/target/i386/translate.c:4476
#1  0x00005555559dd471 in i386_tr_translate_insn (dcbase=0x7fffe933d450, cpu=0x555556b2d990) at /opt/distros/qemu-5.1.0/target/i386/translate.c:8569
#2  0x00005555558c4222 in translator_loop (ops=0x5555565ae9a0 <i386_tr_ops>, db=0x7fffe933d450, cpu=0x555556b2d990, tb=0x7fffac099900 <code_gen_buffer+134846675>, max_insns=512) at /opt/distros/qemu-5.1.0/accel/tcg/translator.c:102
#3  0x00005555559dd643 in gen_intermediate_code (cpu=0x555556b2d990, tb=0x7fffac099900 <code_gen_buffer+134846675>, max_insns=512) at /opt/distros/qemu-5.1.0/target/i386/translate.c:8631
#4  0x00005555558c2258 in tb_gen_code (cpu=0x555556b2d990, pc=18446744071591428680, cs_base=0, flags=4244144, cflags=-16252928) at /opt/distros/qemu-5.1.0/accel/tcg/translate-all.c:1743
#5  0x00005555558be77a in tb_find (cpu=0x555556b2d990, last_tb=0x0, tb_exit=0, cf_mask=524288) at /opt/distros/qemu-5.1.0/accel/tcg/cpu-exec.c:407
#6  0x00005555558bf18e in cpu_exec (cpu=0x555556b2d990) at /opt/distros/qemu-5.1.0/accel/tcg/cpu-exec.c:748
#7  0x00005555559846eb in tcg_cpu_exec (cpu=0x555556b2d990) at /opt/distros/qemu-5.1.0/softmmu/cpus.c:1356
#8  0x0000555555984f41 in qemu_tcg_cpu_thread_fn (arg=0x555556b2d990) at /opt/distros/qemu-5.1.0/softmmu/cpus.c:1664
#9  0x0000555555e5ec8d in qemu_thread_start (args=0x555556b5e0e0) at /opt/distros/qemu-5.1.0/util/qemu-thread-posix.c:521
#10 0x00007ffff3c406db in start_thread (arg=0x7fffe933e700) at pthread_create.c:463
#11 0x00007ffff396971f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

The second is the TCG Back-end (BE) translating the intermediate code into host instructions.

Thread 3 "qemu-system-x86" hit Breakpoint 9, tcg_out_op (s=0x7ffe9c000b20, opc=INDEX_op_ld_i32, args=0x7fffe933d530, const_args=0x7fffe933d4f0) at /opt/distros/qemu-5.1.0/tcg/i386/tcg-target.inc.c:2259
2259        int c, const_a2, vexop, rexw = 0;
#0  0x00005555558447d2 in tcg_out_op (s=0x7ffe9c000b20, opc=INDEX_op_ld_i32, args=0x7fffe933d530, const_args=0x7fffe933d4f0) at /opt/distros/qemu-5.1.0/tcg/i386/tcg-target.inc.c:2259
#1  0x000055555584fb40 in tcg_reg_alloc_op (s=0x7ffe9c000b20, op=0x7ffe9c00a418) at /opt/distros/qemu-5.1.0/tcg/tcg.c:3803
#2  0x000055555585078e in tcg_gen_code (s=0x7ffe9c000b20, tb=0x7fffac129880 <code_gen_buffer+135436371>) at /opt/distros/qemu-5.1.0/tcg/tcg.c:4244
#3  0x00005555558c22f1 in tb_gen_code (cpu=0x555556b2d990, pc=94290869746347, cs_base=0, flags=4244147, cflags=-16252928) at /opt/distros/qemu-5.1.0/accel/tcg/translate-all.c:1766
#4  0x00005555558be77a in tb_find (cpu=0x555556b2d990, last_tb=0x7fffac0ea700 <code_gen_buffer+135177939>, tb_exit=1, cf_mask=524288) at /opt/distros/qemu-5.1.0/accel/tcg/cpu-exec.c:407
#5  0x00005555558bf18e in cpu_exec (cpu=0x555556b2d990) at /opt/distros/qemu-5.1.0/accel/tcg/cpu-exec.c:748
#6  0x00005555559846eb in tcg_cpu_exec (cpu=0x555556b2d990) at /opt/distros/qemu-5.1.0/softmmu/cpus.c:1356
#7  0x0000555555984f41 in qemu_tcg_cpu_thread_fn (arg=0x555556b2d990) at /opt/distros/qemu-5.1.0/softmmu/cpus.c:1664
#8  0x0000555555e5ec8d in qemu_thread_start (args=0x555556b5e0e0) at /opt/distros/qemu-5.1.0/util/qemu-thread-posix.c:521
#9  0x00007ffff3c406db in start_thread (arg=0x7fffe933e700) at pthread_create.c:463
#10 0x00007ffff396971f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Once the TB is generated then tb_find returns with it and cpu_tb_exec runs it.

As prior answers indicate, there is A LOT more to the TCG.

Note that I used host and guest rather than native and target. TCG has the terms somewhat reversed: the source is the guest code that is converted to run on the target, which is the host. In other words, the TCG target is the QEMU host (which makes sense for the VM code generation.)

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.