1

I'm running VirtualBox 4.3.18 on OSX 10.9.5 .

I install Ubuntu Server 14.04 64bit and performed this basic disk write test:

dd if=/dev/zero of=bigfile bs=1024 count=5000000

Setup of host (MacBook Pro 2014):

  • 16GB RAM
  • 500GB SSD

The setup of the guest machine:

  • 2GB RAM
  • 60GB VDMK disk (only occupies 1-2 GB after minimal installation)

The result with 14.04 / 1 CPU: 13.6s / 376 MB /s
The result with 14.04 / 2 CPU: 41.5s / 123 MB /s

On 12.04 64bit with either 1 or 2 CPUs doesn't change the speed from around 376 MB/s

I came up with this simple dd test after noticing that just many things felt slow compared to my previous setup with 12.04. Running puppet, database migration, etc. Everything noticeable degraded after I ran a 14.04 with 2 CPUs.

The setup is made with the official 14.04 amd64 image and the standard installation procedure (just saying yes to all dialogs), no fancy/custom settings.

Any ideas what the problem could be?

Edit: Things I tried (with no effect)

  • use different Motherboard chipset
  • use IDE instead of SCSI (already during installation)
  • use different IDE chipsets

Edit 2: vmstat comparison

Before writing the 5GB zeroes, I startet vmstat and quit it afterwards; these are the logs. Maybe they help someone who understands them?

vmstat with 1 CPU

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 1939068  16684  45944    0    0  2887    11  207 1421  5  9 85  0  0
 1  0      0 1834700  16844 147360    0    0   160     0  136  143  5 21 73  1  0
 0  1      0 1494508  16880 477948    0    0    32 196640 1742  215 17 80  0  3  0
 1  0      0 1216688  16904 748316    0    0    12 336720 3065  357 13 83  0  4  0
 1  1      0 916324  16912 1040380    0    0     8 300164 2490  275 15 85  0  0  0
 1  1      0 623784  16924 1325020    0    0    12 290818 2590  304 14 86  0  0  0
 1  0      0 331540  16932 1609576    0    0     8 277380 2602  261 15 84  0  1  0
 1  0     92  84544   5816 1860588    0   92     8 266408 2373  275 14 85  0  1  0
 3  0    416  69840     96 1880244    0  324     8 326136 2865  493 16 84  0  0  0
 1  0    724  81064     88 1869532    0  308    12 366776 3180  556 16 84  0  0  0
 1  0   1052  69352     92 1881392    0  328    12 336204 3074  541 17 82  0  1  0
 1  1   1480  80160     96 1870952    0  428    16 345776 2909  506 16 84  0  0  0
 1  0   1936  78492    104 1873396    0  456    12 347420 3146  547 16 81  0  3  0
 1  0   2544  67684     96 1884676    0  608     8 320156 2890  481 17 83  0  0  0
 3  1   3108  80780     96 1871980    0  564    12 350136 2974  507 16 83  0  1  0
 2  0   3548  79064     84 1873748    0  440    12 355388 3107  570 16 84  0  0  0
 1  0   3988  72260     96 1881724    0  440    16 319932 2990  505 17 83  0  0  0
 0  0   4112  77012    248 1877096    0  124  2712 269248 2306  350  4 43 38 14  0
 0  0   3980  75284    256 1878256  608    0  1748    76   63  110  0  2 97  1  0

vmstat with 2 CPUs

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0   4728  72916    772 1879116    1    8   166 17813  252  111  7 13 79  1  0
 1  0   4728 1965416    852  37852    0    0    80     0  313  246  6 19 74  1  0
 1  0   4728 1838368    852 161568    0    0     0     0  494   21 21 29 50  0  0
 1  0   4728 1726976    856 269924    0    0     4 106516 1388  237 19 32 44  6  0
 1  0   4728 1604976    860 388704    0    0     4 106500 1357  189 22 36 40  3  0
 1  0   4728 1487140    880 503360    0    0     8 229588 2272  213 21 43 27  9  0
 1  0   4728 1357332    880 629844    0    0     0     0  506   33 22 29 50  0  0
 1  1   4728 1234228    884 749264    0    0     4 137664 1429  103 22 37 39  3  0
 1  0   4728 1112300    888 867996    0    0     4 128772 1629  123 21 37 38  4  0
 1  0   4728 987976    892 989032    0    0     4 106500 1329   83 22 35 41  3  0
 1  0   4728 871432    912 1102492    0    0    12 229424 2270  246 21 43 27  9  0
 1  0   4728 743556    912 1226964    0    0     0     0  486   18 22 29 50  0  0
 2  0   4728 620096    916 1346932    0    0     4 108188 1338   98 21 36 40  3  0
 1  0   4728 498700    920 1465012    0    0     4 158084 1737  145 22 39 35  4  0
 1  0   4728 375956    924 1584600    0    0     4 106500 1335   92 22 36 40  3  0
 1  0   4728 258288    940 1698692    0    0     8 225324 2223  210 21 43 27  9  0
 1  0   4728 132520    940 1821260    0    0     0     0  501   20 22 29 50  0  0
 1  0   4996  66768    112 1885580    0  268   220 106788 1511  219 19 34 41  5  0
 1  2   5012  71568    116 1880736    0   16     8 134292 1479  314 22 37 38  3  0
 1  0   5052  78080    116 1874516    0   40     4 132012 1719  275 21 40 36  4  0
 1  0   5088  67908    120 1884508    0   36     4 225356 2301  361 22 44 26  8  0
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0   5132  74848    112 1877920    0   44     0    44  589  263 22 29 49  0  0
 1  0   5172  72924    112 1879676    0   40     4 106556 1385  249 21 37 40  3  0
 1  1   5240  78988    116 1873796    0   68     4 137672 1476  277 22 37 38  3  0
 1  0   5308  70256    116 1882532    0   68     4 128712 1699  257 22 39 36  4  0
 1  0   5340  77468    136 1875524    0   32    16 225360 2289  366 21 44 27  8  0
 2  0   5388  77840    128 1875324    0   48     0    48  580  242 22 30 49  0  0
 2  0   5440  69500    120 1883484    0   52     4 106576 1387  248 22 37 39  3  0
 2  0   5496  76928    124 1876008    0   56     4 117564 1407  275 22 36 40  3  0
 1  0   5552  68904    124 1883932    0   56     4 148796 1754  281 21 40 35  4  0
 1  0   5600  70356    136 1882664    0   48     8 225372 2320  381 22 44 26  8  0
 1  0   5668  69292    132 1883832    0   68     0    68  572  241 22 30 49  0  0
 1  0   5720  74508    132 1878824    0   52     0 106572 1393  258 21 37 39  3  0
 1  0   5768  67348    120 1885924    0   48     4 106548 1494  272 22 37 39  3  0
 1  1   5836  73112    120 1880196    0   68     4 145992 1566  299 21 38 37  4  0
 1  0   5892  68260    132 1885004    0   56     8 239200 2562  400 22 46 24  9  0
 1  0   5976  66812    128 1886448    0   84     0    84  588  246 21 30 49  0  0
 1  0   6068  74444    132 1878868    0   92     4 106608 1407  316 22 36 39  3  0
 1  0   6140  65820    124 1887572    0   72     4 106572 1389  253 22 37 40  3  0
 1  1   6220  64992    124 1888248    0   80     4 137556 1506  288 22 37 38  3  0
 1  0   6296  69388    140 1884176    0   76    12 247672 2625  421 21 46 23  9  0
 1  0   6400  73392    124 1880312    0  104     0   104  599  237 22 30 49  0  0
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0   6484  71628    124 1882636    0   84   656 106604 1419  276 22 37 39  3  0
 0  1   6556  79784    120 1873452    0   72     8 221772 2087  355 17 37 39  7  0
 0  0   6556  78296    120 1876500    0    0  3036 39232  574  195  0  5 93  2  0

Edit 3: iostat comparison

Since it's too much for the question, I created a pastie with iostat comparison:

iostat with 1 CPU: http://pastie.org/private/ne8llizdt9pgnzt7gu2vbq

iostat with 2 CPUs: http://pastie.org/private/xsgzoq7zrlzarbwqkqdbq

Improve this question
5
  • How much RAM for the VM and is it 32-bit or 64?
    – harrymc
    Commented Nov 6, 2014 at 8:14
  • @harrymc added the RAM (2GB); arch is 64bit but that was already there.
    – mark
    Commented Nov 6, 2014 at 10:36
  • My guess is that this is a bug introduced in the disk driver of Ubuntu 14.04, triggered for multiple CPUs by the virtual disk being too fast for the driver. A way to test this guess might be to change the disk type from IDE to SCSI (or vice verso) and hope that the other driver doesn't have the same bug.
    – harrymc
    Commented Nov 6, 2014 at 12:13
  • @harrymc thanks, I tried that already, makes no difference. I updated the question, thanks. But your mention of a driver/bug makes me curious. Unfortunately I lack the kernel knowledge but would love to dig deeper, any pointers for me?
    – mark
    Commented Nov 6, 2014 at 19:38
  • The bug might then exist in some code that is common to both disk drivers. Not an Ubuntu kernel developer, I cannot point directly to the bad code. I know that such a bug is possible, for example this one from 2009, but you should rather address the Ubuntu developers.
    – harrymc
    Commented Nov 7, 2014 at 7:04

1 Answer 1

Reset to default
1

I've found the culprit to be that the Ubuntu 14.04.1 Server amd64 image kernel is compiled with CONFIG_CONTEXT_TRACKING_FORCE=y which, according to http://cateee.net/lkddb/web-lkddb/CONTEXT_TRACKING_FORCE.html :

Say Y only if you're working on the development of an architecture backend for the context tracking.
Say N otherwise, this option brings an overhead that you don't want in production.

Either recompiling the kernel without the flag or upgrading to a kernel which doesn't have the flag set fixes the problem for me. The difference is noticeable:

Multiple CPU, stock 14.04.1 kernel (3.13.0-32):

vagrant@ubuntu-14:~$ dd if=/dev/zero of=bigfile bs=1024 count=5000000
5000000+0 records in
5000000+0 records out
5120000000 bytes (5.1 GB) copied, 46.6511 s, 110 MB/s

Multiple CPUs, recompiled kernel (3.13.0-39 ; but I tested this specific kernel with both flag enabled/disabled):

vagrant@ubuntu-14:~$ dd if=/dev/zero of=bigfile bs=1024 count=5000000
5000000+0 records in
5000000+0 records out
5120000000 bytes (5.1 GB) copied, 10.4548 s, 490 MB/s

Some further information I found:

Thanks to @harrymc for motivating me to dig deeper.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .