Использование maven для сборки больших модульных c++ проектов на примере Odin Service Automation
- 1. Использование maven для сборки
больших модульных C++ проектов
на примере Odin Service Automation
Михаил Ко��еров
Sr. Software Developer @ Odin / Parallels
- 2. Agenda
It’s all about “last mile” – how to effectively build a product from
your C++ code
1. Product description
2. Old build technology
3. Maven overview
4. Maven for C++ in OSA
5. Adoption costs & conclusion
1
- 3. Odin Service Automation: Purpose and Functions
http://www.odin.com/products/automation/product-overview/
Odin Service Automation – platform to automate cloud services delivery:
– Shared web hosting (Apache, IIS)
– VPS hosting (Virtuozzo)
– Mail hosting (Exchange, Qmail)
– APS-enabled: Office 365, Lync, etc.
• Automates full business cycle of cloud services delivery: online-store,
order placement, payments processing, service provisioning, upsales,
billing
• Users – Cloud Service Providers managing Data Centers
2
- 4. Odin Service Automation: Traits
• Started in 1999, more than >12 major releases
• > 300 of devs / QA / PM in MSK and NSK
• Multiplatform:
– Windows: MS Windows Server 2008, 2012
– Linux: CentOS / RHEL 5, 6, 7
• Distributed deployment: affects testing
• Multilingual: C++, C, Java, C#, Python
3
- 5. Odin Service Automation: Structure
4
Apache
IIS
chief
SaaS
VPS
Exchange
C++ / JAVA
PLATFORM
… - C++ service controller
- communication via CORBA
- 6. CORBA
5
Technology to build distributed programs in a language
agnostic way
APS SC
Apache IDL file
C++ Client
stub
Apache SC
C++ Server
stub
call
generate generate
- C++ CORBA derivatives: .h, .cpp, .inl, …
- 7. Odin Service Automation: Source Code
6
Lang. What is written Files
count
LOC
C/C++ Platform
Libraries
Service controllers
5300* 1.5M
Java Platform
UI
7100 1M
C# Windows
Provisioning Engine
830 325K
Python Upgrade utilities 300 60K
TOTAL 13530 2.9M
80
10
10
Build Time Consumers (%)
C++ Java Other
- 8. Factors which Affects Build System
• Multiplatform – different compilers and tools
• Rich dependencies between service controllers –
correct build order of service controllers is req-d
• Large – small overheads starts to matter
• Distributed deployment – need expensive resources to
check if build is valid (BVT test), not available for devs
• Many people – many commits which could break the
build, ideally need a build for each commit in order to
test it
7
- 9. Old Build System Overview
8
• BSD make (pmake) on Linux, MSBuild on
Windows
• Harness scripts on bash + python + perl
• C++ distributed compilation: Incredibuild / Distcc
• ccache
- 10. Flaws of An Old Build System, p. I
9
1) High “cost of ownership”: hand written makefiles, no
frontend (cmake or qmake):
.for _t in all build package rpm install clean cleanup
${_t}: ${BUILD:S,^,${_t}-,}
.endfor
…
_path-${_c}= ${_srcs-${_c}:H:u:S,^,${.CURDIR}/,}
2) “Monolithic” nature of a build – problem in one module
blocks whole build
- 11. Flaws of An Old Build System, p. II
10
3) Bad incremental builds
– Non-reliable
– Slow
Frequent CLEAN
builds
Windows Linux
other 20 18
c++ 75 58
75
58
20
18
0
10
20
30
40
50
60
70
80
90
100
Minutes
Clean Build Duration (old tech.)
- 12. Build Process
11
1. Read makefiles
2. Build product
dep. graph
3. Recurse for each component
according to dep. graph
build.mk
build.master.mk
build.submake.mk
components.mk
components.sc.mk
…
Submake
?
2. Build component
dep. graph
3. Build component
+-
- 13. Make: Curse of the Recurse, p. I
12
a.cpp b.cpp c.cpp d.cpp g.cppf.cppe.cpp
l.cpp
j.cppi.cpp
a.h b.h c.h d.h g.hf.he.h
l.h
j.h
h.hk.h
i.h
SC1 SC2 SC…n
All
Submake 1
Submake 2 Submake n
- 14. Make: Curse of the Recurse, p. II
13
GIGO principle: Garbage In, Garbage Out
• Poorly defined C++ projects boundaries – no isolation:
implicit dependencies between projects by referencing
files from several global pools
• Aggravated by the use of auto-generated C++ CORBA
derivatives
• Manual tweaking of a build order
- 15. Make: Slowness Reasons
14
• Expensive submakes: each submakes analyzes the same
things as its peers and its master
• Too much overhead on externals programs: C++ auto deps,
files copying, etc.
- 16. OSA Build Technology Requirements
15
• SUPPORT MULTIPROJECT CONFIGURATION WITH
COMPONENTS ISOLATION
• FAST COMPILATION OF C++
• FAST AND RELIABLE INCREMENTAL BUILDS
• Break “monolithic” nature of builds - modularization
• Lower “Cost of ownership”: get rid of scripts “zoo”, easy
customization of a build process (C++ CORBA derivatives
generation, EDL-processing, PPM packaging, etc.)
- 17. What is an Apache Maven?
16
• Build automation tool
• De-facto – industry standard for managing Java projects
• 60% of Java developers use Maven
• More that 100 000 Open Source Projects is managed by
Maven
http://maven.apache.org
- 18. Why Apache Maven? p. I
17
1) Inherent support of multiproject configuration
2) Dependency management and versioning
• Solves common dep. management problems: transitive
dependencies, cycles, different versioning arbitration, scoping
• Inherent support of artifact repositories: local, remote
• Sophisticated artifacts versioning: coordinates { groupId,
artifactId, version, packaging, classifier }
- 19. Why Apache Maven? p. II
18
3) Customization through
“Build Lifecycle” concept
• Predefined set of lifecycles:
default, clean, site
• Lifecycle – set of phases.
• Concrete actions performed on
each phase is customized via
goals
• Goals are executed via plugins
Phase Goal
process-
resources
resources:resources
compile compiler:compile
process-test-
resources
resources:testResou
rces
test-compile
compiler:testCompil
e
test surefire:test
package jar:jar
install install:install
deploy deploy:deploy
- 20. Why Apache Maven? p. III
19
4) Spare costs through “Convention over
Configuration” principle
• Convention of source files layout
• Unified list of plugins used during build
• Build lifecycle
The idea: need to specify only exceptions,
everything else is by default
- 23. MAVEN
REPOSITORY
OSA “CELL” Concept
22
CELL – unified, (deps, lin/win, assembly) isolated C++
maven project for OSA which consumes and contributes
C++ artifacts (proxies) via maven bus
1.
CONSUME
PROXIES
2.
BUILD
3.
CONTRIBUTE
PROXIES
Other cells:
C++ CORBA derivatives
C++ headers
C++ obj files
C++ libraries
This cell:
C++ CORBA derivatives
C++ headers
C++ obj files
C++ libraries
- 24. Cells Dependency Graph
23
cell
Apache
cell
BrandingManager
cell
pem_clientcell
chief
cell
libservices
cell
ConfigurationManager
EDL
EDL
CORBA-FULL
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL
CORBA-CLIENT,
EDL
CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL CORBA-FULL,
CORBA-SDK,
HEADER,
LINKAGE
EDL
- 25. Cells Isolation via Proxies
24
cell
Apache
cell
ConfigurationManager
No C++ cross
component
dependencies
chief
proxies
- 26. C++ Compilation on Maven
25
Plugin Pros Cons
NAR maven plugin
http://maven-nar.github.io/
Power-full Over-engineered
Limits use of classifiers
native-maven-plugin
http://www.mojohaus.org/mave
n-native/native-maven-plugin/
“Good enough” for OSA Development abandoned
- 27. Dep. Graph
Basics of native-maven-plugin
26
1. Initialize
2. Analyze
C++ sources
Include dirs.
Lib dirs
.obj files
libraries
Any
changes?
3. Rebuild
+
-
Compiler opts.
Linker opts.
Other opts.
A B
c d e
target
f
B
e f
- 28. PADependencyImplementor, native-maven-plugin and
Proxies
27
MAVEN LOCAL REPOSITORY (~/.m2)
CELLs’ proxies
(zipped)
native-maven-plugin
UNZIPPED PROXIES
- Headers
- Obj. files
- Inl files
- C++ CORBA derivatives
2. Unzip if
missing to a
shared
location
3. Feed files
locations
(for compile & link)
1. Cell
dependencies:
list of proxies
PADependencyImplementor
- 29. native-maven-plugin and Proxies
28
g++ -g -pipe -fPIC -DPIC -Wall -W -fno-strict-aliasing -Werror -Wno-parentheses -D_REENTRANT
...
-I/root/.m2/_pa_proxy/com-parallels-poa-sh/apache-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-poa-sh/iis-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-poa-peripheralBus/libservices-assembly-PADEP-CORBA-FULL-
RHEL6/6.1.0.2752
-I/root/.m2/_pa_proxy/com-parallels-poa-vps/vpsmanager-assembly-PADEP-CORBA-FULL-RHEL6/6.1.0.3930 -
I/root/.m2/_pa_proxy/com-parallels-poa-sh/configurationmanager-assembly-PADEP-CORBA-FULL-
RHEL6/6.1.0.2727
-I/root/.m2/_pa_proxy/com-parallels-pa/pacryptoapi-PADEP-HEADER-RHEL6/0.3.2.006
...
-o lin/target/objs/Tasks.o -c cpp/Tasks.cpp -DSINGLETON_MODULE=SAAS
- 30. C++ Compilation Tuning
29
Project Linux Maven Profile Window Maven
Profile
cppProject g++, -g, -pipe, … cl.exe, /FD, /EHsc, …
cppExeProject <Lin linker opts> <Win linker opts>
cppDynamicProject -shared, … /DLL, …
cppStaticProject ar, … /LIB, …
-
- 31. C++ Build Lifecycle
30
Phase Plugin Task
initialize native-maven-plugin Read project configuration
PADependencyImple
mentor
Prepare proxies, pass it to native-maven-
plugins
Bin2CProcessor Convert binaries (e.g. png) to a C-code
generate-
sources
EDLProcessor Process .edl files which contain error
description
IDLProcessor Process CORBA interface definition files .idl
precompile native-maven-plugin Generate precompiled headers (.pch)
compile native-maven-plugin Compile & link
package DebugStripper Extract debug info from compiled files to a
separate ones
- 32. OSA Modularity
31
platform
{ cells }
peripheralBus
{ cells }sh
{ cells }
vps
{ cells } dedicated
{ cells }
…
{ cells }
mc
{ cells }
OSA
{ cells }
BENEFITS
• Extra fast OSA
build
• No failure
propagation
• Custom builds
- 33. OSA Build Duration: make-based vs maven-based
32
95
75
20,00
11
0
10
20
30
40
50
60
70
80
90
100
Win32 Linux
Minutes
Platforms
Make
Clean Incremental
39
25
5,50 4
0
10
20
30
40
50
60
70
80
90
100
Win32 Linux
Minutes
Platforms
Maven
Clean Incremental
~63% reduction
for clean build
~68% reduction
for incr. build
- 34. Factors of a Build Speed-up
33
1) Modularity: extensive reuse of prebuild things
2) Much quicker dependency analysis: scope of
search is narrower
3) No expensive subprograms: C++ autodeps,
files ops., etc.
- 35. Factors of Incremental Build Reliability
34
1) Code Isolation
2) Code Isolation
3) … and Code Isolation one more time
There is no difference between clean build and
incremental – there will be exactly the same
environment for each C++ cell.
- 36. Maven for C++ in OSA: adoption costs
35
• Converted on the course of 2 OSA releases without stopping
dev. cycle
• Converted >300 projects
• Took 2 years to complete by 2 devs
• Labor cost: 4 m/y
REALLY WORTH ITS COST
- 38. Generalized Build Scheme
37
Windows
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector
Сборка вместе всех артефакты от
всех платформ – оформление билда
Загрузка оформленного билда на
deliver сервер
Linux 1
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector +distcc+IncrediBuild
Linux …
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector
Linux n
• выкачать дерево
исходников
• скомпилировать
• загрузить на
collector+distcc +distcc