"Swoole: double troubles in c", Alexandr Vronskiy

Intro
Lifestory:
Production monolith
migration to Swoole!

● What existing problems need to solve?
● Why choose Swoole?
● How solve it with Swoole?
● What unexpected troubles could happens?
● Show methods to avoid then (or not?)
● Talk about theory
● Examples in a context of a real project
Intro

Side Effects of
- PHP no dying (long-lived)
- Concurrency Async
- Shared Memory access
Intro

● ZF2 / Laminas / Mezzio / PSR-15 / ..
● PHP 5.6 -> … -> 7.4 --> 8.0 + swoole
● PostgreSQL / ElasticSearch Typesense / Redis / RabbitMQ / PubSub
● Docker / k8s / autoscale / Google Cloud
● Files: 2.5k, LOC: 200k, LLOC: 30k
Intro. About project

Intro. About project
Current AVG metrics:
- GA online users: ~3500
- API RPS: ~200
- DB transactions: ~3000/sec
- Products count: ~25kk
Consuming resources:
- PHP: 60 vCPU
- ElasticSearch: 40 vCPU
- PostgreSQL: 16 vCPU
- Others: 20 vCPU

● Not optimized resources usage.
● Hard tuning horizontal scale.
● Over-complicated infrastructure
● Not well performance (TTFB)
Why? Problems
Database
PHP PHP
PHP PHP
Elastic Elastic
Elastic
Other
4x times

Why? Problems
PHP
4 vCPU
PHP
16 vCPU
VS

Why? Problems
Docker, k8s, Terraform, Helm, GitHub
Actions, Envs: prod+stage+devs+local, Go,
Typescript

Why? Problems

● ASYNC entire ecosystem
● Performance
● Easy to start
- Coroutine based concurrent asynchronous IO
- Event Loop
- Process management
- In-memory storage and management
- Async TCP/UDP/HTTP/WebSocket/HTTP2/FastCGI client
and servers
- Async Task API
- Channels, Locks, Timer, Scheduler
NO PECL: ext-pcntl, ext-pthreads, ext-event
Why Swoole?

Milestone 1: PHP no die
● Run HTTP Server
○ replace NGINX+FPM
○ simplify infrastructure (less DevOps, easy building & k8s configs)
○ change (unified) operations: CI / CD / local env
● Prepare bootstrap
● Implement best practices in shared memory usage to avoid side-effects
Plan

● Server Mode: SWOOLE_PROCESS / SWOOLE_BASE
● Dispatch Mode: 1-9 (Round-Robin, Fixed, Preemptive, etc)
● Worker Num: 1-1000 (CPU*2)
● Max Request: 0-XXX (0)
Other Options: Limits, Timeouts, Memory buffers...
php bin/http-server.php
Swoole HTTP Server
https://www.swoole.co.uk/docs/modules/swoole-server/conﬁguration
MUST SEE:
<?php
$server = new SwooleHTTPServer("127.0.0.1", 9501);
$server->on('Request', function(Swoole/Server/Request $request, Swoole/Server/Response $response)
{
$response->end('<h1>Hello World!</h1>');
});
$server->start();

● Scan config files, env, run reflection, attributes, build DI, generate proxy, warm caches:
● NO NEED cache layer anymore
● NOW it before real start http server
(if no traffic: readiness probe=negative)
PSR-7 HTTP Messages
PSR-15 Middleware
PSR-11 Container
bootstrap in master
http request in worker process
Bootstrap app once
fork state

What are the problems?
- NO SUPER GLOBALS ($_SERVER, ...)
- No PHP Session (it is CLI SAPI)
- Stateful services that should mutate on each request
- DI containers - global state too.
Shared Memory

Any wrong example?
Shared Memory
https://github.com/chrisguitarguy/RequestIdBundle/blob/main/src/EventListener/RequestIdListener.php
public function onRequest(RequestEvent $event) : void
{
if (!$this->isMainRequest ($event)) {
return;
}
$req = $event->getRequest();
if ($this->trustRequest && ($id = $req->headers->get($this->requestHeader )))
{
$this->idStorage->setRequestId ($id);
return;
}
if ($id = $this->idStorage->getRequestId ()) {
$req->headers->set($this->requestHeader , $id);
return;
}
$id = $this->idGenerator ->generate();
}
Empty storage - no return
Generate NEW
Saving to storage
1
HTTP REQUEST:

Any wrong example?
Shared Memory
public function onRequest(RequestEvent $event) : void
{
if (!$this->isMainRequest ($event)) {
return;
}
$req = $event->getRequest();
if ($this->trustRequest && ($id = $req->headers->get($this->requestHeader )))
{
return;
}
if ($id = $this->idStorage->getRequestId ()) {
return;
}
$id = $this->idGenerator ->generate();
}
2
HTTP REQUEST:
Now has ID in storage
THE END
Dead code
https://github.com/chrisguitarguy/RequestIdBundle/blob/main/src/EventListener/RequestIdListener.php

Shared Memory
HTTP Server options:
Max Request: 100
What if...
Then…
In logs request ID is changed.

Best practices:
- Middlewares, Factories, Proxies, Delegators
Shared Memory
use PsrHttpMessageResponseInterface
;
use PsrHttpMessageServerRequestInterface
;
use PsrHttpServerRequestHandlerInterface
;
use PsrLogLoggerInterface
;
class SomeHandler implements RequestHandlerInterface
{
public function __construct(
private Closure $appServiceFactory
,
private LoggerInterface $logger
) {}
// Idempotent method!!!
public function handle(ServerRequestInterface $request) : ResponseInterface
{
$logger = clone $this->logger;
$logger->getProcessor(
'RequestID')->setRequestId(
$request->getAttribute(
'RequestID'));
$appService = ($this->appServiceFactory)(
$logger);
return new JsonResponse($appService->createBook())
;
}
}

Memory leaks
… in Doctrine ORM :(
Shared Memory
https://alejandrocelaya.blog/2019/11/04/how-to-properly-handle-a-doctrine-entity-manager-on-an-expressive-appli
cation-served-with-swoole/
use DoctrineORMDecoratorEntityManagerDecorator as
EMD;
class ReopeningEntityManager extends EMD
{
private $createEm;
public function __construct (callable $createEm)
{
parent::__construct($createEm());
$this->createEm = $createEm;
}
public function open(): void
{
if (! $this->wrapped->isOpen()) {
$this->wrapped = ($this->createEm)();
}
}
class CloseDbConnectionMiddleware implements
MiddlewareInterface
{
public function __construct(
private ReopeningEntityManager $em)
{}
public function process(
ServerRequestInterface $request,
RequestHandlerInterface $handler
) : ResponseInterface
{
$this->em->open() ;
try {
return $handler->handle($request);
} finally {
$this->em->getConnection()->close() ;
$this->em->clear() ;
}
}
}
Saved in DI

Shared Memory
Memory leaks
BUT WHERE???
abstract class EntityManagerDecorator extends
ObjectManagerDecorator
{
/** @var EntityManagerInterface */
protected $wrapped;
public function __construct (EntityManagerInterface $wrapped)
{
$this->wrapped = $wrapped;
}
public function getRepository ($className)
{
return $this->wrapped->getRepository ($className);
}
public function getRepository ($entityName )
{
return $this->repositoryFactory ->getRepository ($this,
$entityName );
}
Somewhere far far away
in EntityManager

PROBLEMS DUE TO: timeouts / lifecycles
Avoid: Stateful convert to stateless
Connections
$redis = $di->get(Redis::class); // FactoryRedis: $redis->connect(...);
$redis->get('KEY1');
$redisFactory = $di->get(RedisFactory::
class);
$redisFactory()->get(‘KEY1’); // + close connect in desctructor
Request Wrapper/Delegator:
new/refresh state on each request
OR

Milestone 2: ASYNC
● Async theory
● Using coroutines
● Non-blocked IO solutions
● Concurrency problems review
● Again memory leaks

● GET /some/action1/ SELECT sleep(1);
○ 1 worker = 1 req/sec
● GET /some/action2/ ﬁbonacci(30);
○ 2 worker = depends on CPU cores
Why/What async?
Try benchmark this:

● GET /some/action1/ SELECT sleep(1);
● GET /some/action2/ ﬁbonacci(30);
MIX 50/50 = 1 req/sec
50% CPU
Why/What async?
Try benchmark this:

Now enable coroutines: http server options: .
'enable_coroutine' => true,
Why/What async?
● GET /some/action1/

Now enable coroutines: http server options: .
Why/What async?
MIX 50/50 = 2 req/sec
100% CPU

go(function () { // FIRST CO
echo '1';
go(function () { // SECOND CO
echo '2';
co::sleep(3); // IO (in 2 CO), will return in 3 sec
echo '6';
go(function () { // THIRD CO
echo '7';
co::sleep(2); // IO
echo "9n";
}); // END THIRD
echo '8';
}); // END SECOND
echo '3';
co::sleep(1); // Again IO but in 1 CO
echo '5';
}); // END FIRST CO
echo '4';
Coroutines
1
2
3
4
5

https://www.swoole.co.uk/docs/modules/swoole-runtime-flags
Swoole Hooks
Coroutine Clients:
● TCP/UDP
● HTTP
● HTTP2
● Socket
● FastCGI
● Redis
● MySQL
● Postgresql
Runtime HOOKS:
● Coroutine support (any IO libraries based on php_stream):
○ Redis (phpredis) or (predis)
○ MySQL (mysqlnd) PDO and MySQLi
○ PHP SOAP
○ file_get_contents, fopen and many more file I/O operations
○ stream_socket_client functions
○ fsockopen
○ CURL with libcurl
● Libraries without coroutine support:
○ MySQL with libmysqlclient
○ MongoDB with mongo-c-client
○ pdo_pgsql, pdo_ori, pdo_odbc, pdo_firebird, php-amqp
VS
Non-blocked IO

$swooleServer->onRequest: go($this->requestHandler);
--------------------------------------------------------------------------
non-blocked: multiple requests in same time/code
Concurrency
function onRequest() {
$class = $di->get(stdClass:class);
if ($class->some === null) {
$class->some = 123;
}
sleep(2);
// && some logic
$class->some = null;
}
First request awaiting here
We expected clean in request end
http server options: .

$swooleServer->onRequest: go($this->requestHandler);
--------------------------------------------------------------------------
non-blocked: multiple requests in same time/code
Concurrency
function onRequest() {
$class = $di->get(stdClass:class);
if ($class->some === null) {
$class->some = 123;
}
sleep(2);
// && some logic
$class->some = null;
}
First request awaiting here
We expected clean in request end
EPIC FAIL! in second request
http server options: .

Request Wrapper/Delegator: new/refresh state on each request
Uncaught SwooleError: Socket# has already been
bound to another coroutine
Connections. Again?
$http = new SwooleHttpServer( '0.0.0.0', '80', SWOOLE_PROCESS) ;
$http->on('request', function (SwooleHttpRequest $request, SwooleHttpResponse $response)
{
if (empty($redis)) {
$redis = new Redis();
$redis->connect('127.0.0.1' , 6379);
$redis->select(1);
$redis->setOption(Redis::OPT_SERIALIZER , Redis::SERIALIZER_PHP) ;
}
try {
$redisJson = $redis->get('key');
} catch (Exception $e) {
// SWOOLE ERROR HERE !
}
$response->end('some response' );
});

https://www.swoole.co.uk/docs/modules/redis-connection-pool
Connections. Again?
POOL

Connections. Again?
use
DoctrineCommonCacheCacheProvider ;
use Redis;
use SwooleDatabaseRedisPool ;
class SwooleRedisCacheProvider extends
CacheProvider
{
public function __construct (
private RedisPool $pool,
) {
}
protected function doFetch($id, ?Redis $redis = null)
{
if (! $redis) {
return $this->run(fn (Redis $redis) : mixed =>
. $this->doFetch($id, $redis));
}
return $redis->get($id);
}
private function run(callable $procedure)
{
$redis = $this->pool->get();
$redis->setOption(Redis::SERIALIZER, $this->getSerializer ());
try {
$result = $procedure($redis);
} catch (Throwable $e) {
throw $e;
} finally {
$this->pool->put($redis);
}
return $result;
}

Memory Leaks. Again?
final class ReopeningEntityManager implements EntityManagerInterface
{
private Closure $entityManagerFactory ;
protected array $wrapped = [];
public function clear($object = null) : void
{
$this->getWrapped()->clear($object);
unset ($this->wrapped[Co::getCid()]); // does not help!
}
private function getWrapped() : EntityManagerInterface
{
if (! isset($this->wrapped[Co::getCid()])) {
$this->wrapped[Co::getCid()] =
($this->entityManagerFactory )();
}
return $this->wrapped[Co::getCid()];
}
public function getConnection ()
{
return $this->getWrapped()->getConnection ();
}

Memory Leaks. Again?
Using WeakMap &
defer in coroutines!
Restart workers?
You are coward!

Using PostgreSQL - no PDO hooks in Swoole
● Use Coroutine Postgresql Client:
extension=swoole_postgresql.so
● Write new Driver for Doctrine ORM
● Be ready to problems
Problem Again

Other miscellaneous:
Manage crons (by timers)
Manage message workers (process manager)
Manage Signals (soft termination)
Milestone 3: others

Cron jobs
- persist Deployment: run “crontab” process + list php CLI command
- CronJob (k8s) periodical run POD with “php bin/app cron:process”
- CronJob as Message (run as “bin/app messenger:consume transport.amqp”)
- + Swoole Timer async callback in PHP master process (instead linux crontab)

Prӕfectus
https://github.com/opsway/praefectus
Supervisor & Metrics server for Symfony Messenger

Prӕfectus
PraefectusListener
use SpiralGoridgeRelay
;
use SpiralGoridgeRPCRPC
;
use SpiralGoridgeRPCRPCInterface
;
/**
* @see SymfonyComponentMessengerWorker
*/
class PraefectusListener implements EventSubscriberInterface
{
private const IPC_SOCKET_PATH_TPL = '/tmp/praefectus_%d.sock'
;
// …
public function onMessageReceived (EventWorkerMessageReceivedEvent $event) : void
{
$this->getRpc()->call('PraefectusRPC.WorkerState' ,['pid'=>getmypid() ,'state'=>self::WORKER_BUSY]);
$this->getRpc()->call('PraefectusRPC.MessageState' , [
'id' => $messageIdStamp ->id(),
'name' => get_class( $event->getEnvelope ()->getMessage()),
'transport' => $event->getReceiverName (),
'bus' => $busName,
'state' => self::MESSAGE_STATE_PROCESSING,
]);
}
}
*not yet released on GitHub

● no compatibility in code:
○ run same code in FPM & build-in http server
○ work without swoole extension (PoC - write stubs?)
○ XDebug - goodbye (use PCOV for coverage)
○ Proﬁling - https://github.com/upscalesoftware
Results

● Doctrine + async = EVIL*
Results
* But is possible, if enough extra memory:
Each concurrency EntityManager = +100-150Mb

● Swoole Table for cache - must have!
Results
Doctrine (any) cache shared between workers.
Solved case: Deployment & migration process without
50x errors

● Benchmarks - good results!
Results
- empty handler (bootstrap) - TTFB before: 140 ms, after: 4 ms
SWOOLE
FPM

● Benchmarks - good results!
Results
- empty handler (bootstrap) - TTFB before: 140 ms, after: 4 ms
- AVG (all traﬃc) TTFB reduced on 40% - before: 250 ms, after: 150 ms
- 95 percentile - before: 1500 ms, after: 600 ms
P.S. Load by same API RPS: ~200

- Now: 20 (PHP) / 20 (Typesense) / 20 (other) / 16 (DB) vCPUs
- 6k$ --> 3.5k$ (per month)
Results
● Resources usage - (ETA) reduced!

Useful links:
- https://github.com/deminy/swoole-by-examples
- https://docs.mezzio.dev/mezzio/v3/why-mezzio/
- https://docs.mezzio.dev/mezzio-swoole/v3/considerations/
- https://www.swoole.co.uk/docs/modules/swoole-server/conﬁguration
- https://github.com/swooletw/awesome-swoole
- https://github.com/k911/swoole-bundle
We looking: DevOps / Backend Developer / Frontend Developer / QA
manual+automation.
Any Questions?
BTW:

"Swoole: double troubles in c", Alexandr Vronskiy

Related slideshows

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

Recommended for you

More Related Content

What's hot

What's hot (20)

Similar to "Swoole: double troubles in c", Alexandr Vronskiy

Similar to "Swoole: double troubles in c", Alexandr Vronskiy (20)

More from Fwdays

More from Fwdays (20)

Recently uploaded

Recently uploaded (20)

"Swoole: double troubles in c", Alexandr Vronskiy