SymfonyCasts stands united with the people of Ukraine
gstreamer0.10-ffmpeg
gstreamer0.10-plugins-good packages.
Yo Friends! It's Symfony Messenger time!!! So, what is Symfony Messenger? It's a tool that allows you to... um... send messages... Wait... that made no sense.
Let's try again. Messenger is a tool that enables a really cool design pattern where you write "messages" and then other code that does something when that message is sent. If you've heard of CQRS - Command Query Responsibility Segregation - Messenger is a tool that enables that design pattern.
That's all great... and we're going to learn plenty about it. But there's a good chance you're watching this because you want to learn about something else that Messenger does: it allows you to run code asynchronously with queues & workers! OooooOOoo. That's the real fanciness of Messenger.
Oh, and I have two more sales pitches. First, Symfony 4.3 has a ton of new features that really make Messenger shine. And second, using Messenger is an absolute delight. So... let's do this!
If you want to become a command-bus-queue-processing-worker-middleware-envelope... and other buzzwords... Messenger master, warm up your coffee and code along with me. Download the course code from this page. When you unzip it, you'll find a start/ directory inside with the same code that you see here. Open up the README.md file for all the details about how to get the project running and a totally-unrelated, yet, lovely poem called "The Messenger".
The last setup step will be to find a terminal and use the Symfony binary to start a web-server at https://localhost:8000:
symfony serve
Ok, let's go check that out in our browser. Say hello to our newest SymfonyCasts creation: Ponka-fy Me. If you didn't already know, Ponka, by day, is one of the lead developers here at SymfonyCasts. By night... she is Victor's cat. Actually... due to her frequent nap schedule... she doesn't really do any coding... now that I think about it.
Anyways, we've been noticing a problem where we go on vacation, but Ponka can't come... so when we return, none of our photos have Ponka in them! Ponka-fy Me solves that: let's select a vacation photo... it uploads... and... yea! Check it out! Ponka seamlessly joined us in our vacation photo!
Behind the scenes, this app uses a Vue.js frontend... which isn't important for what we'll be learning. What is important to know is that this uploads to an API endpoint which stores the photo and then combines two images together. That's a pretty heavy thing to do on a web request... which is why, if you watch closely, it's kinda slow: it will finish uploading... wait... and, yep, then load the new image on the right.
Let's look at the API endpoint so you can get an idea of how this works: it lives at src/Controller/ImagePostController.php. Look for create() this is the upload API endpoint: it grabs the file, validates it, uses another service to store that file - that's the uploadImage() method, creates a new ImagePost entity, saves it to the database with Doctrine and then, down here, we have some code to add Ponka to our photo. That ponkafy() method does the really heavy-lifting: it takes the two images, splices them together and... to make it extra dramatic and slow-looking for the purposes of this tutorial, it takes a 2 second break for tea.
Mostly... all of this code is meant to be pretty boring. Sure, I've organized things into a few services... that's nice - but it's all very traditional. It's a perfect test case for Messenger!
So... let's get it installed! Find your terminal, open a new tab and run:
composer require messenger
When that finishes... we get a "message"... from Messenger! Well, from its recipe. This is great - but we'll talk about all this stuff along the way.
In addition to installing the Messenger component, its Flex recipe made two changes to our app. First, it modified .env. Let's see... it added this "transport" config. This relates to queuing messages - a lot more on that later.
| ... lines 1 - 29 | |
| ###> symfony/messenger ### | |
| # Choose one of the transports below | |
| # MESSENGER_TRANSPORT_DSN=amqp://guest:guest@localhost:5672/%2f/messages | |
| # MESSENGER_TRANSPORT_DSN=doctrine://default | |
| # MESSENGER_TRANSPORT_DSN=redis://localhost:6379/messages | |
| ### |
It also added a new messenger.yaml file, which... if you open that up... is perfectly... boring! It has transports and routing keys - again, things that relate to queuing - but it's all empty and doesn't do anything yet.
| framework: | |
| messenger: | |
| # Uncomment this (and the failed transport below) to send failed messages to this transport for later handling. | |
| # failure_transport: failed | |
| transports: | |
| # https://symfony.com/doc/current/messenger.html#transports | |
| # async: '%env(MESSENGER_TRANSPORT_DSN)%' | |
| # failed: 'doctrine://default?queue_name=failed' | |
| # sync: 'sync://' | |
| routing: | |
| # Route your messages to the transports | |
| # 'App\Message\YourMessage': async |
So... what did installing the Messenger component give us... other than some new PHP classes inside the vendor/ directory? It gave us one new important service. Back at your terminal run:
php bin/console debug:autowiring mess
There it is! We have a new service that we can use with this MessageBusInterface type-hint. Um... what does it do? I don't know! But let's find out next! Along with learning about message classes and message handlers.
Hey Cristian!
We're really sorry to hear you have some troubles with running the course project! And that's why we're here in the comments to help you :)
First of all, I have to mention that we recently rolled out a new internal feature that helps to "test" our course project code and make sure there are no issues that blocking it from running successfully. And that feature is already enabled for this course, and this course passes - I restarted the test build again today to make sure in it. So, this sounds like you did something wrong during the setting it up, but don't worry, let's figure the problem out together, I'd be happy to help you with it.
First of all, could you let me know that if you downloaded the course code and started from the start/ directory? Did you successfully follow all the instructions from the start/README.md file there?
I just downloaded the course code myself, went to the start/ dir, run successfully next commands:
$ composer install
$ yarn install
$ yarn encore dev
So, the latest "yarn encore dev" command was successful for me too, no errors you mentioned. I wonder if you do any extra upgrades of the Yarn dependencies? If so - please, could you re-download the course code and run the exact commands in README.md file again? Do you still see that error? And if so, I'd like to know what Node JS and Yarn versions do you have exactly, you can check them with:
$ node --version // I have v17.0.1 installed
$ yarn --version // I have 1.22.17 installed
I bet the problem might be in the newer version of those packages you're using.
Cheers!
Hey there
In this course we do not cover Mercure but we do cover RabbitMQ in case you are interested. If you want to learn more about Mercure you can check the docs https://symfony.com/doc/cur...
Cheers!
Thanks, i don't really undertstand what is RabitMQ and the Messenger component for now. And my Frenglish don't help me, fortunatly i have subtitles. I saw a conference of Kevin Dunglas in french, about Mercure and it seems really great.
Regards.
Awesome! I love your courses and good job for your english. You are the first foreign teacher i follow and it's a pleasure.
To get rid of yarn errors, I had to use node v16.13.0 (via nvm) and yarn 1.22.17
node 17.0.1 as suggested by Mollokhan didn't work. I had an openSSL error :
Or try to add NODE_OPTIONS=--openssl-legacy-provider before your commands.
Last, I was a bit surprised by SF version 4.3, since this tutorial is in Symfony 5 section. This is worth an upgrade ;-)
With <3 for Ryan & all the sfCasts team anyway ! Cheers
Hey Kojo, thanks for your kind words, and my apologies for the inconvenience of setting the project up. As you noticed, this tutorial runs on Symfony 4, but all the concepts are still relevant, and the Symfony Messenger component has not changed in a significant way. Anyways, we appreciate your feedback, thank you!
Hi,
How to get retry count in Symfony message handler.
Messenger consumer command throws an error and retries it 5 times according to configuration. So I want to get the retry count in the handler so I can add some logic accordingly.
Hey Balaram M.!
Try this:
RedeliveryStamp::getRetryCountFromEnvelope($message);
Where $message is the Envelope object. Ok, two things to explain here:
1) Each time your message is retried, a new RedeliveryStamp is added to the envelope. The above code is a shortcut to get the count.
2) In your message handler you won't actually have access to the Envelope: you are only passed the message object that's inside of the envelope. What I would do is:
A) Register a listener/subscriber on the WorkerMessageReceivedEvent event. If the message matches your message object, then you can use the above code to get the retry count.
B) Now that you have the retry count you can either (i) apply your logic from inside the handler of (ii) modify the underlying message object (e.g. create and call some ->setRetryCount() method) to hold the retry count. Then, later, when your handler is called, it'll have access to this.
Let me know if that helps :).
Cheers!
Sure - probably something like this:
namespace App\EventSubscriber;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\Messenger\Event\WorkerMessageReceivedEvent;
use Symfony\Component\Messenger\Stamp\RedeliveryStamp;
class MessageRetryCounterSubscriber implements EventSubscriberInterface
{
public static function getSubscribedEvents()
{
return [
WorkerMessageReceivedEvent::class => 'onMessageReceived',
];
}
public function onMessageReceived(WorkerMessageReceivedEvent $event)
{
$envelope = $event->getEnvelope();
$message = $envelope->getMessage();
if (!$envelope instanceof YourCustomMessageClass) {
return;
}
$retryCount = RedeliveryStamp::getRetryCountFromEnvelope($message);
// setRetryCount() would be a custom method on your custom class
$message->setRetryCount($retryCount);
}
}
Then, in the YourCustomMessageClass, you would have a setRetryCount() method that sets a retryCount property. This event subscriber will be called before your handler is called. So by the time your handler is called, you can access the retryCount property on the message from inside the handler to do your custom logic.
Cheers!
Hi weaverryan
Thank you so much for replying!
I have tried the solution which you posted. But when I run worker after the message is pushed to queue <b>WorkerMessageReceivedEvent</b> is not triggering.
<b> MessageRetryCounterSubscriber.php </b>
namespace SharedBundle\EventSubscriber;
use SharedBundle\Message\TestMessage;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\Messenger\Event\WorkerMessageReceivedEvent;
use Symfony\Component\Messenger\Stamp\RedeliveryStamp;
class MessageRetryCounterSubscriber implements EventSubscriberInterface
{
/**
* @return string[]
*/
public static function getSubscribedEvents(): array
{
return [
WorkerMessageReceivedEvent::class => 'onMessageReceived',
];
}
public function onMessageReceived(WorkerMessageReceivedEvent $event)
{
$this->logger->info('method called');
$envelope = $event->getEnvelope();
$message = $envelope->getMessage();
if (!$envelope instanceof TestMessage) {
return;
}
$retryCount = RedeliveryStamp::getRetryCountFromEnvelope($envelope);
$message->setRetryCount($retryCount);
}
}
<b> TestMessage.php </b>
namespace SharedBundle\Message;
class TestMessage
{
private int $retryCount;
public function getRetryCount()
{
return $this->retryCount;
}
public function setRetryCount(int $retryCount) : TestMessage
{
$this->retryCount = $retryCount;
return $this;
}
}
<b> TestMessageHandler.php </b>
namespace SharedBundle\MessageHandler;
use Symfony\Component\Messenger\Handler\MessageHandlerInterface;
Use SharedBundle\Message\TestMessage;
class TestMessageHandler implements MessageHandlerInterface
{
public function __invoke(TestMessage $testMessage)
{
print_r($testMessage->getRetryCount());
}
}
<b> services.yaml </b>
worker.event.message_received:
class: SharedBundle\EventSubscriber\MessageRetryCounterSubscriber
tags:
- { name: kernel.event_subscriber}
Do you have any idea what I did wrong?
Really appreciate your help! Thank you!
Hi Balaram M.!
Hmmm. I was assuming you were using service auto-registration and autoconfiguration. However, even if you're not, you properly created the service and tagged it with kernel.event_subscriber... as far as I can tell. If the MessageRetryCounterSubscriber is not being called, there must be something wrong with the service or tag. Try running bin/console debug:event. Do you see "Symfony\Component\Messenger\Event\WorkerMessageReceivedEvent" event as one of the events? And if so, is your subscriber listed below it?
If it is, then your subscriber is being called. If it is not, then there is something wrong with the service configuration.
Cheers!
The best solution to get rid of these enormous number of errors is create new Symfony project and follow instructions:
-- Set your db in .env file.
-- Copy Entity, Controller and Repository from start directory of this Tutorial.
-- Run following commands:
`composer require symfony/orm-pack
composer require --dev symfony/maker-bundle
composer require annotations
composer require twig
composer require doctrine/annotations
composer require sensio/framework-extra-bundle
composer update
php bin/console doctrine:database:create
symfony console make:migration
symfony console doctrine:migrations:migrate
composer require symfony/webpack-encore-bundle
yarn install
yarn encore dev-server` (Don't worry if get any errors!.. Relax and run)
`yarn add bootstrap --dev
yarn add jquery @popperjs/core –dev
yarn add vue@^2.5 vue-loader@^15.9.5 vue-template-compiler –dev<br />After adding vue, run -<br />yarn encore dev`
If you are getting 4 errors from “./node_modules/babel-loader/....” for <b>axios, moment, vue2-dropzone, vue2-dropzone/dist/vue2Dropzone.min.css</b> then:
`yarn add sass-loader@^12.0.0 sass --dev
composer require symfony/validator doctrine/annotations
npm install --save axios moment vue2-dropzone`
Restart your Node Server and Symfony Server for the last time., it will work.
Thanks me later! :D
Hi, great course!
I've got a question about a problem that I am currently facing at work. Imagine a situation when you use Messenger to communicate between microservices written in Symfony.
Everything works great until you have to add more fields to the message and there you'll get a problem with the compatibility.
What's the best way for message versioning in Symfony? Add more messages with version names, create contracts for messages in a shared repository, or any other solution? Do you have any clue?
Best!
Hey Bartlomeij!
That's an excellent question! In a sense, when you are communicating with an "external" system, the messages you send are, sort of n API or contract. If you suddenly change that message, you are breaking the contract. So if this were a situation where your app were sending messages and truly external (i.e. not owned by you) microservices were consuming them, then it would be especially tricky... like you'd need to have versions of the message and maybe have the user "opt into" which one they get.
But since you "own" all of the microservices in question, I'd try to keep it as simple as possible. The two solutions I like best are:
A) Use a mono repo. It might be "too late" if you don't have this already. On Symfonycasts, we have a frontend, which communicates to 3 microservices. But all of the code lives in a monorepository. And then all the code is deployed at the same time. This makes it easy to tweak something on SymfonyCasts and tweak that same thing inside of a microservice at the same time... and deploy those changes together.
B) Private, shared repository. Even though we have a mono repo, we also have a small, private share repository where, among other things, we sometimes share "model classes" that are used by multiple micro-services and need to stay in sync (we could keep them in sync via the mono-repo, but we would still need to duplicate the classes in each project... and so the shared repo avoids that).
So that's just my preference on how to solve this: the simpler the better, since you have control over all of the "parts".
Cheers!
Hi, great course! I am wondering, is Messenger good for tasks such as large CSV file import? I mean, if we have a CSV file with 10000-50000 rows, would it be good decision to use Messenger instead of a Command (currently I use cron to run long lasting commands)? If yes, could you share some tips?
Hey there!
Symfony Messenger is exactly for that kind of processes because they happen out of the request of your users, it comes with a built-in retry functionality, and more. And, another advantage is that your PHP worker for consuming the messages can live in a different server/machine, so your website machine won't consume all of its resources handling those processes
Cheers!
Hi all !
I wanted to follow this tutorial, and I have issue during the source code installation. I must say I am NOT AT ALL at ease with modern Javascript "stuffs", I am purely a Back-End developer, so, I could say stupid things...
I follow the tutorial in the README, and, at the step "yarn encore dev", I have the following message :
`yarn encore dev
yarn run v1.22.10
$ C:\Projects\Divers\knp\messenger\pending\node_modules.bin\encore dev
Running webpack ...
ERROR Failed to compile with 3 errors 08:31:34
error in ./assets/js/components/ImageApp.vue?vue&type=style&index=0&id=3ab43ae7&scoped=true&lang=scss&
Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js):
ModuleBuildError: Module build failed (from ./node_modules/sass-loader/lib/loader.js):
Error: Cannot find module 'node-sass'
Require stack:
@ ./assets/js/components/ImageApp.vue?vue&type=style&index=0&id=3ab43ae7&scoped=true&lang=scss& 1:0-481 1:497-500 1:502-980 1:502-980
@ ./assets/js/components/ImageApp.vue
@ ./assets/js/app.js
error in ./assets/js/components/ImageList.vue?vue&type=style&index=0&id=290cb348&scoped=true&lang=scss&
Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js):
ModuleBuildError: Module build failed (from ./node_modules/sass-loader/lib/loader.js):
Error: Cannot find module 'node-sass'
Require stack:
@ ./assets/js/components/ImageList.vue?vue&type=style&index=0&id=290cb348&scoped=true&lang=scss& 1:0-482 1:498-501 1:503-982 1:503-982 @ ./assets/js/components/
ImageList.vue
@ ./node_modules/babel-loader/lib??ref--0-0!./node_modules/vue-loader/lib??vue-loader-options!./assets/js/components/ImageApp.vue?vue&type=script&lang=js&
@ ./assets/js/components/ImageApp.vue?vue&type=script&lang=js&
@ ./assets/js/components/ImageApp.vue
@ ./assets/js/app.js
error in ./assets/js/components/ImageItem.vue?vue&type=style&index=0&id=1f073b3d&scoped=true&lang=scss&
Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js):
ModuleBuildError: Module build failed (from ./node_modules/sass-loader/lib/loader.js):
Error: Cannot find module 'node-sass'
Require stack:
@ ./assets/js/components/ImageItem.vue?vue&type=style&index=0&id=1f073b3d&scoped=true&lang=scss& 1:0-482 1:498-501 1:503-982 1:503-982 @ ./assets/js/components/
ImageItem.vue
@ ./node_modules/babel-loader/lib??ref--0-0!./node_modules/vue-loader/lib??vue-loader-options!./assets/js/components/ImageList.vue?vue&type=script&lang=js&
@ ./assets/js/components/ImageList.vue?vue&type=script&lang=js&
@ ./assets/js/components/ImageList.vue
@ ./node_modules/babel-loader/lib??ref--0-0!./node_modules/vue-loader/lib??vue-loader-options!./assets/js/components/ImageApp.vue?vue&type=script&lang=js&
@ ./assets/js/components/ImageApp.vue?vue&type=script&lang=js&
@ ./assets/js/components/ImageApp.vue
@ ./assets/js/app.js
Entrypoint app [big] = runtime.js vendors~app.css vendors~app.js app.js
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.`
I installed Noed and Yarn for this tutorial? I work on Windows 10 x64 Pro, with a PHP 7.3.9, and I used the MySQL from a WAMP I had around. I had no issue during the installation prior to this.
If I continued to follow the README, I obviously have errors too, and when starting the SF Server, the page doesn't load (I was not really surprised) properly.
Thank you for your help !
Edit : I don't like when my computuer resists me, so, I search, and search, and I was like "mayyyybe, the line saying it doens't find the node-sass module has something to do ?" so, I typed, partially not knowing what I was doing "npm install sass", and I followed what it says to me.
It said I had vulnerabilities in my projet, and I had to execute other command, I did, and know, I have another erroer when I run "yarn encode dev" :
` WARNING Webpack Encore requires version ^9.0.1 || ^10.0.0 || ^11.0.0 of sass-loader, but your version (7.1.0) is too old. The related feature will probably not work correctly.
WARNING Webpack Encore requires version ^9.0.1 || ^10.0.0 || ^11.0.0 of sass-loader, but your version (7.1.0) is too old. The related feature will probably not work correctly.
WARNING Webpack Encore requires version ^15.9.5 of vue-loader, but your version (15.7.0) is too old. The related feature will probably not work correctly.
RECOMMEND To create a smaller (and CSP-compliant) build, see https://symfony.com/doc/current/frontend/encore/vuejs.html#runtime-compiler-build
[webpack-cli] Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
output options tell webpack how to write the compiled files to disk.I will probaly try to follow what is under "RECOMMEND", I can't be do more harm than not being able to follow the tutorial ? I will post my resolution of the issue in Edis here, in case another JavaScridiot like me has the same problem.
Edit 2 : So, I went into my package.json file, and upgraded my sass-loader version to the lowest required ^9.0.1, and run "npm update", then "yarn encore dev", and It worked fine !
Now, I have issues with the Symfony CLI Server, but, I'm used to these issues, and I used another Webserver, or the web-server bundle in those cases, as I from time to time can't use the Symfony CLI Server, it just freezes, and do nothing, and it looks completlà borken. I usually wait for a few weeks/months, for an update version, and it works, until the next time... it can be anoying, but I haven't found a good and permanent fix or cause for the repeating issue.
Thank you for if you read everything, and I'm looking for foloowing at last this tutorial, and thank you for it !
Byyyyeee !
Hey abyssweb
Woh... We are SO sorry that you faced so many issues during install steps, I can confirm that course has some issues, we registered it and will fix it as soon as possible!
For Symfony cli server issues try to upgrade your php 7.3 to latest minor version 7.3.27 probably it will work, however I can't say that it will totally fix it.
Thanks for learning and staying with us!!!
Cheers!
Hey sadikoff
Thank you for your answer ! I am pretty stubborn when it comes to my computer, and I don't like when it tries to resist me, so, I persisted !
Concerning my (recurrent) issues with the Symfony CLI, I found the cause too, I'm unstoppable today !
So, the Symfony CLI for its server, and in general, will create a logs directory named .symfony in your Windows User directory. And, that's where it all went down. I noted that, when I had freezes, no logs where written, nothing was logged... So, I tried deleting the directory, looking how it was created, trying some commands like the ca:install, and lauching the serveur, and killing my command line wndowd each time it froze to be sure it was killed, etc.
In my company, the Users directory may be a little bit more protected than usual, and I didn't find a way to configure the Symfony CLI to use another directory, so, I uninstalled the Symfony CLI, and re-downloaded the Installer, I lauched the Installer by forcing the Admin Mode (Right-click -> Execute as Admin), and then, the symfony.exe was marked as "Execute as Admin" in his properties. And I also launched a Cmder window as Admin.
And everything went smoothly this time...
So, I think that, when Symfony CLI propose an update (when you execute the Server Start command by example), if you're not doing it from an Admin Commande Line Window, or as an Admin, etc. the application don't have all the rights to all the directories, and you have issues with logs files that can't me accessed...
That's my 2 cents about the issue, and how I was able to continue working this morning.
Sounds like you have great investigation time and you find a way to bypass it. BTW I'd recommend you to look at https://github.com/symfony/... it's official Symfony CLI issue repository, probably there is something that can help you, or maybe you can post your issue with description and someday it will be fixed!
Cheers! Have a nice coding!
The funny thing is... I'm a Developer in a investigation company... Thank you for your support !
Just to jump in here - thanks to your report sadikoff - I have now fixed the code download (we were just ONE minor version too low on sass-loader - the project used 7.1.0 and we needed 7.2.0, which is the first version where sass-loader recognized using "sass" instead of "node-sass").
Sorry again about the trouble - we had upgraded the code to use node-sass instead of sass (which works much more nicely)... but I didn't realize that the sass-loader was too old to support that - totally my fault!
Cheers!
Thank you weaverryan
I had no other issue while installing Messenger during the tutorial, and I LOVE this tutorial, and I think Messenger will be a great addition in futurs projects ! Thank you for everybody's work on it !
Stuck at Chapter 1 after trying `composer require messenger` but getting this error: [UnexpectedValueException] Invalid version string "5.x". The error also displays when running `composer update` soon after `composer install`.
I am running PHP 7.4, Composer 1.10.13. Any ideas?
Same problem with PHP 7.3.22. Here is the full list of packages and version that have been installed with composer install
Yeah all php version are affected, solution is the same update flex plugin with command I mentioned below =)
Cheers!
oh that's complex
try this command to update flex this should fix your issue, and I guess all current issues with composer work
composer update symfony/flex --no-plugins --no-scripts
Cheers!
Greetings,
Our upcoming project heavily uses third party APIs to download JSON data and thousands of images daily. The data is then processed - information is changed and put into our database, images are optimized and sent to cloud. The "default" approach with few commands doing "foreach" up to few hours, which is definitely too long. For demo purposes, I've created scripts that download a set amount of files, but each in different way:
• 1 PHP process that downloads all files one by one in "foreach"
• Multiple PHP processes ran all at once, where each downloads only one file (amount of files = amount of processes)
• Group of multiple PHP processes ran at once, where each downloads only one file (unoptimized - next group is started when all processes from previous group have finished, rather than execute next process anytime when amount of PHP processes isn't exceed)
In the tested environment results were as follows for 200 files: 1st took 52 seconds, 2nd 11 seconds but heavy resource usage, 3rd 7 seconds and less usage.
The test results hinted which direction we need to head to speed up the process. Do You think this is good use case for Symfony Messenger component? We would probably need several dozen of workers - is it possible to run that many? Won't it cause race conditions, database locks or other issues? I've asked this question on Symfony Slack, and apparently someone had problems in similar use case. If it's not the best solution - what would you suggest?
Hi SirRFI!
The short answer is... yea! On a high level, this is the purpose of "workers": you put 200 "messages" into a queue, and then allow 1, 2, 5, 10 or 100 "workers" to take messages from that queue and process them (in your case, the workers would be downloading the files, processing them, etc). The benefit of using a queue system (and Messenger is something that gives you the ability to work with queues nicely). Here is some information:
A) queue systems (and Messenger) are built from the ground-up around the idea of "supporting many works". They have built-in protection that avoids 2 workers from ever receiving the same message. If a worker suddenly fails and exits, the message would then be re-tried using Messenger's retry logic.
B) About race conditions, database locks etc: you won't have any of these problems from Messenger. What I mean is, Messenger will avoid the race condition of handling a single message more than one time. If you are going to have so many workers, I would recommend using RabbitMQ/AMQP as your transport, instead of Doctrine. Using Doctrine with so many workers would increase the activity on your database quite a bit. About "other" race conditions (like race conditions that you might introduce in your own code), it's really no different than a web request: if you have 2 web requests that are both going to write to the same row in a table, it's possible to get a race condition. The same is true in messenger where you have multiple workers. Whether or not that is a real problem depends on the logic in your workers: if your worker/handler code is always downloading a file and inserting a new row into the database to represent that file, then probably you are ok. You would only have a problem if two workers (that are downloading two different files) might need to update the same row in a table with different data. It's possible, but probably not that common.
If you have any other questions, let me know! Btw, you can absolutely run the messenger:consume command with an option (--limit=1 iirc) that tells the worker to only handle 1 message then exit. So, i you wanted to get really fancy, you could probably re-create your "group of multiple PHP processes" setup by leveraging this command with that option. It's actually a good idea, in general, to make sure that your worker processes don't handle too many messages anyways, as exiting often can free up some memory.
Cheers!
Hey guys, thank you so mush for the tuto,
If I understood correctly the purpose of symfony messenger is to set up an asynchronous execution system with php. By cons php it already provides pthreads and generators for this kind of execution.
So why not just mix one of the two solutions with the event subscriber and then send messages and execute it asynchronously?
I know symfony messanger is easier to use, I just want to understand the difference betwwen Symfony Messenger and pthreads or generator and when using each solution.
Thank you!
Hey ahmedbhs
That's a good question and I think I just found the reason: https://github.com/krakjoe/...
Basically, Symfony Messenger works in a standard and accepted way, AMPQ. So, as I see it, it's like the right version of pthreads (and more)
Cheers!
They gonna deprecate the pthpthreads ? Or just talking about how difficult tobuse the phpthreads? Anyway still have generator it's good solution for asynchron?
It seems like they deprecate it since PHP 7.4 in favor of using https://github.com/krakjoe/...
Hey Daria!
The initial project was built from scratch for this Messenger course! Unfortunately, we don't have any course where we're building it step-by-step. The idea was so that we already have a specific project from the beginning that do some job without queues. And then we're going to refactor it using Messenger. So the Ponka-fy project was written specifically for this course.
Cheers!
Thank you for response, Victor! This application is great as base for Messenger course and as standalone project too.
I try to upload an image from the example code downloaded from the tutorial and it give me "GD Library extension not available with this PHP installation." error.
I'm using PHP 7.3 and i am sure the GD Library is enabled. : (
Hey tv78-coder,
Hm, it sounds like you don't have GD library. Please, keep in mind that you need to enable it for both CLI and PHP-FPM - those are 2 different php.ini configs. To make sure it's enabled for CLI, you can run "$ php -m" and see if the list of modules contains "gd". For PHP-FPM - you can try to check "View phpinfo()" link in the Symfony web developer toolbar, just hover over Symfony version in the right bottom corner. SO, are you really see GD for both? Also, make sure to reload your PHP-FPM service to make sure all the php.ini changes are applied.
I hope this helps!
Cheers!
Hi there,
I recently upgraded to PHP 7.4.0 and during the composer install I get the following errors during cache:clear script:
`
FileLoader.php line 166:
Notice: Trying to access array offset on value of type null in config/routes/../../src/Controller/ (which is being imported from "config/routes/annotations.yaml"). Make sure annotations are installed and enabled.
`
My Composer version is 1.9.1, I'm wondering if there is an issue about PHP/Composer/Symfony or am I just missing something ? This won't block me for the course but this error shouldn't be there. Any suggestion ?
<b>EDIT:</b> It is blocking, the application is showing the beautiful red error page with the same message as above.
<b>EDIT 2:</b> running composer update fixed the issue but the code provided in the /start/ directory is broken as the argument of the ImagePostRepository constructor must be type-hinted to Doctrine\Common\Persistence\ManagerRegistry. Then it works !
Sorry if I wasted a bit of your time, maybe this will help other people encountering the same issue.
Hey AymDev!
Yea, sorry about that issue. We've become aware that PHP 7.4.0 has an incompatibility issue with doctrine/annotations. You need to update doctrine/annotations to 1.7.0 or higher to fix it. You're right that the start code should not have an error. We will be working to update the start code for the tutorials to make sure it works with PHP 7.4.0 :).
About the "ManagerRegistry" thing, you would need to update the constructor IF you were upgrading this application to (if i I remember correctly) DoctrineBundle 2.0. That's probably one of the updates you got when running composer update. If you only updated doctrine/annotations, that won't be needed to get things running.
Cheers!
// composer.json
{
"require": {
"php": "^7.1.3",
"ext-ctype": "*",
"ext-iconv": "*",
"composer/package-versions-deprecated": "^1.11", // 1.11.99
"doctrine/annotations": "^1.0", // v1.8.0
"doctrine/doctrine-bundle": "^1.6.10", // 1.11.2
"doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // v2.0.0
"doctrine/orm": "^2.5.11", // v2.6.3
"intervention/image": "^2.4", // 2.4.2
"league/flysystem-bundle": "^1.0", // 1.1.0
"phpdocumentor/reflection-docblock": "^3.0|^4.0", // 4.3.1
"sensio/framework-extra-bundle": "^5.3", // v5.3.1
"symfony/console": "4.3.*", // v4.3.2
"symfony/dotenv": "4.3.*", // v4.3.2
"symfony/flex": "^1.9", // v1.18.7
"symfony/framework-bundle": "4.3.*", // v4.3.2
"symfony/messenger": "4.3.*", // v4.3.4
"symfony/property-access": "4.3.*", // v4.3.2
"symfony/property-info": "4.3.*", // v4.3.2
"symfony/serializer": "4.3.*", // v4.3.2
"symfony/validator": "4.3.*", // v4.3.2
"symfony/webpack-encore-bundle": "^1.5", // v1.6.2
"symfony/yaml": "4.3.*" // v4.3.2
},
"require-dev": {
"easycorp/easy-log-handler": "^1.0.7", // v1.0.7
"symfony/debug-bundle": "4.3.*", // v4.3.2
"symfony/maker-bundle": "^1.0", // v1.12.0
"symfony/monolog-bundle": "^3.0", // v3.4.0
"symfony/stopwatch": "4.3.*", // v4.3.2
"symfony/twig-bundle": "4.3.*", // v4.3.2
"symfony/var-dumper": "4.3.*", // v4.3.2
"symfony/web-profiler-bundle": "4.3.*" // v4.3.2
}
}
This is by far the worst tutorial so far. I had SO many problems installing everything I need for this!!
`yarn run v1.22.17
$ /Applications/MAMP/htdocs/queues/start/node_modules/.bin/encore dev
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db
Why you should do it regularly:
https://github.com/browserslist/browserslist#browsers-data-updating
Running webpack ...
ERROR Failed to compile with 4 errors 6:37:56 PM
These dependencies were not found:
To install them, you can run: npm install --save axios moment vue2-dropzone vue2-dropzone/dist/vue2Dropzone.min.css
Entrypoint app [big] = runtime.js vendors~app.css vendors~app.js app.css app.js
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.`
This error appears when I run
yarn encore dev