Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine

Testing with the "in-memory" Transport

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $12.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

A few minutes ago, in the dev environment only, we overrode all our transports so that all messages were handled synchronously. We commented it out for now, but this is also something that you could choose to do in your test environment, so that when you run the tests, the messages are handled within the test.

This may or may not be what you want. On one hand, it means your functional test is testing more. On the other hand, a functional test should probably test that the endpoint works and the message is sent to the transport, but testing the handler itself should be done in a test specifically for that class.

That's what we're going to do now: figure out a way to not run the handlers synchronously but test that the message was sent to the transport. Sure, if we killed the worker, we could query the messenger_messages table, but that's a bit hacky - and only works if you're using the Doctrine transport. Fortunately, there's a more interesting option.

Start by copying config/packages/dev/messenger.yaml and pasting that into config/packages/test/. This gives us messenger configuration that will only be used in the test environment. Uncomment the code, and replace sync with in-memory. Do that for both of the transports.

framework:
messenger:
transports:
async: 'in-memory://'
async_priority_high: 'in-memory://'

The in-memory transport is really cool. In fact, let's look at it! I'll hit Shift+Shift in PhpStorm and search for InMemoryTransport to find it.

This... is basically a fake transport. When a message is sent to it, it doesn't handle it or send it anywhere, it stores it in a property. If you were to use this in a real project, the messages would then disappear at the end of the request.

But, this is super useful for testing. Let's try it. A second ago, each time we ran our test, our worker actually started processing those messages... which makes sense: we really were delivering them to the transport. Now, I'll clear the screen and then run:

php bin/phpunit

It still works... but now the worker does nothing: the message isn't really being sent to the transport anymore and it's lost at the end of our tests. But! From within the test, we can now fetch that transport and ask it how many messages were sent to it!

Fetching the Transport Service

Behind the scenes, every transport is actually a service in the container. Find your open terminal and run:

php bin/console debug:container async

There they are: messenger.transport.async and messenger.transport.async_priority_high. Copy the second service id.

We want to verify that the AddPonkaToImage message is sent to the transport, and we know that it's being routed to async_priority_high.

Back in the test, this is super cool: we can fetch the exact transport object that was just used from within the test by saying: $transport = self::$container->get() and then pasting the service id: messenger.transport.async_priority_high

... lines 1 - 8
class ImagePostControllerTest extends WebTestCase
{
public function testCreate()
{
... lines 13 - 25
$transport = self::$container->get('messenger.transport.async_priority_high');
... line 27
}
}

This self::$container property holds the container that was actually used during the test request and is designed so that we can fetch anything we want out of it.

Let's see what this looks like: dd($transport).

... lines 1 - 8
class ImagePostControllerTest extends WebTestCase
{
public function testCreate()
{
... lines 13 - 25
$transport = self::$container->get('messenger.transport.async_priority_high');
dd($transport);
}
}

Now jump back over to your terminal and run:

php bin/phpunit

Nice! This dumps the InMemoryTransport object and... the sent property indeed holds our one message object! All we need to do now is add an assertion for this.

Back in the test, I'm going to help out my editor by adding some inline docs to advertise that this is an InMemoryTransport. Below add $this->assertCount() to assert that we expect one message to be returned when we say $transport->... let's see... the method that you can call on a transport to get the sent, or "queued" messages is get().

... lines 1 - 6
use Symfony\Component\Messenger\Transport\InMemoryTransport;
class ImagePostControllerTest extends WebTestCase
{
public function testCreate()
{
... lines 13 - 24
/** @var InMemoryTransport $transport */
$transport = self::$container->get('messenger.transport.async_priority_high');
$this->assertCount(1, $transport->get());
}
}

Let's try it! Run:

php bin/phpunit

Got it! We're now guaranteeing that the message was sent but we've kept our tests faster and more directed by not trying to handle them synchronously. If we were using something like RabbitMQ, we also don't need to have that running whenever we execute our tests.

Next, let's talk deployment! How do we run our workers on production... and make sure they stay running?

Leave a comment!

25
Login or Register to join the conversation
Braunstetter Avatar
Braunstetter Avatar Braunstetter | posted 4 months ago

Hello dear Symfonycasts team,

I would like to use this in Behat.
I have also already injected the transport into my context and it works. But it is always empty, no matter what.

This will probably be because the is cleared on every other http request, right?

Is there a way to use this "in memory" transport in my Behat tests? Do you know something?

Kind regards
Michael

Reply

Hey @Braunstetter!

I have also already injected the transport into my context and it works. But it is always empty, no matter what.
This will probably be because the is cleared on every other http request, right?

I think you have identified the problem correctly :). When you use the "in memory" transport, it means that the messages in the queue are basically just stored in php. So they will exist until that request is over. In Behat, iirc, in practice, most of the ways that Mink makes requests involve making REAL requests to your app. So Mink makes a real HTTP request, you have an in-memory transport ON that request, then the request finishes and that is lost. Back in your context classes, the "in memory" transport you have there is (and never way) the same object in memory as the one that was used during the request :/. OR, perhaps Mink IS making "fake" requests through Symfony directly - the browser kit driver should be doing that. In that case, what you're attempting is possible, but my guess is that the "in memory" transport that's being used during the request is a different object than the one you're holding in the context.

ANYWAYS, if you ARE using browser-kit, I would recommend trying: https://github.com/zenstruck/messenger-test - you'll have to adjust the code a bit fore Behat. But this adds a "test" transport, where messages should be accessible even if the transport used during the request is a different instance than the one you fetch in your tests (because it stores the messages on a static property). If you're truly making different HTTP requests, then you'll need to change to use a real transport (e.g. doctrine) and then process those in your context after the request finishes. So yes, this stuff can be tricky!

Let me know if this helps... or didn't make any sense ;).

Cheers!

Reply
Braunstetter Avatar
Braunstetter Avatar Braunstetter | weaverryan | posted 3 months ago | edited

Thank you Ryan.
I have now solved it as follows. when@test - I have now set the transport to sync and created an EventSubscriber that stores the message in the cache. I reset the whole thing before each scenario. Works great ;).

To be honest, I just copied it from the people at Sylius:
Sylius/Behat/Service/MessageSendCacher.php
Sylius/Behat/Context/Hook/MailerContext.php

Reply

Very cool solution! It's fund to see how you can hook in and, sort of, "steal" the messages for your own purposes ;). I love it!

1 Reply
Covi A. Avatar
Covi A. Avatar Covi A. | posted 2 years ago

when i using getConteainer->get() method it's giving me undefine getContainer method.
then i try to define getContainer() method. is it correct?
if yes then my question is what should need to return from this method?
or is it not correct can you please give me any suggestion what should i do? and how can solve this problem?
thank you.

Reply

Hey Monoranjan,

It looks like you have a misprint in the method name, you say "getConteainer" in the beginning and then "getContainer" later. Please, double check your method names in the code first.

I hope this help!

Cheers!

1 Reply
Covi A. Avatar

but the problem is still

Reply

Hey Monoranjan,

Could you give me a little bit more context? In what file you're calling that getContainer()? In what method? How exactly are you calling it, via "$this->getContainer()"? Too little information to help you further. :)

Cheers!

Reply
Covi A. Avatar
Covi A. Avatar Covi A. | Victor | posted 2 years ago | edited
tests/messenger.yaml
        transports:
            # https://symfony.com/doc/current/messenger.html#transport-configuration
            # async: '%env(MESSENGER_TRANSPORT_DSN)%'
            async: 'in-memory://'  

mytestclass


        public function testMyclass()
        {
            /** @var InMemoryTransport $transport */
            $transport = $this->getContainer()->get('messenger.transport.async');
            dd($transport);
        }

and result is

<blockquote>1) App\Tests\Controller\MessengerQueueTest::testMyclass
Error: Call to undefined method App\Tests\Controller\MessengerQueueTest::getContainer()
</blockquote>

Reply

Hey Monoranjan,

Thanks for providing more info! Ah, ok, what if you change that line to:


$transport = static::getContainer()->get('messenger.transport.async');

or maybe


$transport = self::$container->get('messenger.transport.async');

Does it work this way? Any new error message? :)

If not, please, tell me if you extends any class in that test class?

Cheers!

Reply
Covi A. Avatar
Covi A. Avatar Covi A. | Victor | posted 2 years ago | edited

Ah, sorry it's my fault. i was extending here TestCase But it should be needed WebTestCase.
Thanks for your help. it's working perfectly now with this syntax:


$transport = $this->getContainer()->get('messenger.transport.async');
Reply

Hey Monoranjan,

No problem! I happy it works for you now :) And thanks for sharing the final solution with others, it still might be helpful for others

Cheers!

Reply
Covi A. Avatar

sorry, i am very sorry for this fault.
but in the real code i don't do mistake like this. in the real code i write getContainer() every time.

Reply

Ah, ok, that was the first wild guess :)

Reply
Abdallah Avatar
Abdallah Avatar Abdallah | posted 2 years ago

is there solution to simulate retry with memory transport... ?

Reply

Hey Abdallah

I'm not sure but I don't think so because the Memory Transport it's kind of a dummy implementation of a transport. It's meant to be used on the dev or test environment

Cheers!

Reply
csc Avatar
csc Avatar csc | posted 2 years ago | edited

How can I run process queue (in test), manually ?

<br />/** @var InMemoryTransport $transport */<br />$transport = self::$container->get('messenger.transport.async_priority_high');<br />

I need check statuses before execute that queue

Thanks

Reply
Victor Avatar Victor | SFCASTS | csc | posted 2 years ago | edited

Hey Sebastian,

Good question! I believe you can achieve it with this simple code snippet:


        $number = 1; // How many messages to process

        $transport = $this->getContainer()->get('messenger.transport.async');
        $bus = $this->getContainer()->get(MessageBusInterface::class);
        $eventDispatcher = new EventDispatcher();
        $eventDispatcher->addSubscriber(new StopWorkerOnMessageLimitListener($number));

        $worker = new Worker([$transport], $bus, $eventDispatcher);
        $worker->run([
            'sleep' => 0,
        ]);

Just change your way to get Symfony services from the container, I see you do it via self::$container->get() instead. And don't forget to use your transport service name in that get().

I hope this helps!

Cheers!

2 Reply
Ed C. Avatar

Do you have an example of how to separately test the handler? I've got a test running as above for the message, but I also want to test that the handler is doing what I expect.

Reply

Hey Ed C.

A MessageHandler can be seen as any other service. You can test it by creating its related test class and just calling the invoke method like this:


$handler = new SomeMessageHandler($arg1, $arg2);
$r = $handler(); // Yes, handlers can return something if you need to.

// assert something

I hope it helps. Cheers!

Reply
Ed C. Avatar

Hi Diego,

That was it perfectly thanks!

Reply
Default user avatar

I have a problem using in-memory transport in my test environment. I use Behat scenarios with friends-of-behat/symfony-extension to inject symfony services in Context classes.
The problem is this: in the Context class I get a different instance of the InMemory transport. I know that the instance is different because I tried to inject the transport in both the Controller dispatching the message and in the Behat Context class. I dumped the two objects and are different.
Someone has an hint on why this is happening?
Thank you

Reply
Default user avatar
Default user avatar Marc | Matlar | posted 2 years ago | edited

Hi @Matlar and Matlar

I'm using friends-of-behat/Symfony-extension and in-memory transport too, and I'm facing the same problem. I think that the problem is that there are two different containers (one for the app and another for Mink), and they are not sharing their services. The state of the services is different, so we can't access the in-memory enqueued messages.

Do you know how to solve that without having to renounce to use Mink?

Thanks in advance :)

Reply

Hey Marc

think that the problem is that there are two different containers (one for the app and another for Mink), and they are not sharing their services. The state of the services is different, so we can't access the in-memory enqueued messages.

I'm not 100% sure about that. If you dump each container then you can confirm if they are the same object or not. In the case they're not, I think a good solution would be to use the doctrine transport and call the getMessageCount() method on it to assert if your message was dispatched.

Cheers!

1 Reply

Hey @Matlar

Did you configure your test enviroment to use the InMemory transport? In other words. Did you create a .env.local.test file with such Messenger transport configuration?

Cheers!

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

This tutorial is built with Symfony 4.3, but will work well on Symfony 4.4 or 5.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.1.3",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "doctrine/annotations": "^1.0", // v1.8.0
        "doctrine/doctrine-bundle": "^1.6.10", // 1.11.2
        "doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // v2.0.0
        "doctrine/orm": "^2.5.11", // v2.6.3
        "intervention/image": "^2.4", // 2.4.2
        "league/flysystem-bundle": "^1.0", // 1.1.0
        "phpdocumentor/reflection-docblock": "^3.0|^4.0", // 4.3.1
        "sensio/framework-extra-bundle": "^5.3", // v5.3.1
        "symfony/console": "4.3.*", // v4.3.2
        "symfony/dotenv": "4.3.*", // v4.3.2
        "symfony/flex": "^1.9", // v1.18.7
        "symfony/framework-bundle": "4.3.*", // v4.3.2
        "symfony/messenger": "4.3.*", // v4.3.4
        "symfony/property-access": "4.3.*", // v4.3.2
        "symfony/property-info": "4.3.*", // v4.3.2
        "symfony/serializer": "4.3.*", // v4.3.2
        "symfony/validator": "4.3.*", // v4.3.2
        "symfony/webpack-encore-bundle": "^1.5", // v1.6.2
        "symfony/yaml": "4.3.*" // v4.3.2
    },
    "require-dev": {
        "easycorp/easy-log-handler": "^1.0.7", // v1.0.7
        "symfony/debug-bundle": "4.3.*", // v4.3.2
        "symfony/maker-bundle": "^1.0", // v1.12.0
        "symfony/monolog-bundle": "^3.0", // v3.4.0
        "symfony/stopwatch": "4.3.*", // v4.3.2
        "symfony/twig-bundle": "4.3.*", // v4.3.2
        "symfony/var-dumper": "4.3.*", // v4.3.2
        "symfony/web-profiler-bundle": "4.3.*" // v4.3.2
    }
}
userVoice