Publishing Mercure Updates in PHP

So I'm getting the following error "Key provided is shorter than 256 bits, only 80 bits provided" anyone else had this error?

Hey Kez,

Not something I noticed myself, but maybe our users might face it and give some hints. Could you share a bit more information about what exactly you're doing when seeing this error? Right now it's difficult to say.

Cheers!

Hello ! And imagine that we only want to send an update to one or some users?

Imagine a blog. An admin arbitrarily edits the content of a post, and in this case we only want to notify the original author of the post, how should we go about dynamically targeting users who will receive the updates?

Hey Fabrice!

Sorry for the very slow reply - summer vacations :). I'm not an expert on this, but I believe this needs to be done by creating very targeted topics. For example, you could create a topic called user_{id} where {id} is the dynamic id of a user. Then, to notify that user of something, you publish to that specific topic. You would make that a private topic so that only that specific user can listen to it.

If you have 1000 concurrent users on your site, then it means you'll have 1000 topics being listened/connected to. That, by itself, I don't think is a problem - if you look at the Mercure managed hosting as a guide - https://mercure.rocks/pricing - this would mean that you have 1000 concurrent connections. But that is no different than if you had just ONE topic and all 1000 users are connected to it. So what I mean to say is: I don't think having many topics is a problem.

But there are two things that might become a problem... and they are related:

Suppose every user is listening to a user_{id} topic. Now suppose that someone comments on a post, and we want to notify everyone who has also commented on that post. How do we do that? We have 2 options, and both have pros/cons and highlight the challenge:

A) We could loop over all users who have commented on the post (e.g. 25) and send an update to the user_{id} topic of each one. But, that means 25 updates... which is a lot of work to do (you probably will need to do it via Messenger to avoid sending 25 POST requests from your app while the user is waiting for their comment to be posted) and Mercure (depending on your install) could get overwhelmed/limited if you send TOO many requests too quickly (the pricing page talks about 30 POST request per minute for the "Hobby" plan, for example).

B) OR, you could have everyone who has commented (e.g. 25 users) ALSO listening to a post_{post_id} topic. The problem is that if a user has commented on 5 posts, then they are now listening to 6 topics (user_{id} + 5x post_{post_id}). That might be a lot for one user to be listening too - I'm not sure what performance problems that user might have. But perhaps more importantly, if 1000 concurrent users on your site have each commented on 5 posts, then you now have 6000 concurrent connections to Mercure. Yikes!

So choosing your topics is an art and depends on your project. There may be some tricks that i'm not aware of - but this is my current impression.

Cheers!

Thank you for your detailed answer! In fact, I would have to be very careful.

I saw that you were planning to do some training on Mercury in the future, this may be an opportunity for you to talk about it :)

I have a question on a possible use case for streams.
I have a a backend dashboard for an inventory management system with over 100,000 and entity relations in a database.
I have a page on the backend where a user can upload an excel file to update the database.
1. Upload an xlsx file (this has over 70,000 rows)
1.a. Xlsx file is saved to the server
1.b Xlsx file is read and converted to csv file which is saved to server
2. Sweet Alert modal pops up when file upload completes
3. User clicks confirm to begin the database update
4. The sweet alert modal makes an ajax call to a symfony controller that handles updating the database from the uploaded file
5. While waiting for the controller response, a loading spinner displays in the modal.
6. About 8 minutes later, the controller returns a response and the sweet alert displays the success message.

The spinner is cool for maybe a 10 second request, but for long processes I need a loading bar.
Is there a way to use streams to render a progress bar in the modal?

Hey Nick F.!

That's a very interesting situation :). I think there are two possible ways to handle it. But in both cases, if you're going to have a progress bar, you'll first need to make sure you have a way to check/track the progress on the server. For example, if you upload an xlsx file with 70,000 rows, then (somewhere) you need to keep track of how many rows have been processed so that you can get a "percentage complete". That's... probably a good thing to have anyways, in case the process fails so that you can know how many rows were processed and which rows were unprocessed.

Anyways, assuming you can calculate the percentage complete, then there are two ways to show it to the user:

A) Polling: this is simple, but effective. You setup a Stimulus controller that makes an Ajax call every 10 seconds (or whatever you want). That would return the "% complete", either as JSON or as pre-rendered "loading bar" HTML for that percentage.

B) Streams. First, let's assume that you assign each "upload" a job number - e.g. 123. You could then listen to, for example, an "import_123" Mercure channel. Then, in the code that processes the rows, you could dispatch an Update to Mercure on this channel that could "replace" the existing progress bar element with an updated one set at the new percentage. For 70,000 rows, maybe you only dispatch this message every 100 rows to avoid sending a ton of unnecessary updates.

Really, for this situation, both are fine: polling is easier, and actually fits better in this situation than other situations (because you don't care that the user is notified the EXACT instant that you go from 10% to 11%, updating the progress bar every 10 seconds works perfectly fine).

Somewhat related, you mentioned:

> 4. The sweet alert modal makes an ajax call to a symfony controller that handles updating the database from the uploaded file
> 5. While waiting for the controller response, a loading spinner displays in the modal.
> 6. About 8 minutes later, the controller returns a response and the sweet alert displays the success message.

It sounds like that Ajax call will actually be processing for all 8 minutes. That's possible in PHP, but a little odd - and it ties up one of your php-fpm processes for all 8 minutes (not a huge deal maybe, but something to keep in mind). If you're able, this is a process that would be served really well with Messenger. You could still make an Ajax call to start the process, but that would just dispatch a message to Messenger... which would then be handled async by 1 or more worker processes. The progress bar updating would work exactly the same.

Cheers!

Failed to send update also shows up on windows