Query Extension: Auto-Filter a Collection

With a Subscription, click any sentence in the script to jump to that part of the video!

The CheeseListing entity has a property on it called $isPublished, which defaults to false. We haven't talked about this property much, but the idea is pretty simple: when a CheeseListing is first created, it will not be published. Then, once a user has perfected all their cheesy details, they will "publish" it and only then will it be publicly available on the site.

This means that we have some work to do! We need to automatically filter the CheeseListing collection to only show published items.

Adding a Test

Let's put this into a simple test first. Inside CheeseListingResourceTest add public function testGetCheeseListingCollection() and then go to work: $client = self::createClient() and $user = $this->createUser() with any email and password.

            
            106 lines
            
            tests/Functional/CheeseListingResourceTest.php
        
                                                    ... lines 1 - 9
                            
            class CheeseListingResourceTest extends CustomApiTestCase
        
            {
        
                                                    ... lines 12 - 75
                            
                public function testGetCheeseListingCollection()
        
                {
        
                    $client = self::createClient();
        
                    $user = $this->createUser('cheeseplese@example.com', 'foo');
        
                                                    ... lines 80 - 103
                            
                }
        
            }

We're not logging in as this user simply because the CheeseListing collection operation doesn't require authentication. Now, let's make some cheese! I'll create $cheeseListing1... with a bunch of data... then use that to create $cheeseListing2... and $cheeseListing3. These are nice, boring, delicious CheeseListing objects. To save them, grab the entity manager - $em = $this->getEntityManager() - persist all three... and call flush().

            
            106 lines
            
            tests/Functional/CheeseListingResourceTest.php
        
                                                    ... lines 1 - 75
                            
                public function testGetCheeseListingCollection()
        
                {
        
                                                    ... lines 78 - 80
                            
                    $cheeseListing1 = new CheeseListing('cheese1');
        
                    $cheeseListing1->setOwner($user);
        
                    $cheeseListing1->setPrice(1000);
        
                    $cheeseListing1->setDescription('cheese');
        
                    $cheeseListing2 = new CheeseListing('cheese2');
        
                    $cheeseListing2->setOwner($user);
        
                    $cheeseListing2->setPrice(1000);
        
                    $cheeseListing2->setDescription('cheese');
        
                    $cheeseListing3 = new CheeseListing('cheese3');
        
                    $cheeseListing3->setOwner($user);
        
                    $cheeseListing3->setPrice(1000);
        
                    $cheeseListing3->setDescription('cheese');
        
                    $em = $this->getEntityManager();
        
                    $em->persist($cheeseListing1);
        
                    $em->persist($cheeseListing2);
        
                    $em->persist($cheeseListing3);
        
                    $em->flush();
        
                                                    ... lines 101 - 103
                            
                }
        
                                                    ... lines 105 - 106

The stage for our test is set. Because the default value for the isPublished property is false... all of these new listings will be unpublished. Fetch these by using $client->request() to make a GET request to /api/cheeses.

            
            106 lines
            
            tests/Functional/CheeseListingResourceTest.php
        
                                                    ... lines 1 - 75
                            
                public function testGetCheeseListingCollection()
        
                {
        
                                                    ... lines 78 - 101
                            
                    $client->request('GET', '/api/cheeses');
        
                                                    ... line 103
                            
                }
        
                                                    ... lines 105 - 106

Because we haven't added any logic to hide unpublished listings yet, at this moment, we would expect this to return 3 results. Move over to the docs... and try the operation. Ah, cool! Hydra adds a very useful field: hydra:totalItems. Let's use that! In the test, $this->assertJsonContains() that there will be a hydra:totalItems field set to 3.

            
            106 lines
            
            tests/Functional/CheeseListingResourceTest.php
        
                                                    ... lines 1 - 75
                            
                public function testGetCheeseListingCollection()
        
                {
        
                                                    ... lines 78 - 102
                            
                    $this->assertJsonContains(['hydra:totalItems' => 3]);
        
                }
        
                                                    ... lines 105 - 106

Copy the method name and flip over to your terminal: this test should pass because we have not added any filtering yet. Try it:

php bin/phpunit --filter=testGetCheeseListingCollection

And... green! I love when there are no surprises.

Now let's update the test for the behavior that we want. Allow $cheeseListing1 to stay not published, but publish the other two: $cheeseListing2->setIsPublished(true)... and then paste that below and rename it for $cheeseListing3.

Once we're done with the feature, we will only want the second two to be returned. Change the assertion to 2. Now we have a failing test.

            
            108 lines
            
            tests/Functional/CheeseListingResourceTest.php
        
                                                    ... lines 1 - 75
                            
                public function testGetCheeseListingCollection()
        
                {
        
                                                    ... lines 78 - 89
                            
                    $cheeseListing2->setIsPublished(true);
        
                                                    ... lines 91 - 95
                            
                    $cheeseListing3->setIsPublished(true);
        
                                                    ... lines 97 - 104
                            
                    $this->assertJsonContains(['hydra:totalItems' => 2]);
        
                }
        
                                                    ... lines 107 - 108

Automatic Filtering

So... how can we make this happen? How can we hide unpublished cheese listings? Well... in the first tutorial we talked about filters. For example, we added a boolean filter that allows us to add a ?isPublished=1 or ?isPublished=0 query parameter to the /api/cheeses operation. So... actually, an API client can already filter out unpublished cheese listings!

That's great! The problem is that we no longer want an API client to be able to control this: we need this filter to be automatic. An API client should never see unpublished listings. Filters are a good choice for optional stuff, but not this.

Another option is called a Doctrine filter. It's got a similar name, but is a totally different feature that comes from Doctrine - we talk about it in our Doctrine Queries tutorial. A Doctrine filter allows you to modify a query on a global level... like each time we query for a CheeseListing, we could automatically add WHERE is_published=1 to that query.

The downside to a Doctrine filter.... is also its strength: it's automatic - it modifies every query you make... which can be surprising if you ever need to purposely query for all cheese listings. Sure, you can work around that - but I tend to use Doctrine filters rarely.

The solution I prefer is specific to API Platform... which I love because it means that the filtering will only affect API operations... the rest of my app will continue to behave normally. It's called a "Doctrine extension".... which is basically a "hook" for you to change how a query is built for a specific resource or even for a specific operation of a resource. They're... basically... awesome.

Creating the Query Collection Extension

Let's get our extension going: in the src/ApiPlatform/ directory, create a new class called CheeseListingIsPublishedExtension. Make this implement QueryCollectionExtensionInterface and then go to the Code -> Generate menu - or Command + N on a Mac - and select "Implement Methods" to generate the one method that's required by this interface: applyToCollection().

            
            15 lines
            
            src/ApiPlatform/CheeseListingIsPublishedExtension.php
        
                                                    ... lines 1 - 4
                            
            use ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\QueryCollectionExtensionInterface;
        
            use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
        
            use Doctrine\ORM\QueryBuilder;
        
                                                    ... line 8
                            
            class CheeseListingIsPublishedExtension implements QueryCollectionExtensionInterface
        
            {
        
                public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
        
                {
        
                }
        
            }

Very simply: as soon as we create a class that implements this interface, every single time that API Platform makes a Doctrine query for a collection of results, like a collection of users or a collection of cheese listings, it will call this method and pass us a pre-built QueryBuilder. Then... we can mess with that query however we want!

Tip

If you're loading data from something other than Doctrine (or have a custom "data provider"... a topic we haven't talked about yet), then this class won't have any effect.

In fact, this is how pagination and filters work internally: both are implemented as Doctrine extensions.

Query Collection Extension Logic

Let's see... even though this method will be called whenever API Platform is querying for a collection of any type of object, we only want to do our logic for cheese listings. No problem: if $resourceClass !== CheeseListing::class return and do nothing.

            
            22 lines
            
            src/ApiPlatform/CheeseListingIsPublishedExtension.php
        
                                                    ... lines 1 - 11
                            
                public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
        
                {
        
                    if ($resourceClass !== CheeseListing::class) {
        
                        return;
        
                    }
        
                                                    ... lines 17 - 20
                            
                }

Now... we just need to add the WHERE isPublished=1 part to the query. To do that will require a little bit of fanciness. Start with $rootAlias = $queryBuilder->getRootAlias()[0]. I'll explain that in a minute. Then $queryBuilder->andWhere() with sprintf('%s.isPublished = :isPublished') passing $rootAlias to fill in that %s part. For the isPublished parameter, set it with ->setParameter('isPublished', true).

            
            22 lines
            
            src/ApiPlatform/CheeseListingIsPublishedExtension.php
        
                                                    ... lines 1 - 11
                            
                public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
        
                {
        
                                                    ... lines 14 - 17
                            
                    $rootAlias = $queryBuilder->getRootAliases()[0];
        
                    $queryBuilder->andWhere(sprintf('%s.isPublished = :isPublished', $rootAlias))
        
                        ->setParameter('isPublished', true);
        
                }

This mostly looks like normal query logic... the weird part being that "root alias" thing. Because someone else originally created this query, we don't know what "table alias" they're using to represent CheeseListing. Maybe c... so c.isPublished? Or cheese... so cheese.isPublished? We don't really know. The getRootAlias() method tells us that... and then we hack it into our query.

Oh, and by the way - in addition to $resourceClass, this method receives the $operationName. Normally only the get operation - like GET /api/cheese - would cause a query for a collection to be made... but if you created some custom operations and needed to tweak the query on an operation-by-operation basis, the $operationName can let you do that.

Anyways... we should be done! Run that test again:

php bin/phpunit --filter=testGetCheeseListingCollection

And... green! We are successfully hiding the unpublished cheese listings. Isn't that nice?

Next, let's add logic so that admin users will still see unpublished cheese listings. And what about the item operation? Sure, an unpublished cheese listing won't be included in the collection endpoint... but how can we prevent a user from making a GET request directly to fetch a single, unpublished CheeseListing?

Leave a comment!

Gianluca-F | posted 9 months ago

Hi all,
I have added a Query Extension to add an autofilter to my unpublised content, as in your tutorial and it works GREAT.
Thanks for your work, first of all!

Let you have an endpoint that show an entity WITH related resources; let we want to apply the same business rule ( hide unpublished content ) also on related resource ( es: a company endpoint that show related products, loaded lazy as doctrine entities )

Actually, I have create a Normalizer related to the main resource and I loop over related resource and I skip di unpublished content.

Do you know other way to hide subresources?

I think there is no other way to perform this autofilter ALSO on related resource

weaverryan | SFCASTS | Gianluca-F | posted 8 months ago

Hey Gianluca!

Apologies for the slow reply! But thank you for the kind words ❤️.

I think there is no other way to perform this autofilter ALSO on related resource

I agree. As I'm sure you're already aware, if you make a GET /api/categories/1/products, then, behind the scenes, API Platform simply calls $category->getProducts() to fetch that data. Your idea about a normalize actually isn't one I had thought of - nice job 👍. The only other solution I can think of is to create something like this:

#[ApiSubresource()]
public function getPublishedProducts(): Collection
{
   // ...
}

I actually don't know if this works :). But in theory, this might create a /api/categories/1/published_products type of subresource. IF that works, that would be another solution. In general, subresources are pretty limited in API Platform 2. I've heard that got a bit more robust in API Platform 3, but I haven't looked yet (that's my homework over the next few weeks!)

Cheers!

David B. | posted 1 year ago | edited

Is it possible to completely rewrite the query in applyToItem rather than modify it? I have tried the following

    public function applyToItem(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, array $identifiers, string $operationName = null, array $context = []): void
    {
        //If its a MenuAudience pivot and pull a MealAudienceCategory instead. 
        if(
            MenuAudience::class === $resourceClass 
        ) {
                        $queryBuilder->resetDQLParts(null)->select('mac')->from(MealAudienceCategory::class, 'mac');

        }
    }```


and it seems to still have an association with the old MenuAudience table and throws the following error:

[Semantical Error] line 0, col 116 near 'mealAudience': Error: Class App\Entity\MealAudienceCategory has no association named mealAudienceCategories`

If i run dd($queryBuilder->getQuery()); It shows the correct SQL of SELECT mac FROM App\Entity\MealAudienceCategory mac

weaverryan | SFCASTS | David B. | posted 1 year ago | edited

Hey David B. !

Hmm, interesting question! So, the system doesn't like this - you're bending it in a way it doesn't like. But, probably you can get this to work.

The query extension stuff is called from your "data provider". For example, the ItemDataProvider from Doctrine in this case: https://github.com/api-plat...

I'd recommend replacing the data provider entirely for this situation. We talk about that around this section - https://symfonycasts.com/sc... - including leveraging the logic from the normal data provider. For example, in this ONE situation, you might completely "take over" and do the query and return the result. In every other case, you would call the normal data provider.

This should work, though you will (in this one case where you "take over") lose things like pagination and filtering, because the query logic from those is added via other "query extensions". However, since you're doing an "item provider", which doesn't have pagination or filtering... you probably won't be losing much or anything. But, I've never tried this, so we'll have to see!

Cheers!

ties8 | posted 2 years ago

If i have a more complicated logic which decides if a user can se a CheeseListing, which i put in a Voter, which i call by using isGranted("COMPLICATED_LOGIC", $cheeseListing), is there a way i can implement the Voter to decide on each individual item in the collection? Or do i have to replicate the stuff my voter does inside the querybuilder?

weaverryan | SFCASTS | ties8 | posted 2 years ago | edited

Hey Alex T.!

Excellent question! I remember (WAY before API Platform, just in programming in general) this question driving me crazy :p.

The answer is this: (unless I've absolutely missed some clever way of doing it f or the past 10 years) you need to replicate the voter logic in the query builder. The voter is used AFTER you already have an individual item to determine if access is granted while the query builder is used to get a list of items. Those jobs are *super* similar... but surprisingly tricky (impossible) to share. I don't believe this is an API Platform or Symfony problem either: it's just tricky to share :).

Cheers!

1 Reply

ties8 | weaverryan | posted 2 years ago

Thanks for the answer, i wonder, would it be possible, and "good practice" to create one Doctrine Criteria and share it between the Voter and the Extension? Or is there a other way to "centralize" the code for my access controlls? I worry that ill use the voter for most of the access controll stuff and then forget that the decision taking on whether the user sees certain items inside collections is completly detatched from that

weaverryan | SFCASTS | ties8 | posted 2 years ago | edited

Hey Alex T.!

I hear you... but I can't think of a really good way to share the code... I'm trying to think. For example, if a user only had access to posts where some isPublished field = true, then to check that in PHP, you would do if ($listing->isPublished()) but in a query you would be effectively adding WHERE listing.isPublished = 1. Those are frustratingly similar... but not identical.

If you're worried, one idea is just to keep the logic "in the same place". Like, create a CheeseListingAccessManager service and put methods in there that relate to access - like findAllForUser() that would return an array of CheeseListing and also canUserEditListing(CheeseListing $listing): bool. The code wouldn't be shared, but it would live in one spot at least.

Sorry I can't offer any magic solution - if you DO think of something clever, let me know - I'd love to hear it ;).

Cheers!

ties8 | weaverryan | posted 2 years ago

Thank you for your help. If my app was a Cheese App, i'd have Users which can Join a "CheeseLoversGroup". Inside that group there are group specific roles, one user might be able to create cheeseListings in GROUP_A but only read them in GROUP_B. In the Future there might be the possibility to share data between some but not all groups. My App seemd so easy to me but i have the feeling i need everything API platform has to offer 😂 Also the fear to now, unexperienced with symfony, write some code i will suffer from later is unpleasant. But "complaining does not help" and thanks to your and your teams awesome tutorials im confident that i'll succeed 👍😁

This tutorial works great for Symfony 5 and API Platform 2.5/2.6.

Symfony 4 API Platform 2.4

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.1.3, <8.0",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "api-platform/core": "^2.1", // v2.4.5
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "doctrine/annotations": "^1.0", // 1.13.2
        "doctrine/doctrine-bundle": "^1.6", // 1.11.2
        "doctrine/doctrine-migrations-bundle": "^2.0", // v2.0.0
        "doctrine/orm": "^2.4.5", // v2.7.2
        "nelmio/cors-bundle": "^1.5", // 1.5.6
        "nesbot/carbon": "^2.17", // 2.21.3
        "phpdocumentor/reflection-docblock": "^3.0 || ^4.0", // 4.3.1
        "symfony/asset": "4.3.*", // v4.3.2
        "symfony/console": "4.3.*", // v4.3.2
        "symfony/dotenv": "4.3.*", // v4.3.2
        "symfony/expression-language": "4.3.*", // v4.3.2
        "symfony/flex": "^1.1", // v1.18.7
        "symfony/framework-bundle": "4.3.*", // v4.3.2
        "symfony/http-client": "4.3.*", // v4.3.3
        "symfony/monolog-bundle": "^3.4", // v3.4.0
        "symfony/security-bundle": "4.3.*", // v4.3.2
        "symfony/twig-bundle": "4.3.*", // v4.3.2
        "symfony/validator": "4.3.*", // v4.3.2
        "symfony/webpack-encore-bundle": "^1.6", // v1.6.2
        "symfony/yaml": "4.3.*" // v4.3.2
    },
    "require-dev": {
        "hautelook/alice-bundle": "^2.5", // 2.7.3
        "symfony/browser-kit": "4.3.*", // v4.3.3
        "symfony/css-selector": "4.3.*", // v4.3.3
        "symfony/maker-bundle": "^1.11", // v1.12.0
        "symfony/phpunit-bridge": "^4.3", // v4.3.3
        "symfony/stopwatch": "4.3.*", // v4.3.2
        "symfony/web-profiler-bundle": "4.3.*" // v4.3.2
    }
}

Edit on GitHub

TRACK

Symfony 5 >

COURSE

API Platform Part 2: Security

Query Extension: Auto-Filter a Collection

Keep on Learning!

Adding a Test

Automatic Filtering

Creating the Query Collection Extension

Query Collection Extension Logic

Leave a comment!

What PHP libraries does this tutorial use?

	... lines 1 - 9
	class CheeseListingResourceTest extends CustomApiTestCase
	{
	... lines 12 - 75
	public function testGetCheeseListingCollection()
	{
	$client = self::createClient();
	$user = $this->createUser('cheeseplese@example.com', 'foo');
	... lines 80 - 103
	}
	}

	... lines 1 - 75
	public function testGetCheeseListingCollection()
	{
	... lines 78 - 80
	$cheeseListing1 = new CheeseListing('cheese1');
	$cheeseListing1->setOwner($user);
	$cheeseListing1->setPrice(1000);
	$cheeseListing1->setDescription('cheese');

	$cheeseListing2 = new CheeseListing('cheese2');
	$cheeseListing2->setOwner($user);
	$cheeseListing2->setPrice(1000);
	$cheeseListing2->setDescription('cheese');

	$cheeseListing3 = new CheeseListing('cheese3');
	$cheeseListing3->setOwner($user);
	$cheeseListing3->setPrice(1000);
	$cheeseListing3->setDescription('cheese');

	$em = $this->getEntityManager();
	$em->persist($cheeseListing1);
	$em->persist($cheeseListing2);
	$em->persist($cheeseListing3);
	$em->flush();
	... lines 101 - 103
	}
	... lines 105 - 106

	... lines 1 - 75
	public function testGetCheeseListingCollection()
	{
	... lines 78 - 101
	$client->request('GET', '/api/cheeses');
	... line 103
	}
	... lines 105 - 106

	... lines 1 - 75
	public function testGetCheeseListingCollection()
	{
	... lines 78 - 102
	$this->assertJsonContains(['hydra:totalItems' => 3]);
	}
	... lines 105 - 106

	... lines 1 - 75
	public function testGetCheeseListingCollection()
	{
	... lines 78 - 89
	$cheeseListing2->setIsPublished(true);
	... lines 91 - 95
	$cheeseListing3->setIsPublished(true);
	... lines 97 - 104
	$this->assertJsonContains(['hydra:totalItems' => 2]);
	}
	... lines 107 - 108

	... lines 1 - 4
	use ApiPlatform\Core\Bridge\Doctrine\Orm\Extension\QueryCollectionExtensionInterface;
	use ApiPlatform\Core\Bridge\Doctrine\Orm\Util\QueryNameGeneratorInterface;
	use Doctrine\ORM\QueryBuilder;
	... line 8
	class CheeseListingIsPublishedExtension implements QueryCollectionExtensionInterface
	{
	public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
	{

	}
	}

	... lines 1 - 11
	public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
	{
	if ($resourceClass !== CheeseListing::class) {
	return;
	}
	... lines 17 - 20
	}

	... lines 1 - 11
	public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
	{
	... lines 14 - 17
	$rootAlias = $queryBuilder->getRootAliases()[0];
	$queryBuilder->andWhere(sprintf('%s.isPublished = :isPublished', $rootAlias))
	->setParameter('isPublished', true);
	}