Coding the API Upload Endpoint

Thanks a lot for this tutorial series! Most of this would have been a trip through the documentation jungle to figure out on my own!

It was awesome! Thanks a lot!

Hello everyone, I am French speaking and I use google translate to translate.
First of all, thank you for this series. By following it, it helped me in many projects. However, I have a need that I would like to express to you, maybe it will be interesting in the context of a big project.
Let's imagine that I have a hundred files. How I do pagination and a logic for search?
Thanks in advance

Hi Diarill!

Nice to chat with you! And I'm sorry for replying so slowly!

Ok, so you will have MANY files and you want to paginate or search for them. Let's assume that you have a system where you can upload many "article files" (just files that you may want to attach to an article). So, you decide to create an ArticleFile entity. So if you have 1000 files, it means you have 1000 ArticleFile rows in the database.

The key to searching or paginating the files is really to ad search/pagination for this ArticleFile entity. For pagination, see https://symfonycasts.com/screencast/symfony-doctrine/pagination

For searching, you have two options:

A) Keep it simple. Create a simple:

<form action="get">
    <input type="search" name="term" value="{{ app.query.get('term') }}">
    <button type="submit">Search</button>
</form>

Then, in your controller (the same controller that renders this page with pagination), use the "term" query parameter to modify the "query builder" that you eventually pass into the pagination library. So, it would look something like:

    public function browse(ArticleFileRepository $repository, Request $request): Response
    {
        $term = $request->query->get('term');
        // inside this method, create a query builder
        // and if there IS a term, then filter by that
        // e.g. ->andWhere('article_file.name LIKE :term')->setParameter('term', '%'.$term.'%')
        $queryBuilder = $repository->createQueryBuilderFromSearch($term);
        $adapter = new QueryAdapter($queryBuilder);
        $pagerfanta = Pagerfanta::createForCurrentPageWithMaxPerPage(
            $adapter,
            $request->query->get('page', 1),
            10
        );

        return $this->render('article_files/list.html.twig', [
            'pager' => $pagerfanta,
        ]);
    }

With this setup, you can search for something. And then, when you go to "page 2" of the results, Pagerfanta will keep the search term so that you ARE seeing page 2 of that search (and not page 2 or all of the results).

B) If you need a more intelligent search, then you can turn to Elastic search. But for simple files, I don't think that would be needed.

Cheers!

Thank you for answering me and sorry for my late return. I was on a trip for my job.
Sorry again, I probably forgot to mention that the multiple files I'm referring to here are article reference files. how to make a pagination and configure a search field for the references of an article?
Thanks in advance.

Hi Diarill!

No problem :). I think the answer is the same. If you have an ArticleReference entity, then you can create a search and paginator like I mentioned above. You would use ArticleReferenceRepository instead of ArticleFileRepository.

To make the pagination & search only return references for one specific article, pass the id of the article in the URL to the browse() action (from my above example) - e.g.:

#[Route('/article/references/{id}']
public function browse(ArticleFileRepository $repository, Article $article, Request $request): Response

In this case, the {id} is used to query for the Article object. When you create the query builder for ArticleReference, you can use this (e.g. ->andWhere('article_reference.article = :article')->setParameter('article', $article))

Let me know if this helps :)

Cheers!

Sorry Rayan, maybe I'm not transcribing my thought well. In the series, you created a javascript class to manage the references of an article via ajax (adding, editing, deleting,...). similarly, how to handle pagination and search always via ajax?

thanks

Ah... I understand better now! The problem was probably more on me ;). I sometimes can't remember, at first glance, some things I do in a tutorial. If I had remembered better, I bet I would have understood earlier!

Ok, so we want that little ArticleReference "widget" on the upper right to have pagination and search? First, if I wrote this today, I'd write that same JavaScript, but inside of a Stimulus controller. It wouldn't change much, but it would guarantee that if we, for example, AJAX-loaded in a new set of article references (e.g. because the user went to "page 2"), all of the JavaScript functionality would still be attached. Probably the biggest difference when using Stimulus is that I would render all of the markup (what is currently done in the render() method of the JavaScript class) in the Twig template like normal. Then, the Stimulus controller would just be used to attach the event listeners and send the "save" Ajax requests, etc. The Ajax endpoints would now return HTML: the HTML for this "widget" area (and then you would just do something like this.element.innerHTML = the html from the Ajax call. I would isolate the HTML for the article widget area (everything INSIDE of the element that has the stimulus_controller) in a template partial so that you can render JUST that template for the Ajax responses).

So then, the challenge is just that we need (A) a search box and (B) pagination links that on submit/click, will reload that "widget" area with a fresh set of content. I can suggest two approaches to this:

1) Use Turbo Frames. You don't have to use the rest of Turbo (you could disable Turbo Drive), but this is a wonderful use-case for Turbo Frames. Wrap that entire widget in a <turbo-frame id="article-references-widget">. Then add all the pagination and searching logic I described above to the controller that renders this entire page. So, you would eventually be passing something like a articleReferencesPager into the template and using that to render the article references and the pagination links. The great thing about Turbo Frames is that... you don't need to do anything else. The form submit and pagination links will make an Ajax call to this main page (but now with things like ?page=2), the Ajax response will come back with the full page, but then Turbo frames will grab just the <turbo-frame id="article-references-widget"> and then put it in that spot. Thanks to Stimulus, all of the JavaScript will keep working.

2) If you skip using Turbo Frames, I would do the same as above... except maybe you render the pagination links manually so that you can attach {{ stimulus_action() }} to each of them. Then manually make an Ajax call to the URL to some specific endpoint that returns just the HTML for this widget section. Then, put that into this.element.

Let me know if that helps. The way I write JavaScript has changed enough since I made this tutorial (not that my original approach is wrong, but I think Stimulus is better) that adding search+pagination the way I'd do it requires some refactoring.

Cheers!

Thank you very much. I will try to implement all your recommendations. It's true that I haven't practiced enough in the Turbo and stimulus technologies yet. It's probably time to put myself on it.

Hello everyone and I hope you are doing well.
I use google for translation.
I have a problem, maybe not directly related to this series, but I made a connection in my project.
In fact, by the same logic implemented in this series, the scenario is this... the registration of an order can contain several products and for each product, we upload an image of the product. Basically I use a form with a collection to load the products. So my problem is how to save this collection of product with their images. Thanks for your help.

    /**
     * @Route("/new", name="store_new", methods={"GET", "POST"})
     * @IsGranted("ROLE_IMMO")
     */
    public function new(Request $request): Response
    {
        $store = new Store();
        $store->setAuthor($this->getUser());

        $form = $this->createForm(StoreType::class, $store)
            ->add('saveAndCreateNew', SubmitType::class)
        ;
        $form->handleRequest($request);

        if ($form->isSubmitted() && $form->isValid()) {
            /** @var Store $store */
            $store = $form->getData();

            /** @var UploadedFile $uploadReceiptNote */
            $uploadReceiptNote = $form['receiptFile']->getData();

            if ($uploadReceiptNote) {
                $newReceiptNoteFilename = $this->getUploaderHelper()->uploadReceiptFile(
                    $uploadReceiptNote,
                    $store->getReceipt()
                );
                $store->setReceipt($newReceiptNoteFilename);
            }

            /** @var UploadedFile $uploadDeliveryNote */
            $uploadDeliveryNote = $form['deliveryNoteFile']->getData();

            if ($uploadDeliveryNote) {
                $newDeliveryNoteFilename = $this->getUploaderHelper()->uploadDeliveryFile(
                    $uploadDeliveryNote,
                    $store->getDeliveryNote()
                );
                $store->setDeliveryNote($newDeliveryNoteFilename);
            }

            // -Problem start-
            $products = $store->getProducts();

            foreach ($products as $product) {
                for ($i = 0; $i < count($form['products']); $i++) {
                    /** @var UploadedFile $uploadProductFile */
                    $uploadProductFile = $form['products'][$i]['productImageFile']->getData();

                    $newProductFilename = $this->getUploaderHelper()->uploadProductFile(
                        $uploadProductFile,
                        $product->getFilename()
                    );
                }
                $product->setFilename($newProductFilename);
            }
            // -Problem end-

            $this->getManager()->persist($store);
            $this->getManager()->flush();

            $this->getLogger()->info(
                $this->getTranslator()->trans('info.new_stock_added')
            );

            $this->addFlash(
                "success",
                $this->getTranslator()->trans('flash.product_added')
            );

            if ($form->get('saveAndCreateNew')->isClicked()) {
                return $this->redirectToRoute('store_new');
            }

            return $this->redirectToRoute('store_list');
        }

        return $this->render('store/create.html.twig', [
            'form' => $form->createView(),
            'store' => $store
        ]);
    }

Hey Maxime,

Hm, I'm not sure I understand what problem exactly you have. You mentioned "Problem start/end" code... but what exactly problem? They code seems OK, but I don't know what error / issue you have.

Regarding your "Problem start/end" part of the code, let's focus on the form... and so you can get access to the products via "$form['products'])" - great, let's iterate over them. And better to use foreach instead of "for" loop for simplicity. So, your code should look like this:

            // -Problem start-
            foreach ($form['products'] as $productData) {
                    /** @var UploadedFile $uploadProductFile */
                    $uploadProductFile = $productData['productImageFile']->getData();

                    // Upload the image with your helper class
                    $newProductFilename = $this->getUploaderHelper()->uploadProductFile();

                    // Set the correct uploaded filename or path in the DB.
            }
            // -Problem end-

I'm not sure you need to iterate over all products in your $products variable... it feels like you need to iterate over all $productData['productImageFile']. Also, I don't know the signature of your uploadProductFile(), but make sure you pass the correct path to it, like it should be a full path and the upload folder should be writable.

Don't forget about debug helpers like dd() or dump() functions - you can dump some data from the form to see what you get in form and then handle it correctly in your code.

But in general, what you're doing is OK for uploading images for a collection.

I hope this helps!

Cheers!

Awesome tutorial, thank you so much! I learned so much :)

How would that work, when I use API-Platform. Is there an easy way to use LiipImagineBundle and AWS S3?

Hey Michael,

Yes, it should work I think. Just look into docs, for example: https://github.com/liip/Lii... - on how to resolve link in PHP. And it should be no matter if you use local storage or AWS S3.

I hope this helps!

Cheers!

GG

Thanks edin . GL, HF! And stay tuned we have a lot of cool stuff!

What approach would you recommend for uploading in an api platfrom based app? The official way ( https://api-platform.com/do... ) doesn't seem to allow entity specific validations.

Hey Andrei V.!

That's an excellent question :). We talk about uploads in general in our uploads tutorial (https://symfonycasts.com/sc... and include a section at the end about how uploads *sometimes* look in an API (but not in API Platform). The docs around uploads in API Platform basically come down to the fact that uploads are a bit custom and, at least according to the docs right now, are best done with a custom controller. Basically, I would use the same approach as shown in the docs, except I probably wouldn't use VichUploaderBundle - I just personally prefer processing the upload stuff myself. The key thing is that, unless you follow how we do uploads in the last 2 chapters of our upload tutorial (which is not a better approach, just an alternate approach), upload endpoints are weird: you're not sending data as JSON, you're sending it as multipart form data.

I'm not sure I've given you a great answer :). You mentioned "doesn't seem to allow entity specific validations" - what do you mean by that? Do you want an endpoint where your API client sends a file upload and other fields, and you want to be able to validate those fields (via the annotations on your entity)?

Cheers!

Thanks for the reply!
I'll try to explain my "entity specific validations" phrase:

I have a standard Symfony app which handles uploads as that:
`

/**
 * @var string
 *
 * @ORM\Column(name="picture", type="string", nullable=true)
 */
private $picture;

/**
 * @var File
 *
 * @Vich\UploadableField(mapping="good_picture", fileNameProperty="picture")
 *
 * @Assert\Image(
 *     maxWidth=247,
 *     maxHeight=170,
 *     maxSize="500k",
 *     )
 */
private $pictureFile;`

So as you may guess from this code I just use a regular form with an unmapped property and after validation and successful upload I store the filepath into a mapped one. I have a bunch of such entities. Some of them have multiple files with different validations.

Now I move my application to api platform and if I understood their approach correctly it boils down to:

1. Have separate entity for all files (MediaObject)
2. Create custom controller, creating such files via submitting multipart requests and a listener, handling file paths
3. On the client side first upload a file, and then with another normal api platform request we attach this file to our entity.

The issue I see here is that the normal file validation annotation is applicable only to MediaObject entity. And if one of my files has maxSize of 500kb and the other one 50mb — I have no control over it at the "upload" stage.
I could create some custom validation on ManyToOne side inside my entity (ie. when we attach already uploaded file), but it then makes me implement logic for removing invalid files.

Duplicating the whole MediaObject for each file property is a bit redundant)

So for now I ended up with the following solution:
`
final class CreateMediaObjectAction {
...
public function __invoke(Request $request): MediaObject

{
    $uploadedFile = $request->files->get('file');
    if (!$uploadedFile) {
        throw new BadRequestHttpException('"file" is required');
    }
    $type = $request->get('type');

    $mediaObject = new MediaObject();
    $mediaObject->setFile($uploadedFile);
    $mediaObject->setType($type);

    $validator = $this->getValidator($type);
    $violationList = $validator->validate($mediaObject);

    if ($violationList->count()) {
        throw new ValidationException($violationList);
    }

    return $mediaObject;
}

protected function getValidator($type)
{
    $baseDir = $this->kernel->getProjectDir() . '/config/validation/media';
    $filePath = $baseDir.'/'.$type.'.yml';
    if (!$this->filesystem->exists($filePath)) {
        $filePath = $baseDir.'/default.yml';
    }
    return Validation::createValidatorBuilder()
        ->addYamlMapping($filePath)
        ->addMethodMapping('loadValidatorMetadata')
        ->getValidator();
}

...
}`

So the Controller still handles all files, but it also accepts "type" field that is used for searching for a specific validation rule. It looks like this:
`App\Entity\MediaObject:
properties:

file:
  - Image:
      maxWidth: 500`

And the entity validation just checks whether the submitted file has the right type:
`

/**
 * @ORM\ManyToOne(targetEntity="App\Entity\MediaObject", cascade={"persist"})
 * @Assert\Expression("!this.getPicture() or this.getPicture().getType() === 'food_picture'")
 */
private $picture;

`

It works, but I was trying to find more elegant solution for such a trivial task)

Hey Andrei V.!

Sorry again for my slow reply! I think you're reading too much into the recommendations from API Platform. Specifically, the MediaObject idea is simply *one* pattern for handling file uploads that (I think) they choose as an example for talking about uploads in their docs. Well, that's probably not *entirely* true. The idea of a MediaObject makes it possible to have nice RESTful endpoints, because you can, for example, POST to /medias to create a "Media" resource, then, for example, make a PUT request to /products to attach that Media resource to some Product resource. However, as good as it is to try to think in RESTful ways, sometimes breaking the rules is more pragmatic, and should allow you to use your current way of doing things without too much trouble. Here's a high-level description of how it would look - using some "Product" entity as an example.

1) You already have a Product entity and it's an ApiResource. Let's also assume you've got your standard picture and pictureFile properties on it like you wrote above.

2) Create a custom operation for your Product entity e.g. POST to /products/{id}/image (or something like that) - this custom operation will follow very closely what you see in their docs (it'll be multipart-form-data for example).

3) Create the custom controller for the operation and, more or less, handle the file upload like normal. In this endpoint, you would set the "picture" property on the Product entity after processing the upload.

This should give you what you want with the setup you already have - but let me know if you hit issues or have questions. The non-RESTful part of this is the /products/{id}/image endpoint ... as what you're *really* doing by uploading is editing the Product resource, so you're not supposed to create a new URL for this. As long as you're aware when you're bending the rules and do it responsibly, I think you're in good shape.

Cheers!

What such a great tutorial! You Guys did it again, making things look easier (and funnier) than they really are.

LOL - thanks Edison V. :D