SymfonyCasts stands united with the people of Ukraine
gstreamer0.10-ffmpeg
gstreamer0.10-plugins-good packages.
I love our new setup! So it's time to talk about optimizing our build files for production. Yep, it's time to get serious, and make sure our files are minified and optimized to kick some performance butt!
Because, right now, if you check out the size of the build directory:
ls -la public/build
... yea! These files are pretty huge - rep_log.js is over 1 megabyte and so is layout.js! If you looked inside, you would find the problem immediately:
| ... lines 1 - 2 | |
| import $ from 'jquery'; | |
| ... lines 4 - 24 |
jQuery is packaged individually inside each of these! That's super wasteful! Our users should only need to download jQuery one time.
No problem! Webpack has an awesome solution. Open webpack.config.js. Move the layout entry to the top - though, order doesn't matter. Now, change the method to createSharedEntry():
| ... lines 1 - 3 | |
| Encore | |
| ... lines 5 - 10 | |
| .createSharedEntry('layout', './assets/js/layout.js') | |
| ... lines 12 - 25 | |
| ; | |
| ... lines 27 - 30 |
Before we talk about this, move back to your terminal and restart Encore:
yarn run encore dev --watch
Then, I'll open a new tab - I love tabs! - and, when it finishes, check the file sizes again:
ls -la public/build
Woh! rep_log.js is down from 1 megabyte to 300kb! layout.js is still big because it does still contain jQuery. But login.js - which was almost 800kb is now... 4!
What is this magical shared entry!? To slightly over-simplify it, each project should have exactly one shared entry. And its JS file and CSS file should be included on every page.
When you set layout.js as a shared entry, any modules included in layout.js are not repeated in other files. For example, when Webpack sees that jquery is required by login.js, it says:
Hold on!
jqueryis already included inlayout.js- the shared entry. So, I don't need to also put it inlogin.js.
It's a great solution to the duplication problem: if you have a library that is commonly used, just make sure that you import it in layout.js, even if you don't need it there. You can experiment with the right balance.
As soon as you do this, if you refresh, it works! I'm kidding - you'll totally get an error:
webpackJsonpis not defined
To fix that, in your base layout, right before layout.js, add one more script tag. Point it to a new build/manifest.js file:
| ... lines 1 - 96 | |
| {% block javascripts %} | |
| ... lines 98 - 100 | |
| <script src="{{ asset('build/manifest.js') }}"></script> | |
| <script src="{{ asset('build/layout.js') }}"></script> | |
| {% endblock %} | |
| ... lines 104 - 107 |
The reason we need to do this is... well.. a bit technical. But basically, this helps with long-term caching, because it allows your giant layout.js file to change less often between deploys.
Ok, this is great, but the files are still pretty big because they're not being minified. How can we tell Encore to do that? In your terminal, run:
yarn run encore production
That's it! This will take a bit longer: there's more magic happening behind the scenes. When it finishes, go back to your first open tab and run:
ls -la public/build
Let's check out the file sizes! The development rep_log.js that was 310kb is down to 74! Layout went from about 1Mb to 125kb. The CSS files are also way smaller. Yep, building for production is just one command: Encore handles all the details.
Oh, and here's a trick to be even lazier. Open package.json. I'm going to paste a new script section:
| { | |
| "devDependencies": { | |
| ... lines 3 - 11 | |
| }, | |
| "scripts": { | |
| "dev-server": "encore dev-server", | |
| "dev": "encore dev", | |
| "watch": "encore dev --watch", | |
| "build": "encore production" | |
| } | |
| } |
This gives you different shortcut commands for the different ways that you'll run Encore. Oh, we didn't talk about the dev-server, but it's another option for local development.
Anyways, now, in the terminal, we can just say:
yarn watch
Or any of the other script commands - like yarn build for production.
Talking about production, there's one last big question we need to answer: how the heck do you deploy your assets to production? Do we need to install Node on the production server?
The answer is.... it depends. It depends on how sophisticated your deployment system is. Honestly, if you have a very simple deploy system - like a simple script, or maybe even some commands you run manually - then the easiest option is to install Node and yarn on your server and run encore production on your server after pulling down the latest files.
I know: this isn't a great solution: it's a bummer to install Node just for this reason. But, it is a valid option and totally simple.
A better solution is to run Encore on a different machine and then send the final, built files to your server. This highlights an important point: after you execute Encore, 100% of the files you need live in public/build. So, for example, after you execute:
yarn run encore production
you could send the public/build directory to your production machine and it would work perfectly. If you have a "build" server, that's a great place to run this command. Or, if you watched our Ansistrano Tutorial, you could run Encore locally, and use the copy module to deploy those files.
If you have any questions on your specific situation, you can ask us in the comments.
Hey Mesut,
Hm, are you talking about running on this command in this project? Like, did you download the course code, go to start/ directory and try to build the assets for production? Or are you trying to build assets for your private project? Because this course is a bit outdated and we have a new version of this course, you can check it out here: https://symfonycasts.com/sc...
Anyway, regarding your question, it depends on what Node JS version you have I suppose. If you have the latest, most probably you would need to upgrade your JS dependencies, in particular, at least the "caniuse-lite browserslist".
Also, did you try to install the "fs" package as suggested in the second output? It seems this packages is used but missing in your dependencies. I suppose you use Yarn, then it should be: "yarn add fs --dev", and then "yarn upgrade caniuse-lite browserslist". Let me know where you stuck in this process.
I hope this helps!
Cheers!
Hello Mesut and Victor,
I have exactly the same problem here on win10 and did "yarn add fs --dev" and "yarn upgrade caniuse-lite browserlist".
The commands ran well but always the same error when running "yarn encore dev --watch"
Hey Jules,
Thank you for confirming running those 2 commands helped you! Probably fs is not a required dependency of a package, so you have to install it yourself, not sure exactly why so. Btw, here's a possible workaround I suppose: https://github.com/webpack-...
I hope it helps!
Cheers!
Hey there
Quick question. Is there any "simple" way to tell if something has changed in the assets? I'm asking because my `yarn encore run production` takes like 5 minutes to complete (yeah i know - lot of stuff), so my CI builds takes forever. I would like to be able to check if something has changed, and only then build assets from scratch.
Hey Zorpen!
Hmm, there's nothing that I'm aware of specifically, though depending on your CI environment, this should be possible. Basically, most CI environments have the ability to cache directories, so it's very easy to cache your public/build directory. However, the tricky part is knowing when you need to invalidate this cache. In theory, if you got a "checksum" if your entire assets/ directory, you could use that as the "cache key" for caching your public/build directory. Then, if any files change, the checksum would change, and the assets would rebuild.
Sorry I can't give you a more specific answer. We use CircleCI, and they do allow for cache keys like this. Anyways, it's a very interesting idea! Let me know what you find out.
Cheers!
Hi weaverryan
Sorry for bothering you, but i have another question ;)
I was able to cache assets on circleCI this way:
# assets cache
- run: git log --pretty=format:'%H' -n 1 -- assets/ > assets_checksum
- run: echo "$(cat assets_checksum)"
- restore_cache:
keys:
- project-{{ .Branch }}-{{ checksum "./assets_checksum" }}-v1
- run: ls -la public/build #here i can see that public/build directory was downloaded from cache
- run: yarn run encore production
- save_cache:
key: project-{{ .Branch }}-{{ checksum "./assets_checksum" }}-v1
paths:
- public/build
I can see that when build is ran for the first time, everything is cached like i want to, but on the next runs `- run: yarn run encore production` is always triggered. Soooo how can i check if cache was hit? I mean something like "if assets were downloaded from cache then skip yarn run encore production step"? Hope you understand what i mean :)
Hey Zorpen,
Hm, interesting question! Well, most of the time we neglect it because saving a few seconds in your entire build is not a huge performance improvement. But if you want to optimize things more - I think it's possible, but be aware of some weird edge cases, like when you upgraded webpack encore library but your assets wasn't changed and you'll have a blind spot where the new version of Encore won't be changed :) So, it's always tricky with some edge cases, so my point of view is do not complicate things much and it's not a big deal to re-build your assets every time. Though, if you have a lot of assets and you know you can save a good time with this strategy - why not, go fo it.
I quickly look at CircleCI docs and didn't find anything they would suggest for this. But I bet you can easily do it yourself - just check if "public/build/" directory is empty and only then execute "yarn run encore production" command. I think it's possible to do with one-line command, but you probably also can create a separate shell script and put all the logic there. Unfortunately, my knowledge of Unix commands are limited, so I can't quickly give you a complete command, but I think if you google - you can find commands you can use for this.
Cheers!
Hey victor
Thanks, that is exactly the thing i have done. I mean checking public/build directory :)
Hey Zorpen,
Yeah, so probably this is the best way to implement it, at least I can't think about alternative solutions :)
Cheers!
Hello,
Seems like shared entry is no longer the best solution for sharing assets in Symfony's documentation.
Will there be an update of this tutorial ?
Hey Cédric G.!
You're right! And I did it to myself :p. We released the new Webpack Encore version 3 days ago, which uses Webpack 4 where the shared entry is no longer the "recommended" solution. We will upgrade this tutorial, but I'm not sure exactly when - I want to make sure we're done with big changes inside of Encore before we do that. Until then, we would be happy to answer any questions about the new "split chunks" feature. And, the shared entry will not go away any time soon - it's still a valid solution.
Cheers!
After building for production (npm run build) and I navigate directly to the build JS file (ex: <host>/build/app.f40a0e24.js) I get the following error in Chrome:
Error: Parse error on line 1:
[0],{"7t+N":function(e,
---^
Expecting 'EOF'
When building for dev, I get no such error. The manifest file, gets no such error either. But, the shared JS file and page specific JS files do get this error when I build for production. The odd things is that I do not see any errors in my console when loading the page and such and the JavaScript that I do have on the page still seems to work... I only seem to get the error when opening the file directly in the browser.. any idea why this would be occurring? I'm a little hesitant to actually ship the code to production with the error present.
The first couple lines of the file getting the error are:
webpackJsonp([0],{"7t+N":function(e,t,n){var r,i;/*!
* jQuery JavaScript Library v3.3.1
* https://jquery.com/
*
* Includes Sizzle.js
* https://sizzlejs.com/
*
* Copyright JS Foundation and other contributors
* Released under the MIT license
* https://jquery.org/license
*
* Date: 2018-01-20T17:24Z
*/
!function(t,n){"use strict";"object"==typeof e&&"object"==typeof e.exports?e.exports=t.document?n(t,!0):function(e){if(!e.document)throw new Error("jQuery requires a window with a document");return n(e)}:n(t)}
Hey Eric
Hmm, interesting, makes me wonder if you run `yarn build` instead of `npm run build` would cause any difference. Also try updating your node & yarn version
If nothing changes, let us know!
Cheers!
Hi MolloKhan
Thank you for the speedy reply. I updated everything I could think of (node, npm, yarn) and error still shows up when I navigate to the file directly. I am not sure why Chrome is showing the error. I was going to update Chrome as well but it says I am already on the latest version.
My versions of packages are:
node: v10.9.0
npm: 6.2.0
yarn: 1.9.4
Running yarn build instead of npm run build yields the same result as well. I downloaded Firefox and cant see any error or anything. Maybe something with Chrome on mac? I've tried searching online for the error but have not come up with anything concrete. Very odd.
Indeed it's odd, I don't have that problem on Chrome but I'm on Windows. You said that the site works correctly, right? Probably a Chrome's plugin that you have installed is causing this weird behavior, just for curiosity, try disabling them all and open a new window in "incognito" mode.
Welp... you are right. Tried an incognito window and it showed no error. Then I started toggling the plugins for Chrome one-by-one and the culprit was the "JSONView" plugin. Totally forgot I had that plugin installed. My apologies for the wild goose chase! But thank you for helping me figure it out!
That sucks about prod... I wonder if you could use something like salt to build it on the master and then push it to the minion when you deploy.
What about just committing your build files?
Hey Matt,
So there're a few strategies:
- Deploy changes to prod and build assets on the prod server, such tools as Ansistrano allow you to do so in a background, i.e. your website won't be totally style-broken when you re-build your assets.
- Or build assets locally and just push them to production, so you even do not need to have NodeJS installed on production. And once again, Ansistrano handles it well.
In case your wondering about Ansistrano deploy tool, we have a screencast about it here: https://knpuniversity.com/s...
> What about just committing your build files?
That's another, 3rd and good strategy... but have some disadvantages, but mostly depends on your configuration. Here's a one of them which I do not like a lot: If you use a hash strategy to solve the cache assets problem (like file's names as main_9foiwhf.js), each build the same files will have a totally different names which means Git unable to manually understand that it was just renaming, and Git will think you removed old file and created a totally new file, and as a result, your Git repository will be slowly bloated. Btw, sometimes you want to handle some images with Encore which will copy to the build directory that also will bloat your Git repo.
So, I think 3rd strategy is not a good one, so I'd recommend to choose between those 2 I mentioned in the beginning.
Cheers!
Hi guys, I am using Ansistrano to deploy my Symfony 3 app and my deploy playbook was working fine until today that I had an weird error executing the task Install Webpack Encore Assets.
The error is this: FAILED! => {"changed": true, "cmd": ["./node_modules/.bin/encore", "production"], "delta": "0:02:22.350866", "end": "2018-05-15 18:16:34.750211", "failed": true, "msg": "non-zero return code", "rc": -9, "start": "2018-05-15 18:14:12.399345", "stderr": "", "stderr_lines": [], "stdout": "Running webpack ...", "stdout_lines": ["Running webpack ..."]}
I didn't find anything related to this in Google and my app is working in localhost. Also, I have tried to execute the command './node_modules/.bin/encore production' directly on my server and is working. I didn't make any changes to my webpackconfig.js. So, I don't know what this can be. I have tried many times to deploy and the same error is still there.
Can you give me a tip or some advice to fix this? I hope you can help me.
Hey Cesar,
I think somehow the command returns not a 0 code that's why Ansible thinks it fails... Could you double check it? On the prod server run the command manually and then check its status:
./node_modules/.bin/encore production
echo $?
Does it return 0 or a different number? Because that's weird that you can run this command successfully manually, but not with Ansible :/
Btw, had you upgraded your Ansible before that error occurred?
Cheers!
Hi Victor,
I was working with ansible 2.4 and I had the error. Now, I upgrade to ansible 2.5 and I have the same error.
I run the command manually in my server and it works fine (I run it in the current directory). Also, it returns 0.
Do you have any idea what the problem can be?
Cesar
Hey Cesar,
Hm, 0 is good here, ok, could you show this entire Ansible task to us? Probably the problem in it. Also, do you use command Ansible module? If so, try to replace it with shell module and try again. Or do the reverse if you use shell module.
Cheers!
Hey Victor,
This is task and it's located in after_symlink_shared.yml
- name: Install Webpack Encore Assets
command: './node_modules/.bin/encore production'
args:
chdir: '{{ ansistrano_release_path.stdout }}'
`</pre >
Your proposal is to change command for shell?
Please, let me know.
Hey Cesar,
Hm, your task looks valid, let's try to change it slightly, try to run encore via "nodejs":
- name: Install Webpack Encore Assets
command: nodejs ./node_modules/.bin/encore production
args:
chdir: '{{ ansistrano_release_path.stdout }}'
environment:
NODE_ENV: production
And probably setting NODE_ENV env var could help here. If not, try to change command module to shell one and try again.
Cheers!
Hi Victor,
It's not working from Ansible in any way. Neither with command, shell or via nodes that you told me.
But, I was trying to do it manually in the current directory in my server and sometimes it didn't work and it was giving me an exit code 137. I was looking in the internet about that and, apparently, it's related to memory. However, I don't understand how is that possible because I check my server and it was fine in terms of memory and I am trying in a pre-production environment.
Have you ever had this error?
Cesar
Hey Cesar,
OK, great job to find the root of your problem, well done! Well, I haven't heard about this problem before, so nothing much I can tell you about solutions, probably better to google for a good advices in your case. But if you won't find any working solution on the internet I do see a few solutions for you:
- Increase memory, looks like you really need just a bit more memory to make it working;
- Or stop installing deps and building assets on your prod server. Instead, install and build them locally, then just use synchronize Ansible module to *rsync* the built assets to your prod server. This way you even do not need to have NodeJS on your server
Cheers!
Thanks Victor. I didn't want to increase the memory because is a server of pre-production. I have followed your second recommendation and it's working well and the deploy it's much faster.
Cheers!
Hey Cesar,
Glad you find a working solution for you. And actually many devs lean to this deploy strategy because do not want to install NodeJS on their production server at all. So I even do not consider this as a workaround. Anyway, if it works for you - that's awesome!
Cheers!
I'm using webpack encore in my new project, everything going fine but i don't find any manifest.js in my build folder.
Actually, my console not notifing me the webpackJsonp error but i'm not sure about that.
Is it normal?
Is this file generate by encore only in certain situation or i missing something?
Thanks in advance
Hey Andrea
The "manifest.json" file is only generated when you execute Encore, when the process finishes, then you will see all your assets and the manifest file inside your build folder
Cheers!
Sorry, i'm talking about manifest.js and not manifest.json (the second one is correctly inside the build folder, the first one not.).
Hey Andrea,
OK, what is manifest.js then? :) Because IIRC we do not talk about manifest.js file but manifest.json instead. Does it your custom JS file? If so, probably some name conflicts? Could you try to rename this file into a different file name?
Cheers!
I'm talking about the file named in this tutorial, this is the link: https://knpuniversity.com/s...
Maybe i missed something?
Hey Andrea,
Ah, sorry, I totally missed it! Yes, you're right, we do have manifest.js file here due to the shared entry. So, did you change the line for layout.js to:
Encore
.createSharedEntry('layout', './assets/js/layout.js')
Keep in mind, you don't have to have any other line related to "layout.js" except this one in your webpack.config.js.
And another important condition: You need to restart the Encore, i.e. stop watching files and re-run that command again:
yarn run encore dev --watch
Unless you restart the Encore, you won't see that manifest.js file in the build/ directory.
Cheers!
// composer.json
{
"require": {
"php": "^7.2.0",
"ext-iconv": "*",
"composer/package-versions-deprecated": "^1.11", // 1.11.99
"doctrine/doctrine-bundle": "^1.6", // 1.8.1
"doctrine/doctrine-cache-bundle": "^1.2", // 1.3.2
"doctrine/doctrine-fixtures-bundle": "~3.0", // 3.0.2
"doctrine/doctrine-migrations-bundle": "^1.2", // v1.3.1
"doctrine/orm": "^2.5", // v2.7.2
"friendsofsymfony/jsrouting-bundle": "^2.2", // 2.2.0
"friendsofsymfony/user-bundle": "dev-master", // dev-master
"sensio/framework-extra-bundle": "^5.1", // v5.1.5
"symfony/asset": "^4.0", // v4.0.4
"symfony/console": "^4.0", // v4.0.4
"symfony/flex": "^1.0", // v1.17.6
"symfony/form": "^4.0", // v4.0.4
"symfony/framework-bundle": "^4.0", // v4.0.4
"symfony/lts": "^4@dev", // dev-master
"symfony/monolog-bundle": "^3.1", // v3.1.2
"symfony/polyfill-apcu": "^1.0", // v1.7.0
"symfony/serializer-pack": "^1.0", // v1.0.1
"symfony/swiftmailer-bundle": "^3.1", // v3.1.6
"symfony/twig-bundle": "^4.0", // v4.0.4
"symfony/validator": "^4.0", // v4.0.4
"symfony/yaml": "^4.0", // v4.0.4
"twig/twig": "2.10.*" // v2.10.0
},
"require-dev": {
"symfony/debug-pack": "^1.0", // v1.0.4
"symfony/dotenv": "^4.0", // v4.0.4
"symfony/phpunit-bridge": "^4.0", // v4.0.4
"symfony/web-server-bundle": "^4.0" // v4.0.4
}
}// package.json
{
"devDependencies": {
"@symfony/webpack-encore": "^0.19.0", // 0.19.0
"bootstrap": "3", // 3.3.7
"copy-webpack-plugin": "^4.4.1", // 4.4.1
"font-awesome": "4", // 4.7.0
"jquery": "^3.3.1", // 3.3.1
"node-sass": "^4.7.2", // 4.7.2
"sass-loader": "^6.0.6", // 6.0.6
"sweetalert2": "^7.11.0", // 7.11.0
"webpack-notifier": "^1.5.1" // 1.5.1
}
}
Hello Ryan i would like to thank you for all awesome tutorials. i have an issue about webpack. yarn run encore dev is building assets but production not.Running webpack ...
Browserslist: caniuse-lite is outdated. Please run next command `yarn upgrade caniuse-lite browserslist`
ERROR Failed to compile with 5 errors 2:49:44 PM
This dependency was not found:
* fs in ./node_modules/destroy/index.js, ./node_modules/etag/index.js and 3 others
To install it, you can run: npm install --save fs
Entrypoint admin = runtime.fa8f03f5.js admin.aad16fd9.css admin.8ffd58e7.js
Entrypoint site = runtime.fa8f03f5.js site.e0886fee.css site.af26703b.js
error Command failed with exit code 2.
i allways get error try to yarn run encore production. it makes clear deleting all assets and not building again. i made much search but cant find solution. i use ubuntu as os. any idea?