Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine

CommonsChunkPlugin: Shared/Vendor Entry

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $12.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

Restart Webpack!

./node_modules/.bin/webpack --watch

Woh, those JavaScript files are huge! And there's a really simple reason why: jQuery. Yep, login.js imports jquery:

... lines 1 - 2
import $ from 'jquery';
... lines 4 - 24

rep_log.js imports jquery:

import $ from 'jquery';
... lines 2 - 13

And layout.js? Yep, it imports jquery too:

... lines 1 - 2
import $ from 'jquery';
... lines 4 - 12

That's good! We use it in each module. But, it means that jQuery is packaged in each output file individually. That's super wasteful! Instead, jQuery, well, really any code that's needed on most pages, should probably live in its own file that's included on every page, and removed from everywhere else.

How can we do that? Magic. Or, the CommonsChunkPlugin.

Adding the CommonsChunkPlugin

Tip

There is a different solution for Webpack 4 called SplitChunksPlugin. Feel free to ask questions if you want to know how to get it working. The easiest way is to use Webpack Encore, which supports this feature out-of-the-box.

Open webpack.config.js. Under the plugins section, add new webpack.optimize.CommonsChunkPlugin() and pass that an object with two keys: name set to vendor and minChunks set to 2:

113 lines webpack.config.js
... lines 1 - 30
module.exports = {
... lines 32 - 93
plugins: [
... lines 95 - 105
new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
minChunks: 2,
})
],
... line 111
};

Before we talk about what this does. Try it. Restart Webpack!

./node_modules/.bin/webpack --watch

Woh! There are two really important things! First... a new file! Welcome! Webpack is now outputting vendor.js. And second, layout.js, rep_log.js and login.js are now all much smaller. Heck, login.js is tiny!

This is the power of the CommonsChunkPlugin. Wait, what the heck is a chunk? Webpack uses the word "chunk" a lot... and yet... somehow... nobody can seem to agree on a definition for chunk. But basically, a chunk refers to a bundle of code... in a generic sense. CommonsChunkPlugin has its name because it allows us to move common, shared, code into a separate chunk... i.e. a separate output file.

Thanks to this configuration, whenever Webpack sees a module that is required two or more times, it is put into the vendor.js chunk and removed from all other chunks. Yep, since jquery is imported in all three of these files, Webpack puts it in vendor.js and then does not put it in layout.js, rep_log.js, or login.js.

Including the vendor.js File

But, for this to work, vendor.js needs to be included on every page. Open the base layout: app/Resources/views/base.html.twig. Then, before layout.js, add a script tag for vendor.js:

... lines 1 - 94
{% block javascripts %}
... lines 96 - 97
<script src="{{ asset('build/vendor.js') }}"></script>
<script src="{{ asset('build/layout.js') }}"></script>
{% endblock %}
... lines 101 - 104

In web/build, open up vendor.js. Yea! You can totally see jQuery right inside. And if you looked at the other built files, you would not find it anymore.

Include everything from node_modules/?

That's amazing right? Let's make our entry files even smaller. Here's another common configuration. Instead of minChunks set to a number, you can pass a callback function with a module argument. I'll paste a bit of code:

115 lines webpack.config.js
... lines 1 - 30
module.exports = {
... lines 32 - 93
plugins: [
... lines 95 - 105
new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
minChunks: function (module) {
return module.context && module.context.indexOf("node_modules") !== -1;
}
})
],
... line 113
};

This simply says: if a module comes from the node_modules/ directory, put it in the vendor.js file. Re-run Webpack now:

./node_modules/.bin/webpack --watch

Wow! The results are super dramatic: rep_log.js is almost empty. But... there's a problem. Do you see it? Configuring the "commons" entry is not an exact science. By blindly including everything from node_modules/, we have probably hurt performance!

How? Well, imagine login.js requires some giant module from node_modules/. Even though this module is only needed for login, it will now be included in vendor.js. That means your users will need to download this giant module just to see the homepage... even though the homepage doesn't use it!

Yep, you need to find a balance between small entry files and a small vendor file, since it's included on every page.

I'll show you my favorite setup next.

Leave a comment!

12
Login or Register to join the conversation
tomchkkk Avatar
tomchkkk Avatar tomchkkk | posted 4 years ago

I'm really enjoying this course. I've been wanting to improve my javascript as well as get some exposure to webpack, and it's entertaining! But... two things are bothering me: 1) I'm failing to see how a .css file is a dependency of a .js file, as in login.js, for example. In my mind both of those files are dependencies of the template. 2) When we removed jQuery from the template script tag, I was like "Yes, I like where this is going". But we've just added vendor.js in it's place. I'm not really sure we're solving anything; just making things more complex...

Reply

Hey tomchkkk!

Ah, very interesting "botherings" :).

1) Hmm. Ok, I can understand how you think of both your .js file and your .css file as a dependency of your template. And, part of the reason that you think this way vs "Webpack" thinks a different way is that the creators of webpack typically have more of a single-page-app mentality, where your JavaScript is actually responsible for *creating* your markup - and there really is no template.

But let me see if I can just "spin" this idea slightly. Think of every page on your site as its own "app": with HTML (from your template), JavaScript & CSS. If you think about it this way, the HTML, CSS & JS are all dependencies of this abstract idea of an "app" - you need them all to make everything work smoothly. In a single-page app, your JavaScript is the absolute heart - it *is* the app - it even builds the markup. In a more traditional app, we still have templates, so the perfect "your app is your JavaScript" idea isn't quite as clean. So, we can't "include" the markup in the JavaScript file - it needs to live in its own template. But we *can* still (at least) include the CSS from the JavaScript "app". The final feeling isn't quite as clean and awesome as a single-page-app, but this is where that mentality comes from.

And, some of the bigger reasons are just practical ones: by processing your CSS through Webpack, you get minification, and the ability tp "pull things out" into your vendor entry (which is part of your next question).

2) This is another great point. But, I want to look at each step independently, so we can see what IS and what is NOT necessary.

The problem with putting jQuery as a script tag is that this creates a global jQuery object. As you know (because you've been watching the tutorial!) this means that we just need to "hope" that jQuery was already included whenever we reference the $ variable.

So, the most important change we're making is that we can now properly require jQuery wherever we need it, instead of assuming it will be globally available. This is the *really* important thing that I want us to think about first, because *it* is the bigf win! We can require variables instead of globals! :)

Of course, then you get to this chapter. Up to this point, we've written very clean, simple code: if you are in a .js file and you need jQuery, require it. Boom! How nice is that? But now, our "overall" build is not optimized - jQuery is being included in too many places. So, unfortunately, yes, this part adds a bit of complexity because you need to think a bit about your "overall" build so that you can optimize it. BUT, I want to be very clear about something: when we require jQuery in vendor.js, this does NOT make it a global variable - it simply makes sure that this file is packaged in vendor.js, but NOT also in all of our other entry files. For example, if you required jQuery in vendor.js, but then in another entry file, just started trying to use the "$" variable, it would fail! You are still responsible for requiring everything you need. The vendor.js trick is just a build optimization (caveat to this - if you followed this chapter - https://symfonycasts.com/sc... - then you actually CAN reference $ and jQuery anywhere without requiring it first. That's actually a bummer, and you shouldn't rely on it. The point of that chapter was to allow "badly written" libraries to cheat. But, it also allows us to cheat, unfortunately).

Phew! By the way, in the next version of Webpack (which was released a few months ago), this CommonsChunkPlugin went away for a different solution to this problem. I won't say more about that new solution here (because it deserves its own video probably!) but it will be supported in the next version of Webpack Encore, which I hope to release soon. It's called "split chunks" - and you can read about it now, or how to use it with Encore soon.

Cheers and keep going! Webpack is not a perfect world, but it IS a MUCH better world.

Reply
tomchkkk Avatar
tomchkkk Avatar tomchkkk | weaverryan | posted 4 years ago | edited

Thanks, weaverryan for going to the effort of writing such a thorough response.

Reply
David P. Avatar
David P. Avatar David P. | posted 5 years ago

Well rats. WebPack V4 removed CommonsChunkPlugin() and I have been unable to figure out how to get its replacement (SplitChunksPlugin()) to work.

I've added it to my webpack.config.js file and specified chunks: 'all', but to no apparent avail.
I have no new output files and my other output files are the same size as before.

An update here in the Conversation would be most welcome.

Thanks.

Reply
David P. Avatar
David P. Avatar David P. | David P. | posted 5 years ago | edited

Okay. After much digging around, I've realized that the replacement is actually just a new configuration section, specifically optimization.splitChunks.

I've added this to my webpack.config.js file:

optimization: {
    splitChunks: {
        chunks: 'all'
    }

This appears to result in the creation of additional chunks with the common modules:

$ ./node_modules/.bin/webpack
Hash: ec802fcda25872729d3a
Version: webpack 4.2.0
Time: 4312ms
Built at: 3/26/2018 9:05:35 AM
                           Asset       Size                        Chunks             Chunk Names
                       layout.js   2.44 MiB                        layout  [emitted]  layout
        dumbbell-mini-41a097.png  684 bytes                                [emitted]
  fontawesome-webfont-912ec6.svg    434 KiB                                [emitted]
  fontawesome-webfont-b06871.ttf    162 KiB                                [emitted]
fontawesome-webfont-af7ae5.woff2   75.4 KiB                                [emitted]
 fontawesome-webfont-fee66e.woff   95.7 KiB                                [emitted]
  fontawesome-webfont-674f50.eot    162 KiB                                [emitted]
                        login.js   30.5 KiB                         login  [emitted]  login
                      rep_log.js    612 KiB                       rep_log  [emitted]  rep_log
 vendors~layout~login~rep_log.js    771 KiB  vendors~layout~login~rep_log  [emitted]  vendors~layout~login~rep_log
       vendors~layout~rep_log.js    178 KiB        vendors~layout~rep_log  [emitted]  vendors~layout~rep_log
             static/dumbbell.png   6.66 KiB                                [emitted]
Entrypoint rep_log = vendors~layout~login~rep_log.js vendors~layout~rep_log.js rep_log.js
Entrypoint login = vendors~layout~login~rep_log.js login.js
Entrypoint layout = vendors~layout~login~rep_log.js vendors~layout~rep_log.js layout.js
[./assets/css/login.css] 1.1 KiB {login} [built]
[./assets/css/main.scss] 1.43 KiB {layout} [built]
[./assets/images/dumbbell-mini.png] 70 bytes {layout} [built]
[./assets/js/layout.js] 512 bytes {layout} [built]
[./assets/js/login.js] 977 bytes {login} [built]
[./assets/js/rep_log.js] 702 bytes {rep_log} [built]
[./node_modules/css-loader/index.js??ref--5-1!./assets/css/login.css] ./node_modules/css-loader??ref--5-1!./assets/css/login.css 4.68 KiB {login} [built]
[./node_modules/css-loader/index.js??ref--6-1!./node_modules/resolve-url-loader/index.js??ref--6-2!./node_modules/sass-loader/lib/loader.js??ref--6-3!./assets/css/main.scss] ./node_modules/css-loader??ref--6-1!./node_modules/resolve-url-loader??ref--6-2!./node_modules/sass-loader/lib/loader.js??ref--6-3!./assets/css/main.scss 501 KiB {layout} [built]
[./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 509 bytes {vendors~layout~rep_log} [built]
    + 364 hidden modules
$ 

The application fails, though with
<blockquote>Uncaught ReferenceError: $ is not defined at lift:218</blockquote>

Actually, the app fails in other ways as well. It appears that none of the CSS is delivered to the browser.
In fact, there are no style CSS link tags in the head.

As far as I can tell, none of the common js files is ever references in the browser.
Looking at the network tab in Chrome only shows the FOS router stuff and the app-specific JS files.

I have spent hours trying to get this to work.

Advice would be most appreciated.

Thanks

Reply

Hey David P.!

Unfortunately, I haven't yet worked with Webpack 4 - it's on my "short list" to look at soon (so we can update Webpack Encore), but it hasn't happened yet :/. From my initial reading it seems like the common chunking thing only happens (out of the box) for async chunks - the stuff we cover in the "Code Splitting" chapter. By setting the config to "all" like you did, you ARE activating it for all chunks... but apparently this will require you to add different script tags to your page - Sokra mentions it just a *tiny* bit in the first few sentences: https://gist.github.com/sok.... It appears that the chunking is more automatic, and so the exact script tags you need to have in your HTML source will vary and you need to build the script tags dynamically. That's no small feat.

Again, I need to look into it more. I believe there is a way to control these filenames - check out this guide: https://gist.github.com/gri... - search for "Let's just try to add the new config for the CommonsChunkPlugin replacement and see what happens". Again, I don't have the answers yet, just trying to give the best direction I can at this time.

Let me know what you find out - would love to hear! Cheers!

Reply
David P. Avatar
David P. Avatar David P. | weaverryan | posted 5 years ago | edited

weaverryan,

Thanks for the response.

I saw @gricard's post over the weekend and didn't really get anything out of it at the time.
Going back today, I reread it <i>and</i> all of the comments.

Using his optimization configuration:

  optimization: {
    splitChunks: {
      cacheGroups: {
        commons: {
          test:   /[\\/]node_modules[\\/]/,
          name:   'vendor',
          chunks: 'all'
        }
      }
    }
  },```


<b>and</b> placing an additional script tag in the index.html.twig template before the buiild/rep_log.js tag, (as mentioned in the comments), seems to have done the trick.

&lt;script src="{{ asset('build/vendor.js') }}">&lt;/script>
&lt;script src="{{ asset('build/rep_log.js') }}">&lt;/script>



Thanks
Reply

Hey David P.!

This is GREAT. Nice work, and thank you for sharing! This indeed seems like the "equivalent" for setting up the CommonsChunkPlugin (when you want all of node_modules to be included in it). The next question will be to see if we can make that vendor.js file act like we do in the NEXT chapter: where we make the commons chunk layout.js, and simply prevent anything in layout.js from being in other files (instead of packaging everything in node_modules). It's also possible, considering how much work they've done with the chunk stuff in v4, that thinking about the common chunk in this way is no longer the *best* way. ALL things I need to research soon :).

Again, thanks for sharing - it's a treat for me when I get to learn from the comments :).

Cheers!

Reply
David P. Avatar
David P. Avatar David P. | weaverryan | posted 5 years ago | edited

@waeverryan,

So here is something simple that you could add to the existing course at the point where we install webpack:

yarn add webpack@3 webpack-cli --dev```


You could add it to the comments and add one of those nice little banners that you use in the video itself.

As I'm sure you're aware, the @3 will cause yarn to install V3. That should remove any hassles for your students until you're ready to release versions for V4.

Just a thought.

Thanks!
Dave
--
Reply

Hey David,

Thanks for mentioning it! Yes, exactly what we're going to do. Actually, note about "webpack@3" is already added to this chapter: https://knpuniversity.com/s... . More Webpack4-related notes will be added later. Thanks for your help on it!

Cheers!

Reply
David P. Avatar
David P. Avatar David P. | posted 5 years ago

Ryan,

Since I want to finish this course and proceed to the Webpack/Encore course, should I uninstall webpack 4 and install webpack 3?
Or, is there a reasonable way to have both of them installed?

Thanks.

Reply

Hey David P.!

Great question. At the beginning of the Encore course, we actually *totally* uninstall Webpack before we install Encore (because Encore installs Webpack v3 for you). So yea, basically you'll continue with Webpack V3. But don't worry, you've already tackled (for better or worse) the BIG difference between Webpack v3 and v4 - the common chunk stuff.

Hopefully, not too long from now, we'll upgrade Encore from Webpack v3 to v4. Most likely, as an Encore user, you won't notice much difference, except that your builds will (supposedly) be faster :).

Cheers!

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

This tutorial explains the concepts of an old version of Webpack using an old version of Symfony. The most important concepts are still the same, but you should expect significant differences in new versions.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.2.0",
        "symfony/symfony": "3.3.*", // v3.3.16
        "twig/twig": "2.10.*", // v2.10.0
        "doctrine/orm": "^2.5", // v2.7.0
        "doctrine/doctrine-bundle": "^1.6", // 1.10.3
        "doctrine/doctrine-cache-bundle": "^1.2", // 1.3.5
        "symfony/swiftmailer-bundle": "^2.3", // v2.6.3
        "symfony/monolog-bundle": "^2.8", // v2.12.1
        "symfony/polyfill-apcu": "^1.0", // v1.4.0
        "sensio/distribution-bundle": "^5.0", // v5.0.22
        "sensio/framework-extra-bundle": "^3.0.2", // v3.0.26
        "incenteev/composer-parameter-handler": "^2.0", // v2.1.2
        "friendsofsymfony/user-bundle": "^2.0", // v2.1.2
        "doctrine/doctrine-fixtures-bundle": "~2.3", // v2.4.1
        "doctrine/doctrine-migrations-bundle": "^1.2", // v1.3.2
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "friendsofsymfony/jsrouting-bundle": "^1.6" // 1.6.0
    },
    "require-dev": {
        "sensio/generator-bundle": "^3.0", // v3.1.6
        "symfony/phpunit-bridge": "^3.0" // v3.3.5
    }
}

What JavaScript libraries does this tutorial use?

// package.json
{
    "dependencies": [],
    "devDependencies": {
        "babel-core": "^6.25.0", // 6.25.0
        "babel-loader": "^7.1.1", // 7.1.1
        "babel-plugin-syntax-dynamic-import": "^6.18.0", // 6.18.0
        "babel-preset-env": "^1.6.0", // 1.6.0
        "bootstrap-sass": "^3.3.7", // 3.3.7
        "clean-webpack-plugin": "^0.1.16", // 0.1.16
        "copy-webpack-plugin": "^4.0.1", // 4.0.1
        "core-js": "^2.4.1", // 2.4.1
        "css-loader": "^0.28.4", // 0.28.4
        "extract-text-webpack-plugin": "^3.0.0", // 3.0.0
        "file-loader": "^0.11.2", // 0.11.2
        "font-awesome": "^4.7.0", // 4.7.0
        "jquery": "^3.2.1", // 3.2.1
        "lodash": "^4.17.4", // 4.17.4
        "node-sass": "^4.5.3", // 4.5.3
        "resolve-url-loader": "^2.1.0", // 2.1.0
        "sass-loader": "^6.0.6", // 6.0.6
        "style-loader": "^0.18.2", // 0.18.2
        "sweetalert2": "^6.6.6", // 6.6.6
        "webpack": "^3.4.1", // 3.4.1
        "webpack-chunk-hash": "^0.4.0", // 0.4.0
        "webpack-dev-server": "^2.6.1", // 2.6.1
        "webpack-manifest-plugin": "^1.2.1" // 1.2.1
    }
}
userVoice