If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.
With a Subscription, click any sentence in the script to jump to that part of the video!
Login SubscribeMy favorite new feature in Symfony 4.4 and 5 - other than the fact that Messenger and Mailer are now stable - is probably the new secrets management system, which is as cool as it sounds.
Here's the deal: every app has a set of config values that need to be different from machine to machine, like different on my local machine versus production. In Symfony, we store these as environment variables.
One example is MAILER_DSN
:
... lines 1 - 38 | |
###> symfony/mailer ### | |
MAILER_DSN=null://null | |
# in Symfony 4.4 and higher, the syntax is | |
# MAILER_DSN=null://default | |
### | |
... lines 44 - 62 |
While developing, I want to use the null
transport to avoid sending real emails. But on production, this value will be different, maybe pointing to my SendGrid account.
We reference environment variables with a special syntax - this one is in config/packages/mailer.yaml
: %env()%
with the variable name inside: MAILER_DSN
:
framework: | |
mailer: | |
dsn: '%env(MAILER_DSN)%' |
If you look at the full list of environment variables, you'll notice that there are two types: sensitive and non-sensitive variables.
For example, MAILER_DSN
is a "sensitive" variable because the production value probably contains a username & password or API key: something that, if someone got access to it, would allow them to use our account. So, it's not something that we want to commit to our project.
But other values are not sensitive, like WKHTMLTOPDF_PATH
:
... lines 1 - 44 | |
###> knplabs/knp-snappy-bundle ### | |
WKHTMLTOPDF_PATH=/usr/local/bin/wkhtmltopdf | |
WKHTMLTOIMAGE_PATH=/usr/local/bin/wkhtmltoimage | |
### | |
... lines 49 - 62 |
This might need to be different on production, but the value is not sensitive: we don't need to keep it a secret. We could actually commit its production value somewhere in our app to make deployment easier if we wanted to.
So... why are we talking about this? Because, these sensitive, or "secret" environment variables make life tricky. When we deploy, we need to somehow set the MAILER_DSN
variable to its secret production value, either as a real environment variable or probably by creating a .env.local
file. Doing that safely can be tricky: do you store the secret production value in a config file in this repository or in some deploy script? You can, but then it's not very secure: the less people that can see your secrets - even people on your team - the better.
One general solution to this problem is something called a vault. The basic idea is simple: you encrypt your secrets - like the production value for MAILER_DSN
- and then store the encrypted value. The "place" where the encrypted secrets are stored is called the "vault". The secrets inside can only be read if you have the decryption password or "private key".
This makes life easier because now your secrets can safely be stored in this "vault", which can just be a set of files on your filesystem or even a cloud vault service. Then, when you deploy, the only "secret" that you need to have available is the password or private key. Some vaults also allow other ways to authenticate.
None of this "vault" stuff has anything to do with Symfony: it's just a cool concept and there are various services & projects that support the idea - the most famous being HashiCorp's Vault.
But, in Symfony 4.4, a new secrets system was added to let us do all this cool stuff out-of-the-box.
Here's the goal: instead of having MAILER_DSN
as an environment variable, we're going to move this to be an "encrypted secret".
To see how this all works clearly, let's add some debugging code to dump the MAILER_DSN
value. Open config/services.yaml
and add a new bind - $mailerDsn
set to %env(MAILER_DSN)%
- so we can use this as an argument somewhere:
... lines 1 - 11 | |
services: | |
# default configuration for services in *this* file | |
_defaults: | |
... lines 15 - 17 | |
# setup special, global autowiring rules | |
bind: | |
... lines 20 - 24 | |
$mailerDsn: '%env(MAILER_DSN)%' | |
... lines 26 - 51 |
I forgot my closing quote... which Symfony will "gently" remind me in a minute.
Next, open src/Controller/ArticleController.php
. In the homepage action, thanks to the bind, we can add a $mailerDsn
argument. Dump that and die:
... lines 1 - 13 | |
class ArticleController extends AbstractController | |
{ | |
... lines 16 - 28 | |
public function homepage(ArticleRepository $repository, $mailerDsn) | |
{ | |
dump($mailerDsn);die; | |
... lines 32 - 36 | |
} | |
... lines 38 - 64 | |
} |
Now, refresh the homepage. Booo. Let's go fix my missing quote in the YAML file. Refresh again and... perfect: the current value is null://null
.
That's no surprise: that's the value in .env
and we are not overriding it in .env.local
:
... lines 1 - 38 | |
###> symfony/mailer ### | |
MAILER_DSN=null://null | |
... lines 41 - 62 |
Ok, as soon as you have an environment variable that you want to convert to a secret, you need to fully remove it as an environment variable: do not set it as an environment variable anywhere anymore. I'll remove MAILER_DSN
from .env
and if we were overriding it in .env.local
, I would also remove it from there:
... lines 1 - 38 | |
###> symfony/mailer ### | |
# MAILER_DSN=null://null | |
... lines 41 - 62 |
Not surprisingly, when you refresh, we're greeted with a great big ugly error:
The environment variable is not found.
So how do we make MAILER_DSN
an encrypted secret? With a fancy new console command:
php bin/console secrets:set MAILER_DSN
That will ask us for the value: I'll go copy null://null
- you'll learn why I'm choosing that value in a minute - and paste it here. You don't see the pasted value because the command hides the input to be safe.
Hit enter and... awesome! Because this was the first time we added something to the secrets vault, Symfony needed to create the vault - and it did that automatically. What does that actually mean? It means that it created several new files in a config/secrets/dev
directory.
Let's go check them out: config/secrets/dev
. Ooooo.
To "create" the secrets vault, Symfony created two new files, which represent "keys": a private decrypt key and a public encrypt key. If you look inside, they're just fancy text files: they return a long key value.
The public encrypt file is something that is safe to commit to your repository. It's used to add, or "encrypt" a secret, but it can't read encrypted secrets. By committing it, other developers can add new secrets.
The private decrypt key - as its name suggests - is needed to decrypt and read secrets.
Now normally, the "decrypt" key is private and you would not commit it to your repository. However, as you may have noticed, Symfony maintains a different set of secrets per environment. The vault we created is for the dev
environment only. In the next chapter, we'll create the vault for the prod
environment.
Anyways, because secrets in the dev
environment usually represent safe "defaults" that aren't terribly sensitive, it's ok to commit the private key for the dev
environment. Plus, if you didn't commit it, other developers on your team wouldn't be able to run the app locally... because Symfony wouldn't be able to read the dev secrets.
Let's add these to git:
git status
Then git add config/secrets
and also add .env
:
git add config/secrets .env
This added all 4 files. The other two files store info about the secrets themselves: each secret will be stored in its own file and the "list" file just helps us get the full list of secrets that exist. Commit this:
git commit -m "setting up dev environment vault"
And now I have a pleasant surprise: go over and refresh the homepage. It works! That's by design: the %env()%
syntax is smart:
... lines 1 - 11 | |
services: | |
# default configuration for services in *this* file | |
_defaults: | |
... lines 15 - 17 | |
# setup special, global autowiring rules | |
bind: | |
... lines 20 - 24 | |
$mailerDsn: '%env(MAILER_DSN)%' | |
... lines 26 - 51 |
It first looks for a MAILER_DSN
environment variable. If it finds one, it uses it. If it does not, it then looks for a MAILER_DSN
secret. That's why... it just works.
To get a list of all the encrypted secrets, you can run:
php bin/console secrets:list
Yep - just one right now. Add --reveal
to see the values. By the way, this "reveal" only works because the decrypt file exists in our app.
Next: our app will not currently work in the prod
environment because there is no prod
vault and so no MAILER_DSN
prod
secret. Let's fix that and talk a bit about deployment.
Hey Mattias,
Using the bind key is just for convenience when a parameter will be used in many places.
I wouldn't inject the ParameterBag, I'd only inject the parameter that I need:
parameters:
basepath: '%env(basepath)%'
services:
App\SomeService:
arguments:
$basePath: '%env(basepath)%'
Cheers!
Oh, but I use my parameters in normal controllers, not in services.
Is my way the correct way of doing it then? Since all of my controllers aren't defined as services at all.
Are you sure your controllers are not services? That's the default since Symfony 4.4, if you can inject services into your controller's arguments, then they are defined as services. I don't usually inject parameters into my controllers, what I usually do is inject the parameter directly into the service that's going to use it. But, if for some reason you're going to use it in a controller, then you can do precisely what you did or just add a new bind key
Cheers!
Hi there, I managed to create a first secret for the prod env, but when I try to set a second, it just raise an error:Fatal error: Symfony\Component\DependencyInjection\Exception\ParameterCircularRe<br />ferenceException: Circular reference detected for parameter "env(base64:default:<br />:SYMFONY_DECRYPTION_SECRET)" ("env(base64:default::SYMFONY_DECRYPTION_SECRET)" ><br /> "env(default::SYMFONY_DECRYPTION_SECRET)" > "env(SYMFONY_DECRYPTION_SECRET)" ><br />"env(base64:default::SYMFONY_DECRYPTION_SECRET)"). in C:\laragon\www\DESmdc\vend<br />or\symfony\dependency-injection\Container.php:389
Any clue ?
Hey picks!
Ah, circular reference!
... runs away
Let's... see if we can figure this out ;). It looks like, somehow, you have a parameter that is referencing itself. If you follow the error message, it looks like eventually Symfony sees that the SYMFONY_DECRYPTION_SECRET
environment variable is set to env(base64:default::SYMFONY_DECRYPTION_SECRET)")
, which then causes a loop.
Can you post the contents of all of the relevant files? Like .env, .env.local, services.yaml (if you have anything relevant there)?
Cheers!
Sure!
My .env file has:
APP_ENV=dev
APP_SECRET=03cbf876e68a6826b4009ef99b30acd9
GOOGLE_ANALYTICS_TRACKING_ID=null
HTTP_PROTOCOL=https
SITE_BASE_SCHEME=$HTTP_PROTOCOL
SITE_BASE_HOST=desmdc.local
SITE_BASE_URL=$SITE_BASE_SCHEME://$SITE_BASE_HOST
CORS_ALLOW_ORIGIN=^https?://(localhost|127\.0\.0\.1)(:[0-9]+)?$
I don't use a .env.local for now as I don't need to override anything
And in my services.yml I have many declarations of env parameters (which some are supposed to be stored in the secret vault) like :
parameters:
asset.request_context.base_path: '%env(SITE_BASE_URL)%'
asset.request_context.secure: true
router.request_context.host: '%env(SITE_BASE_HOST)%'
router.request_context.scheme: '%env(SITE_BASE_SCHEME)%'
App\Service\Geocoder:
arguments:
$mapQuestApiKey: '%env(GEOCODING_API_KEY)%'
$env: '%env(APP_ENV)%'
App\Service\MyDesoutterCloud:
arguments:
$iwidApiBaseUri: '%app.iwid_api_base_uri%'
$iwidApiAdminEmail: '%env(IWID_API_EMAIL)%'
$iwidApiAdminPassword: '%env(IWID_API_PWD)%'
$env: '%env(string:APP_ENV)%'
$cache: '@mdc.cache'
Hello, any news on this subject? I am facing the same problem (<b>php 7.3.5</b>).
I was able to add a first variable in <b>PROD</b>, but when I wanted to edit it, I got the same error.
I also do not have the environment variable "SYMFONY_DECRYPTION_SECRET
", I did not manage to generate it. When I tried the command : php -r 'echo base64_encode(require "config/secrets/prod/prod.decrypt.private.php");'
:
I got this errorParse error: syntax error, unexpected 'private' (T_PRIVATE) in Command line code on line 1
Hi Kiuega this has been fixed and merged into SF3.4 apparently => https://github.com/symfony/...
Hey ! (ami Français), On SF 4 and 5, there seems to be nothing. No update available for recipes, it's strange.
On the other hand, did you end up succeeding in generating the environment variable SYMFONY_DECRYPTION_SECRET
?
EDIT : Nicolas Grekas said that it was OK, and that we will just have to wait for the 5.1.3 release
Just saw that yes Kiuega ;) Thanks! In the meantime I could make it work by hardcoding SYMFONY_DECRYPTION_SECRET in my .env prod file with its encoded value...
Good work around! The Pr has been merged, but not tagged yet. I would expect a tag fairly soon, as the last one was 3 weeks ago.
Cheers!
Hey picks!
Hmm. I don't see SYMFONY_DECRYPTION_SECRET
anywhere in these files? Where/how are you setting this?
To give you a bit more context, the default value of the "decryption environment value" is base64:default::SYMFONY_DECRYPTION_SECRET
. What I mean is, when Symfony tries to decrypt your fault, it looks for this SYMFONY_DECRYPTION_SECRET
environment variable , then "base64 decodes" it. It almost looks like the SYMFONY_DECRYPTION_SECRET
value is set to itself SYMFONY_DECRYPTION_SECRET
... or something similar.
Cheers!
I have no idea, I don't remember setting this value anywhere in my project!
I this something that needs to be done?
Hey picks
Yes, you whether set that env variable up or you ensure that the file config/secrets/prod/prod.decrypt.private.php
exists in your project. If that's not the case, double check your env vars and parameters to see if something is trying to use the SYMFONY_DECRYPTION_SECRET
env var.
Cheers!
Hi MolloKhan,
If I do a project search for "SYMFONY_DECRYPTION_SECRET" within PhpStorm, it only find references in the cache...
This env is not set anywhere in my project, and I have generated a config/secrets/prod/prod.decrypt.private.php (that's how I managed to generate the first secret, but not the second...). This is weird, really.
So even weirder, if I manually set a SYMFONY_DECRYPTION_SECRET env var within my .env.prod.local file, it now works!!! But I still have my config/secrets/prod/prod.decrypt.private.php...
I'm lost.
MolloKhan I was using a solid 4.4 version at the start of the project, then I upgraded it to 5.0 and 5.1... Maybe it is missing a package or something?
I have the same problem with a second project now! I downloaded a fresh copy of SF5.1 and installed the latest packages I needed, adapted and copied the "old" files(aseets, src, templates, config...), and it did the exact same : first prod secret is generated, second one raises the issue. Unless I add the SYMFONY_DECRYPTION_SECRET env var in my .env.prod.local file... This is insane.
Hey picks!
Yea, this sounds SUPER weird, so I'm checking into it further. So far, I can't repeat the error - I must be missing something that you have. Here's what I've tried so far:
1) I started a new project, and tried to create 2 secrets:
symfony new prod_decrypt_bug
cd prod_decrypt_bug
./bin/console secrets:set --env=prod SECRET1
./bin/console secrets:set --env=prod SECRET2
Both secrets were created without any problems.
2) I downloaded the course code from this page, unzipped it, and moved into the "finish" directory. Then I:
composer install
# I deleted the prod vault so that I could start from scratch
rm -rf config/secrets/prod
./bin/console secrets:set --env=prod MAILER_DSN
./bin/console secrets:set --env=prod MAILER_DSN2
Again, this had no problems.
Am I doing something different than you are? I assumed that you're getting the error right when you call secrets:set for the 2nd secret, is that correct? Or do you get the error at some other point?
Let me know!
Cheers!
weaverryan yes that's correct, that's only occuring for the 2nd secret... I'll try to set a blanck copy like you did to see if has something to do with my local dev environment... and let you know! Thanks as always.
Awesome! Let me know what you find out! It definitely sounds "fishy", but I can't repeat it :/
weaverryan the issue has been raised at SF, it seems like it is indeed a bug ==> https://github.com/symfony/...
Hey picks!
Great! I mean, not great having a bug... of course :). But at least we know others have the issue and there will be an effort to fix it. I just "pushed" on that PR to try to keep it moving. It's interesting that, on that issue, some people are able to repeat and others cannot. What version of PHP are you on? If we could identify the reason why it happens on some machines but not others, that could help a lot. I tried on PHP 7.4 (and did not get the bug).
Cheers!
Come on picks! Where is your brand consistency? :p Ok, when I hover over picks44, I can see VinZ :D
Btw, I see you're from Nantes! We visited there years ago - after the last SymfonyCon in Paris - lovely place - saw the machines de l'île of course ;)
weaverryan haha yes, cool! You should consider participating to the "web2day" event, it is quite big now, and there are lots of conferences dedicated to tech ;) could be awesome.
PS: I registered on Disqu's using FB login, that's why!
Hey TomaszGasior
Yep you are totally right, you can use dd()
for it, since Symfony 4.1.
Cheers!
// composer.json
{
"require": {
"php": "^7.3.0",
"ext-iconv": "*",
"antishov/doctrine-extensions-bundle": "^1.4", // v1.4.2
"aws/aws-sdk-php": "^3.87", // 3.110.11
"composer/package-versions-deprecated": "^1.11", // 1.11.99
"doctrine/doctrine-bundle": "^2.0", // 2.0.6
"doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // 2.1.2
"doctrine/orm": "^2.5.11", // v2.7.2
"doctrine/persistence": "^1.3.7", // 1.3.8
"easycorp/easy-log-handler": "^1.0", // v1.0.9
"http-interop/http-factory-guzzle": "^1.0", // 1.0.0
"knplabs/knp-markdown-bundle": "^1.7", // 1.8.1
"knplabs/knp-paginator-bundle": "^5.0", // v5.0.0
"knplabs/knp-snappy-bundle": "^1.6", // v1.7.0
"knplabs/knp-time-bundle": "^1.8", // v1.11.0
"league/flysystem-aws-s3-v3": "^1.0", // 1.0.23
"league/flysystem-cached-adapter": "^1.0", // 1.0.9
"league/html-to-markdown": "^4.8", // 4.8.2
"liip/imagine-bundle": "^2.1", // 2.3.0
"nexylan/slack-bundle": "^2.1", // v2.2.1
"oneup/flysystem-bundle": "^3.0", // 3.3.0
"php-http/guzzle6-adapter": "^2.0", // v2.0.1
"sensio/framework-extra-bundle": "^5.1", // v5.5.3
"symfony/asset": "5.0.*", // v5.0.2
"symfony/console": "5.0.*", // v5.0.2
"symfony/dotenv": "5.0.*", // v5.0.2
"symfony/flex": "^1.0", // v1.17.6
"symfony/form": "5.0.*", // v5.0.2
"symfony/framework-bundle": "5.0.*", // v5.0.2
"symfony/mailer": "5.0.*", // v5.0.2
"symfony/messenger": "5.0.*", // v5.0.2
"symfony/monolog-bundle": "^3.5", // v3.5.0
"symfony/security-bundle": "5.0.*", // v5.0.2
"symfony/sendgrid-mailer": "5.0.*", // v5.0.2
"symfony/serializer-pack": "^1.0", // v1.0.2
"symfony/twig-bundle": "5.0.*", // v5.0.2
"symfony/twig-pack": "^1.0", // v1.0.0
"symfony/validator": "5.0.*", // v5.0.2
"symfony/webpack-encore-bundle": "^1.4", // v1.7.2
"symfony/yaml": "5.0.*", // v5.0.2
"twig/cssinliner-extra": "^2.12", // v2.12.0
"twig/extensions": "^1.5", // v1.5.4
"twig/inky-extra": "^2.12" // v2.12.0
},
"require-dev": {
"doctrine/doctrine-fixtures-bundle": "^3.0", // 3.3.0
"fzaninotto/faker": "^1.7", // v1.8.0
"symfony/browser-kit": "5.0.*", // v5.0.2
"symfony/debug-bundle": "5.0.*", // v5.0.2
"symfony/maker-bundle": "^1.0", // v1.14.3
"symfony/phpunit-bridge": "5.0.*", // v5.0.2
"symfony/profiler-pack": "^1.0", // v1.0.4
"symfony/var-dumper": "5.0.*" // v5.0.2
}
}
Hi, what is the difference between putting application-wide parameters in the top section <b>parameters:</b> of the <b>services.yaml</b> file, and putting them as you did there under
This is how I currently do it
1) In the .env file define for example a variable called <b>basepath</b>
2) In services.yaml I do this:
class EntryController extends AbstractController
{
4) To actually get one of the variables inside a function, I then do this:
$basepath = $this->globalParameters->get('basepath');
IS there just an easier way to do this now, where I can put all app-wide variables under that <b>bind:</b> section and just pass them as an argument to my controller functions?