Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine
TRACK

Dev Tools >

COURSE

Animated Deployment with Ansistrano

Buy Access to Course

CircleCI: Auto-Deploy my Code!

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $9.00
Previous Chapter

Next Chapter
Edit on GitHub

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

This is not a tutorial about testing... but we couldn't resist! Our project actually does have a small test suite. Find your local terminal. To run them, execute:

./vendor/bin/simple-phpunit

This is a wrapper around PHPUnit. It will install some dependencies the first time you try it and then... go tests go! They pass! Despite our best efforts, we haven't broken anything.

So here is my lofty goal: I want to configure our project with continuous integration on CircleCI and have CircleCI deploy for us, if the tests pass. Woh.

CircleCI Setup

In your browser, go to https://circleci.com and login. I'll make sure I'm under my own personal organization. Then go to projects and add a new project: our's is called ansistrano-deploy.

To use CircleCI, we will need a config.yml file. Don't worry about that yet! Live dangerously and just click "Start Building": this will activate a GitHub webhook so that each code push will automatically create a new CircleCI build. The power!

Actually, this starts our first build! But since we don't have that config.yml file yet, it's not useful.

Creating .circleci/config.yml

Head back to your editor. If you downloaded the "start" code for the course, you should have a tutorial/ directory with a circleci-config.yml file inside. To make CircleCI use this, create a new .circleci directory and paste it there: but call it just config.yml:

87 lines .circleci/config.yml
version: 2
jobs:
build_and_test:
working_directory: ~/mootube
docker:
- image: php:7.1
- image: mysql:5.7
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
steps:
# Installation
- run:
name: Install System Packages
command: apt-get update && apt-get -y install git unzip zlib1g-dev
- run:
name: Install PHP Extensions
command: docker-php-ext-install pdo pdo_mysql zip
- run:
name: Install Composer
command: |
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \
php composer-setup.php && \
php -r "unlink('composer-setup.php');" && \
chmod +x ./composer.phar && \
mv ./composer.phar /usr/local/bin/composer
# Dependencies
- checkout
- restore_cache:
key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
- run: composer install --prefer-dist --no-interaction
# Force pulling Simple PHPUnit dependencies to be able to cache them as well
- run: ./vendor/bin/simple-phpunit --version
- save_cache:
key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
paths:
- '/root/.composer/cache'
- './vendor'
# Database
- run: ./bin/console doctrine:database:create --env=test
- run: ./bin/console doctrine:schema:create --env=test
- run: ./bin/console hautelook_alice:doctrine:fixtures:load --no-interaction
# To use server:start we need to install pcntl extension
- run:
name: Run web server in background
command: ./bin/console server:run
background: true
# Testing
- run: ./bin/console lint:yaml app/config
- run: ./bin/console lint:twig app/Resources
- run: ./vendor/bin/simple-phpunit
deploy:
working_directory: ~/mootube
docker:
- image: ansible/ansible:ubuntu1604
steps:
# Installation
- run:
name: Install System Packages
command: pip install --upgrade pip && pip install ansible
# Dependencies
- checkout
- restore_cache:
key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
- run: ansible-galaxy install -r ansible/requirements.yml
- save_cache:
key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
paths:
- '/root/.ansible/roles'
# @TODO Deploy to AWS here...
workflows:
version: 2
build_test_and_deploy:
jobs:
- build_and_test
- deploy:
requires:
- build_and_test

We will talk about this file in a minute... but heck! Let's get crazy and just try it first! Back on your local terminal, add that directory and commit:

git add .circleci/
git commit -m "Adding CircleCI config"

Push wrecklessly to master! This should create a new build... there it is! It's build #7... because - to be totally honest - I was doing a bit of practicing before recording. I usually try to hide that... but I'm busted this time...

Anyways, click into the build. Ah, we're on some "Workflow" screen, and you can see two different builds: build_and_test and deploy.

Builds and Workflows in config.yml

Go back to config.yml. Under jobs, we have one called build_and_test:

87 lines .circleci/config.yml
version: 2
jobs:
build_and_test:
... lines 4 - 87

It sets up our environment, installs composer, configures the database and... eventually, runs the tests!

87 lines .circleci/config.yml
version: 2
jobs:
build_and_test:
working_directory: ~/mootube
docker:
- image: php:7.1
- image: mysql:5.7
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
steps:
# Installation
- run:
name: Install System Packages
command: apt-get update && apt-get -y install git unzip zlib1g-dev
- run:
name: Install PHP Extensions
command: docker-php-ext-install pdo pdo_mysql zip
- run:
name: Install Composer
command: |
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \
php composer-setup.php && \
php -r "unlink('composer-setup.php');" && \
chmod +x ./composer.phar && \
mv ./composer.phar /usr/local/bin/composer
# Dependencies
- checkout
- restore_cache:
key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
- run: composer install --prefer-dist --no-interaction
# Force pulling Simple PHPUnit dependencies to be able to cache them as well
- run: ./vendor/bin/simple-phpunit --version
- save_cache:
key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
paths:
- '/root/.composer/cache'
- './vendor'
# Database
- run: ./bin/console doctrine:database:create --env=test
- run: ./bin/console doctrine:schema:create --env=test
- run: ./bin/console hautelook_alice:doctrine:fixtures:load --no-interaction
# To use server:start we need to install pcntl extension
- run:
name: Run web server in background
command: ./bin/console server:run
background: true
# Testing
- run: ./bin/console lint:yaml app/config
- run: ./bin/console lint:twig app/Resources
- run: ./vendor/bin/simple-phpunit
... lines 56 - 87

But we also have a second job: deploy:

87 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
... lines 58 - 87

The whole point of this job is to install Ansible and get ready to run our Ansistrano deploy. We're not actually doing this yet... but the environment should be ready:

87 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
working_directory: ~/mootube
docker:
- image: ansible/ansible:ubuntu1604
steps:
# Installation
- run:
name: Install System Packages
command: pip install --upgrade pip && pip install ansible
# Dependencies
- checkout
- restore_cache:
key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
- run: ansible-galaxy install -r ansible/requirements.yml
- save_cache:
key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
paths:
- '/root/.ansible/roles'
# @TODO Deploy to AWS here...
... lines 78 - 87

The real magic is down below under workflows:

87 lines .circleci/config.yml
... lines 1 - 78
workflows:
version: 2
build_test_and_deploy:
jobs:
- build_and_test
- deploy:
requires:
- build_and_test

The one workflow lists both builds. But, thanks to the requires config, the deploy job will only run if build_and_test is successful. That's super cool.

Back on CircleCI, that job did finish successfully, and deploy automatically started. This should setup our Ansible-friendly environment... but it will not actually deploy yet.

CircleCI Environment Vars and the Vault Pass

It's time to fix that! In config.yml, under deploy, run the normal deploy command: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --ask-vault-pass:

88 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
... lines 58 - 76
# Deploy
- run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --ask-vault-pass
... lines 79 - 88

And in theory... that's all we need! But... do you see the problem? Yep: that --ask-vault-pass option is not going to play well with CircleCI.

We need a different solution. Another option you can pass to Ansible is --vault-password-file that points to a file that holds the password. That's better... but how can we put the password in a file... without committing that file to our repository?

The answer! Science! Well yes, but more specifically, environment variables!

Back in CircleCI, configure the project. Find "Environment Variables" and add a new one called ANSIBLE_VAULT_PASS set to beefpass. Back in config.yml, before deploying, we can echo that variable into a file: how about ./ansible/.vault-pass.txt:

90 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
... lines 58 - 76
# Deploy
- run: echo $ANSIBLE_VAULT_PASS > ./ansible/.vault-pass.txt
... lines 79 - 90

Use that on the next line: --vault-password-file= and then the path:

90 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
... lines 58 - 76
# Deploy
- run: echo $ANSIBLE_VAULT_PASS > ./ansible/.vault-pass.txt
- run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --vault-password-file=./ansible/.vault-pass.txt
... lines 80 - 90

To be extra safe, delete it on the next line:

90 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
... lines 58 - 76
# Deploy
- run: echo $ANSIBLE_VAULT_PASS > ./ansible/.vault-pass.txt
- run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --vault-password-file=./ansible/.vault-pass.txt
- run: rm ./ansible/.vault-pass.txt
... lines 81 - 90

And... I'll fix my ugly YAML.

Setting Ansible Variables

Ok, problem solved! Time to deploy, right!? Well... remember how we added that prompt at the beginning of each deploy? Yep, that's going to break things too! No worries: Ansible gives us a way to set variables from the command line. When we do that, the prompt will not appear. How? Add a -e option with: git_branch=master:

90 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
... lines 58 - 78
- run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini -e "git_branch=master" --vault-password-file=./ansible/.vault-pass.txt
... lines 80 - 90

Disabling Host Key Checking

Ready to deploy... now!? Um... not so fast. Scroll up a little. Under the docker image, we need to add one environment variable: ANSIBLE_HOST_KEY_CHECKING set to no:

92 lines .circleci/config.yml
version: 2
jobs:
... lines 3 - 56
deploy:
working_directory: ~/mootube
docker:
- image: ansible/ansible:ubuntu1604
environment:
ANSIBLE_HOST_KEY_CHECKING: no
... lines 63 - 92

Whenever you SSH to a machine for the first time, SSH prompts you to verify the fingerprint of that server. This disables that. If you have a highly sensitive environment, you may need to look into actually storing the fingerprints to your servers instead of just disabling this check.

Finally... I think we're ready! Go back to your local terminal, commit the changes, and push!

Adding ssh Keys

Go check it out. Ah, here is the new build: the build_and_test job starts off immediately. Let's fast-forward. But watch, when it finishes.... yes! Visually, you can see it activate the second job: deploy.

Inside this job, it sets up the environment first. When it starts running our tasks... woh! It fails! Ah:

Failed to connect to host via ssh... no such identity... permission denied

Of course! CircleCI is trying to SSH onto our servers, but it does not have access. This works on our local machine because, when we deploy to the aws hosts, the group_vars/aws.yml file is loaded. This tells Ansible to look for the SSH key at ~/.ssh/KnpU-Tutorial.pem:

6 lines ansible/group_vars/aws.yml
---
... line 2
ansible_ssh_private_key_file: ~/.ssh/KnpU-Tutorial.pem
... lines 4 - 6

That path does not exist in CircleCI.

So... Hmmm... We could leverage environment variables to create this file... but great news! CircleCI gives us an easier way. Open up the key file and copy all of its contents. Then, in CircleCI, configure the project and look for "SSH Permissions". Add a new one: paste the key, but leave the host name empty. This will tell CircleCI to use this key for all hosts.

We are ready! In CircleCI, I'll just click rebuild. It skips straight to the deploy job and starts setting up the environment. Then... yes! It's running our playbook! OMG, go tell your co-workers! The machines are deploying the site to the other machines! It takes a minute or two... but it finishes! CircleCI just deployed our site automatically.

There's no visible difference, but we are setup!

Next, let's talk about some performance optimizations that we need to make to our deploy.

Leave a comment!

Most Useful Newest Oldest
2
Login or Register to join the conversation
Default user avatar
Default user avatar Permana Jayanta | posted 3 years ago | edited

Somehow I always get error on checkout step.

Directory (/home/circleci/MY_PROJECT_FOLDER) you are trying to checkout to is not empty and not a git repository

Probably someone had this kind of error before?

Reply
weaverryan Avatar

Yo Permana Jayanta !

Hmm, that's very interesting! I haven't gotten this before. Before this "step" that fails on CircleCI, I would add a ls -lR /home/circleci/MY_PROJECT_FOLDER - let's see what's inside that directory and find out why it's not empty!

Cheers!

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

While the fundamentals of Ansistrano haven't changed, this tutorial is built using Symfony 3, which has significant differences versus Symfony 4 and later.
Symfony 3 Ansible 2.1

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": ">=5.5.9",
        "doctrine/doctrine-bundle": "^1.6", // 1.6.8
        "doctrine/orm": "^2.5", // v2.7.2
        "incenteev/composer-parameter-handler": "^2.0", // v2.1.2
        "sensio/distribution-bundle": "^5.0.19", // v5.0.20
        "sensio/framework-extra-bundle": "^3.0.2", // v3.0.26
        "symfony/monolog-bundle": "^3.1.0", // v3.1.0
        "symfony/polyfill-apcu": "^1.0", // v1.4.0
        "symfony/swiftmailer-bundle": "^2.3.10", // v2.6.3
        "symfony/symfony": "3.3.*", // v3.3.5
        "twig/twig": "^1.0||^2.0", // v1.34.4
        "doctrine/doctrine-migrations-bundle": "^1.2", // v1.2.1
        "predis/predis": "^1.1", // v1.1.1
        "composer/package-versions-deprecated": "^1.11" // 1.11.99
    },
    "require-dev": {
        "sensio/generator-bundle": "^3.0", // v3.1.6
        "symfony/phpunit-bridge": "^3.0", // v3.3.5
        "doctrine/data-fixtures": "^1.1", // 1.3.3
        "hautelook/alice-bundle": "^1.3" // v1.4.1
    }
}

What Ansible libraries does this tutorial use?

# ansible/requirements.yml
-
    src: DavidWittman.redis
    version: 1.2.4
-
    src: ansistrano.deploy
    version: 2.7.0
-
    src: ansistrano.rollback
    version: 2.0.1
Edit on GitHub
Previous Chapter
Next Chapter
userVoice