Builds with Custom Scenarios

A few chapters ago, we created this scenario.bkf file:

20 lines scenario.bkf

        
            name "Various scenarios for the site"
        
            # override with --endpoint option
        
            endpoint "https://localhost:8000"
        
            scenario
        
                name "Basic Visit"
        
                visit url("/")
        
                    name "Homepage"
        
                    expect status_code() == 200
        
                    expect css("tbody.js-sightings-list tr").count() > 10
        
                    # won't work until we're using Blackfire environment
        
                    assert metrics.sql.queries.count < 30
        
                click link("Log In")
        
                    name "Login page"
        
                    expect status_code() == 200
        
                                                    ... lines 19 - 20

It's written in a special blackfire-player language where we write one or more "scenarios" that, sort of, "crawl" a web page, asserting things, clicking on links and even submitting forms. This a simple scenario: the tool can do a lot more.

On the surface, apart from its name, this has nothing to do with the Blackfire profiler system: blackfire-player is just a tool that can read these scenarios and do what they say. At your terminal, run this file:

blackfire-player run scenario.bkf --ssl-no-verify

That last flag avoids an SSL problem with our local web server. When we hit enter... it goes to the homepage, clicks the "Log In" link and... it passes.

Scenarios in .blackfire.yaml

This is cool... but we can do something way more interesting. Copy the entire scenario from this file, close it, and open .blackfire.yaml. Add a new key called scenarios set to a |:

23 lines .blackfire.yaml

        
                                                    ... lines 1 - 6
                            
            scenarios: |
        
                                                    ... lines 8 - 23

That's a YAML way of saying that we will use multiple lines to set this.

Below, indent, then say #!blackfire-player:

23 lines .blackfire.yaml

        
                                                    ... lines 1 - 6
                            
            scenarios: |
        
                #!blackfire-player
        
                                                    ... lines 9 - 23

That tells Blackfire that we're about to use the blackfire-player syntax... which is the only format supported here... but it's needed anyways. Below, paste the scenario. Make sure it's indented 4 spaces:

23 lines .blackfire.yaml

        
                                                    ... lines 1 - 6
                            
            scenarios: |
        
                #!blackfire-player
        
                scenario
        
                    name "Basic Visit"
        
                    visit url('/')
        
                        name "Homepage"
        
                        expect status_code() == 200
        
                        expect css("tbody.js-sightings-list tr").count() > 10
        
                        # won't work until we're using Blackfire environment
        
                        assert metrics.sql.queries.count 
        
                    click link("Log In")
        
                        name "Log in page"
        
                        expect status_code() == 200

The cool thing is that we can still execute the scenario locally: just replace scenario.bkf with .blackfire.yaml. The player is smart enough to know that it can look under the scenarios key for our scenarios.

blackfire-player run .blackfire.yaml --ssl-no-verify

But if you run this... error!

Unable to crawl a non-absolute URI /. Did you forget to set an endpoint?

Duh! Our scenario.bkf file had an endpoint config:

20 lines scenario.bkf

        
                                                    ... lines 1 - 2
                            
            # override with --endpoint option
        
            endpoint "https://localhost:8000"
        
                                                    ... lines 5 - 20

You can copy this into your .blackfire.yaml file. Or you can define the endpoint by adding --endpoint=https://localhost:8000:

blackfire-player run .blackfire.yaml --ssl-no-verify --endpoint=https://localhost:8000

Now... it works!

Building the Custom Scenario

So... why did we move the scenario into this file? To find out, add this change to git... and commit it.

git add .
git commit -m "moving scenarios into blackfire config file"

Then deploy:

symfony deploy --bypass-checks

Once that finishes... let's go see what changed. First, if we simply went to our site and manually created a profile - like for the homepage - the new scenarios config would have absolutely no effect. Scenarios don't do anything to an individual profile. Instead, scenarios affect builds.

Let's start a new one: I'll give this one a title: "With custom scenarios". Go!

Awesome!! Now, instead of that "Untitled Scenario" that tested the two URLs we configured, it's using our "Basic visit" scenario! It goes to the homepage, then clicks "Log In" to go to that page.

Yep, as soon as we add this scenarios key to .blackfire.yaml, it no longer tests these URLs. In fact, these are now meaningless. Instead, we're now in the driver's seat: we control the scenario or scenarios that a build will execute.

Per Page Assertions/Tests

Even better, we have a lot more control now over the assertions - or "tests"... Blackfire uses both words - that make a build pass or fail.

For example, the "HTTP requests should be limited to one per page" will be run against all pages in the scenarios - that's 2 pages right now. But the homepage also has its own assert: that the SQL queries on this page should be less than 30. If you look back at the build... we can see that assertion! We can even click into the profile, click on "Assertions", and see both there.

So not only do we have a lot of control over which pages we want to test - even including filling out forms - but we can also do custom assertions on a page-by-page basis in addition to having global tests. I love that. And now I can remove the comment I put earlier above assert:

23 lines .blackfire.yaml

        
                                                    ... lines 1 - 6
                            
            scenarios: |
        
                                                    ... lines 8 - 9
                            
                scenario
        
                                                    ... lines 11 - 12
                            
                    visit url('/')
        
                                                    ... lines 14 - 16
                            
                        # won't work until we're using Blackfire environment
        
                                                    ... lines 18 - 23

Now that we're running this from inside an environment, this does work.

Next, let's use our power to carefully add more time-based assertions on a page-by-page basis. We'll also learn how you can add your own metrics in order to, well, write performance assertions about pretty much anything you can dream up.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.1.3",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "blackfire/php-sdk": "^1.20", // v1.20.0
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "doctrine/annotations": "^1.0", // v1.8.0
        "doctrine/doctrine-bundle": "^1.6.10|^2.0", // 1.11.2
        "doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // v2.0.0
        "doctrine/orm": "^2.5.11", // v2.6.4
        "phpdocumentor/reflection-docblock": "^3.0|^4.0", // 4.3.2
        "sensio/framework-extra-bundle": "^5.4", // v5.5.1
        "symfony/console": "4.3.*", // v4.3.10
        "symfony/dotenv": "4.3.*", // v4.3.10
        "symfony/flex": "^1.9", // v1.18.7
        "symfony/form": "4.3.*", // v4.3.10
        "symfony/framework-bundle": "4.3.*", // v4.3.9
        "symfony/http-client": "4.3.*", // v4.3.10
        "symfony/property-access": "4.3.*", // v4.3.10
        "symfony/property-info": "4.3.*", // v4.3.10
        "symfony/security-bundle": "4.3.*", // v4.3.10
        "symfony/serializer": "4.3.*", // v4.3.10
        "symfony/twig-bundle": "4.3.*", // v4.3.10
        "symfony/validator": "4.3.*", // v4.3.10
        "symfony/webpack-encore-bundle": "^1.6", // v1.7.2
        "symfony/yaml": "4.3.*", // v4.3.10
        "twig/extensions": "^1.5" // v1.5.4
    },
    "require-dev": {
        "doctrine/doctrine-fixtures-bundle": "^3.2", // 3.2.2
        "easycorp/easy-log-handler": "^1.0.7", // v1.0.9
        "fzaninotto/faker": "^1.8", // v1.8.0
        "symfony/browser-kit": "4.3.*", // v4.3.10
        "symfony/css-selector": "4.3.*", // v4.3.10
        "symfony/debug-bundle": "4.3.*", // v4.3.10
        "symfony/maker-bundle": "^1.13", // v1.14.3
        "symfony/monolog-bundle": "^3.0", // v3.5.0
        "symfony/phpunit-bridge": "^5.0", // v5.0.3
        "symfony/stopwatch": "4.3.*", // v4.3.10
        "symfony/var-dumper": "4.3.*", // v4.3.10
        "symfony/web-profiler-bundle": "4.3.*" // v4.3.10
    }
}

Edit on GitHub

TRACK

Dev Tools >

COURSE

Blackfire.io: Revealing Performance Secrets with Profiling

Builds with Custom Scenarios

Video not working?

Scenarios in .blackfire.yaml

Building the Custom Scenario

Per Page Assertions/Tests

Leave a comment!

What PHP libraries does this tutorial use?

	name "Various scenarios for the site"

	# override with --endpoint option
	endpoint "https://localhost:8000"

	scenario
	name "Basic Visit"

	visit url("/")
	name "Homepage"
	expect status_code() == 200
	expect css("tbody.js-sightings-list tr").count() > 10
	# won't work until we're using Blackfire environment
	assert metrics.sql.queries.count < 30

	click link("Log In")
	name "Login page"
	expect status_code() == 200
	... lines 19 - 20

	... lines 1 - 6
	scenarios: \|
	#!blackfire-player
	... lines 9 - 23

	... lines 1 - 2
	# override with --endpoint option
	endpoint "https://localhost:8000"
	... lines 5 - 20

	... lines 1 - 6
	scenarios: \|
	... lines 8 - 9
	scenario
	... lines 11 - 12
	visit url('/')
	... lines 14 - 16
	# won't work until we're using Blackfire environment
	... lines 18 - 23