Overview

Most Python projects work with Wallaby out of the box — no configuration file is needed. A configuration file can optionally be provided to customize behavior such as setup/teardown or to adjust other configuration options.

Configuration file

Wallaby configuration file is a Python file named wallaby.py, containing a configure function that returns a dictionary with the configuration properties listed below.

For example:

wallaby.py

def configure():
    return {
        'reportConsoleErrorAsError': True,
    }

Caution

Wallaby determines your configuration once, when it first starts. This means that configuration changes will not take effect until Wallaby is next started.

Auto Detect Configuration Setting

The autoDetect property indicates whether Wallaby should automatically detect and configure the testing environment for your project. When set to true (default), Wallaby will attempt to identify whether your testing framework supports automatic configuration.

Wallaby’s automatic detection currently works with pytest and unittest frameworks. If you are using multiple frameworks that are supported by Wallaby, you can specify the framework you want to use by setting the autoDetect property.

def configure():
    return {
        # Default True setting will detect and configure
        # in the following order: ['pytest', 'unittest']

        # Example override: only detect and configure 'pytest'
        'autoDetect': ['pytest'],
    }

Please note: if files or tests settings are added as a list of strings, then the autoDetect setting default will be False and will need to be explicitly provided (e.g. 'autoDetect': True).

Delays setting

The delays object property specifies how much time (in milliseconds) Wallaby should wait before proceeding to the next stage of the automated test run workflow. It’s recommended to skip setting the property and leave it up to Wallaby to decide what timeouts to use, until you actually encounter a case where you think tuning the delays may help. Depending on how frequently you’d like to run your tests or how powerful your system is, fine-tuning the delays may help to optimize Wallaby and your system performance.

The run numeric delay property specifies the number of milliseconds to wait before letting Wallaby automatically run your tests as a result of your code changes.

In the configuration sample below, Wallaby will wait 0.5 seconds after editing a source file before running any affected tests.

def configure():
    return {
        'delays': {
            'run': 500,
        },
    }

Debugging

The trace boolean property, when set to true, produces a detailed Wallaby log viewable in the Wallaby Console. It can be used to investigate any possible issues and can be provided when reporting bugs.

def configure():
    return {
        'trace': True,
    }

Maximum console messages logged per test

Wallaby has a default limit (100) of console messages logged per test. If you see the Number of console messages per test exceeded maximum allowed value message in Wallaby Console and would like to increase the limit, you may use the maxConsoleMessagesPerTest setting.

def configure():
    return {
        'maxConsoleMessagesPerTest': 10000,
    }

Slow tests

Wallaby reports slow tests in Wallaby app. If you would like to set the time above which the executed test duration is reported as slow, you may use the slowTestThreshold setting (the default value is 75 ms).

def configure():
    return {
        'slowTestThreshold': 200,  # 200 ms
    }

Run Mode (Automatic / On Save)

Wallaby’s runMode settings provides two options for when test runs will be triggered, automatic and onsave. The default mode is automatic and will be used if the setting is not provided. The automatic option makes your current Wallaby session start a test run whenever code is changed in your editor or when code is saved on disk. The onsave option starts a test run ONLY when your code is saved to disk.

You may also change the runMode setting after your Wallaby session has started using editor commands.

Please note: you may also need to change your editor settings to not save files automatically in order to use this feature.

def configure():
    return {
        'runMode': 'onsave',
    }

Low code coverage level

Wallaby reports low code coverage level in Wallaby app. If you would like to set the value below which the coverage is reported as low, you may use the lowCoverageThreshold setting (the default value is 80 percent).

def configure():
    return {
        'lowCoverageThreshold': 70,  # 70%
    }

Project name

def configure():
    return {
        'name': 'My Project Name',
    }

This will change the displayed project name (the project folder name by default) to the My Project Name text in wallaby app.

Excluding file patterns from code coverage calculation

Sometimes you may just need to exclude the file from code coverage calculation, and still be able to see its coverage indicators, inline errors and messages, etc., when it is opened.

You may use the filesWithNoCoverageCalculated setting in these scenarios. For example, the following config will exclude all files that name ends with -helper.js from code coverage calculation:

def configure():
    return {
        'filesWithNoCoverageCalculated': ['src/**/*_helper.py'],
    }

Running all tests in changed or affected test files

By default, Wallaby re-runs a minimal set of tests for a code change. For example, if you are changing a particular test, wallaby will only be re-running this test. If you want all tests in the test file to run every time when one/multiple test(s) of the file are affected by your code changes, then you may use the runAllTestsInAffectedTestFile setting.

def configure():
    return {
        'runAllTestsInAffectedTestFile': True,
    }

Running all tests when change is not related to any code change

Sometimes a change to one or more project files may not be related to any application or test code (e.g. binary files, and any non-code files that are used by your application/tests).

By default, when a change is not associated with any test or application code, Wallaby run all project tests. You may change this behavior by setting the runAllTestsWhenNoAffectedTests setting to false:

def configure():
    return {
        'runAllTestsWhenNoAffectedTests': False,
    }

Also note that this setting always behaves runAllTestsWhenNoAffectedTests set to false when using the Smart Start or Exclusive Test Run features.

Maximum Log Entry Size

By default, Wallaby truncates logged messages that exceed 16,384 characters in length. You may change the maximum number of characters using the maxLogEntrySize setting:

def configure():
    return {
        'maxLogEntrySize': 32768,
    }

Maximum Trace Steps

Wallaby limits the number of steps that are recorded during a debug session to limit memory and processor use. By default, 999,999 steps are recorded. You may change the maximum number steps to record using the maxTraceSteps setting:

def configure():
    return {
        'maxTraceSteps': 2000000,
    }

Note: Increasing this setting may negatively impact Wallaby’s memory and CPU usage.

Capture Console Log Output

By default, Wallaby will capture and display any console.log values that are executed by your code and tests. This behavior can be changed with the captureConsoleLog setting:

def configure():
    return {
        'captureConsoleLog': False,
    }

File Scanning Configuration

By default, Wallaby uses your system’s native file system scanner to efficiently detect file modifications in real-time.

However, if you have issues with the native file system scanner, you configure Wallaby to poll your file system for changes by setting the fileScanMethod configuration option to poll. If you choose to use the polling file system scanner, you can also set the fileScanPollInterval configuration option to specify the interval (in milliseconds) at which the system will poll the file system to detect changes.

def configure():
    return {
        # Override the default scan method to 'poll'
        'fileScanMethod': 'poll',

        # Applicable only if fileScanMethod is 'poll'
        # Default is 100ms and binary files will be 3 times less frequent
        'fileScanPollInterval': 100,
    }

Log Limits

Wallaby has carefully determined various limits that are used when logging values to ensure that logged values are as readable as possible and that processing logged values is not too computationally expensive.

These limits can be overridden by using the logLimits setting:

def configure():
    return {
        'logLimits': {
            'inline': {
                # The depth to log for values displayed inline beside your code
                'depth': 5,

                # The maximum number of elements to log for values displayed
                # inline beside your code
                'elements': 5000,
            },
            'values': {
                'default': {
                    # The string length at which strings are truncated within
                    # log messages
                    'stringLength': 8192,
                },
                'autoExpand': {
                    # The string length at which strings are truncated when
                    # using the auto-expand feature
                    'stringLength': 8192,

                    # The maximum number of elements to log for values displayed
                    # using the auto-expand feature
                    'elements': 5000,

                    # The maximum depth to log for values displayed using the
                    # auto-expand feature
                    'depth': 10,
                },
            },
        },
    }