Advanced Testing Techniques (Fuzz Testing, Symbolic Execution)

As a blockchain developer, it is crucial to ensure that your smart contracts are thoroughly tested to prevent vulnerabilities and bugs from being deployed to the mainnet. In addition to manual testing and the use of testing frameworks, there are advanced testing techniques that can help you catch even more issues before deployment. These techniques include fuzz testing and symbolic execution.

Fuzz Testing

Fuzz testing, also known as “fuzzing,” is a technique that involves feeding a program random or semi-random data in an attempt to crash it or uncover vulnerabilities. Fuzzing can be an effective way to catch edge cases and unexpected behaviors that may not have been discovered through manual testing or automated testing with predetermined inputs.

To perform fuzz testing on a Solidity contract, you can use a tool like solgraph to generate a control flow graph of the contract and use it to identify potential points for fuzz testing. Then, you can use a fuzzing library like solidity-fuzzer to generate random inputs and pass them to the contract.

Here is an example of how to use solidity-fuzzer to perform fuzz testing on a contract:

pragma solidity ^0.6.0;

import "https://github.com/OpenZeppelin/openzeppelin-solidity/contracts/math/SafeMath.sol";
import "https://github.com/nomiclabs/solidity-fuzzer/contracts/Fuzzer.sol";

contract FuzzMe {
    using SafeMath for uint;

    uint public balance;

    function sendEther(address recipient, uint amount) public {
        recipient.transfer(amount);
        balance = balance.sub(amount);
    }
}

contract FuzzTest {
    Fuzzer fuzzer;
    FuzzMe fuzzMe;

    constructor(FuzzMe _fuzzMe) public {
        fuzzer = new Fuzzer();
        fuzzMe = _fuzzMe;
    }

    function runFuzzTest() public {
        fuzzer.fuzzFunction(fuzzMe, "sendEther", 10);
    }
}

Fuzz testing can be especially useful for finding vulnerabilities in complex contracts with many possible input combinations. It is important to note that fuzz testing is not a replacement for thorough manual testing and should be used in addition to other testing methods.

Symbolic Execution

Symbolic execution is a technique that involves evaluating a program’s behavior by replacing variables with symbolic values, rather than concrete values. This allows for the exploration of all possible paths through the code and can help uncover vulnerabilities and bugs that may not have been discovered through traditional testing methods.

To perform symbolic execution on a Solidity contract, you can use a tool like Mythril, which has built-in symbolic execution capabilities. Mythril can analyze a contract and generate a report detailing potential vulnerabilities and unsafe code paths.

Here is an example of how to use Mythril to perform symbolic execution on a contract:

$ myth -x MyContract.sol

Symbolic execution can be particularly useful for identifying vulnerabilities in contracts with complex logic or conditional statements. It is important to note that symbolic execution is a resource-intensive process and may not be practical for analyzing large contracts or contracts with very high code complexity.

Conclusion

In conclusion, fuzz testing and symbolic execution are advanced testing techniques that can be used to uncover vulnerabilities and bugs in your smart contracts that may not have been discovered through traditional testing methods. Fuzz testing involves feeding a program random or semi-random data in an attempt to crash it or uncover vulnerabilities, while symbolic execution involves evaluating a program’s behavior by replacing variables with symbolic values. While these techniques can be useful in finding issues in complex contracts, they should be used in addition to manual testing and automated testing with predetermined inputs. It is important to always thoroughly test your contracts to ensure their security and reliability on the mainnet.

Exercises

To review these concepts, we will go through a series of exercises designed to test your understanding and apply what you have learned.

pragma solidity ^0.6.0;

import "https://github.com/OpenZeppelin/openzeppelin-solidity/contracts/math/SafeMath.sol";

contract FuzzTester {
    using SafeMath for uint;

    uint public balance;

    function runFuzzTest() public {
        uint startTime = now;
        uint endTime = startTime + 1 minutes; // run for 1 minute
        while (now < endTime) {
            uint randomAmount = random();
            balance = balance.add(randomAmount);
            try {
                balance = balance.sub(randomAmount);
            } catch (Exception) {
                // log error
                require(false, "Error occurred during fuzz test.");
            }
        }
    }
}

pragma solidity ^0.6.0;

import "https://github.com/OpenZeppelin/openzeppelin-solidity/contracts/math/SafeMath.sol";

contract SymbolicExecutionTester {
    using SafeMath for uint;

    uint public balance;

    function evaluateContract(Contract contract, uint[] inputValues) public view returns (uint[] outputValues) {
        for (uint i = 0; i < inputValues.length; i++) {
            outputValues[i] = contract.execute(inputValues[i]);
        }
    }
}

pragma solidity ^0.6.0;

import "https://github.com/OpenZeppelin/openzeppelin-solidity/contracts/math/SafeMath.sol";

contract DivisionContract {
    using SafeMath for uint;

    function divide(uint a, uint b) public view returns (uint) {
        require(b > 0, "Cannot divide by zero.");
        return a.div(b);
    }

    function runFuzzTest() public {
        uint randomA = random();
        uint randomB = random();
        try {
            divide(randomA, randomB);
        } catch (Exception) {
            // log error
            require(false, "Error occurred during fuzz test.");
        }
    }

    function evaluateContract(uint[] inputValuesA, uint[] inputValuesB) public view returns (uint[] outputValues) {
        for (uint i = 0; i < inputValuesA.length; i++) {
            outputValues[i] = divide(inputValuesA[i], inputValuesB[i]);
        }
    }
}

pragma solidity ^0.6.0;

import "https://github.com/OpenZeppelin/openzeppelin-solidity/contracts/math/SafeMath.sol";

contract FuzzTester {
    using SafeMath for uint;

    uint public total;

    function runFuzzTest() public {
        uint startTime = now;
        uint endTime = startTime + 1 minutes; // run for 1 minute
        while (now < endTime) {
            uint randomAmount = random();
            total = total.add(randomAmount);
            try {
                total = total.sub(randomAmount);
            } catch (Exception) {
                // log error
                require(false, "Error occurred during fuzz test.");
            }
        }
    }
}

pragma solidity ^0.6.0;

import "https://github.com/OpenZeppelin/openzeppelin-solidity/contracts/math/SafeMath.sol";

contract SymbolicExecutionTester {
    using SafeMath for uint;

    function evaluateContract(Contract contract, uint[] inputValues) public view returns (uint[] outputValues) {
        for (uint i = 0; i < inputValues.length; i++) {
            outputValues[i] = contract.execute(inputValues[i]);
        }
    }
}