Introduction

MagicMirror is a white-box fuzzing tool written mainly in Python 3 for Solidity Smart Contracts. It supports the detection of 9 popular security vulnerabilities. It is easy to use and provides various informative reports as output. MagicMirror is fast and can generally achieve high code coverage on many contracts. MagicMirror utilizes techniques that include constraint solving, random test generation, random state exploration, coverage and data dependency guided fuzzing, and combinatorial testing.

MagicMirror works on both Linux and Windows with minimal dependency requirements, as well as a Docker image that would work in any environment as long as Docker is supported.

Features

  • Detects 9 security vulnerabilities, e.g., reentrancy, exception disorder, and dangerous delegate call.

  • Achieve high code coverage on most contracts.

  • Fast transaction execution via production Geth EVM.

  • Supports contracts written with Solidity >= 0.4.0.

  • Multiplatform support, releases in Docker image, Windows executable, Linux executable.

  • Automatic solc compiler version detection and switching. Users do not need to manually install and switch solc compilers.

  • Geth EVM included and fully configured with a custom wrapper. Users do not need to configure EVM on their own.

  • Informative coverage reports, detailed test cases for reproducing every detected vulnerability.

Vulnerability Detectors

We adopted the following detectors from the ContractFuzzer paper as vulnerability detectors for MagicMirror.

  1. Gasless Send, sending ether viasend() with out of gas error. Due to minimal gas provided for the transaction, receiving contracts with expensive fallback function won't be able to receive either.

  2. Reentrancy, a function call transferring ether appearing more than once in a chain of nested calls.

  3. Exception Disorder, in a chain of nested calls, an exception thrown at a lower-level call not propagating to the upper-level call.

  4. Freezing Ether, a contract that can receive ether not having transfer/send/call/suicide code within it.

  5. Dangerous Delegate Call, a delegate call with inputs that can be directly provided by the caller.

  6. Block Number Dependency, a transaction checking block number while sending ether.

  7. Block Time Dependency, a transaction checking block time while sending ether.

  8. Integer Overflow, a transaction with integer overflow that is not reverted.

  9. Integer Underflow, a transaction with integer underflow that is not reverted.

Input

MagicMirror takes the source code of smart contracts written in Solidity as input. Quick example here.

MagicMirror cannot fuzz contracts that only have bytecode available.

Output

MagicMirror outputs the following files:

  • .summary-reportfile reports the summary of opcode and edge coverage for both contract and function level, and detected vulnerabilities.

  • .edge_coverage_report file reports the covered edges based on the contract's runtime bytecode.

  • .opcode_coverage_report file reports the covered opcode on the runtime opcode level, the opcode coverage is also mapped to the source code level.

  • .constructor_opcode_coverage_report file reports the covered opcode on the deploy-time opcode level, excluding the to-be-deployed runtime opcode. Coverage is also mapped to the source code level.

  • .vulnerability_report file reports the functions, transactions that are detected with vulnerabilities, and all the inputs necessary for reproducing the vulnerable transactions.

  • .new_coverage_txs_report file records every transaction that triggered new code coverage during fuzzing, user can use this information to reproduce the exact coverage MagicMirror achieved.

  • -sourcecode_coverage_report.html file highlights the source code coverage constructed based on opcode coverage using source mapping.

  • .error file, in case MagicMirror fails, this file stores the trace and reason for why the failure happened.

  • .log file, when verbose is enabled, this file stores the intermediate logs produced during fuzzing.

See examples:

Summary for <Bbay.sol: Bbay>
Configuration: {'contract_dir': './sample_contracts/Bbay.sol', 'timeout': 2, 'contract_name': None, 'solc_version': None, 'solc_path': None, 'output_dir': '/home/report', 'num_of_accounts': 2, 'num_of_attacking_contracts': 2, 'exploration_depth': None, 'exploration_width': None, 'max_times_per_function': None, 'ct_strength': 2, 'disable_ct': False, 'using_dependency_guided_state_exploration': False, 'verbose': False, 'attacker_contract': Agent, 'contract': Bbay, 'file_name': 'Bbay.sol'}

	Total tx count: 1138
	Elapsed time: 0:02:01.731477

Overall coverage:
	Contract Edge Coverage: 76.30% (161/211)
	Contract Opcode Coverage: 94.12% (2351/2498)
	Successfully executed functions: burn, fallback, approve, withdrawEther, transfer, transferFrom, unfreeze, freeze
	Unable to execute functions (due to revert): N/A


Function level coverage:
	edge_cov: 100.00% (0/0) opcode_cov: 100.00% (0/0)  <safeMul(uint256,uint256)>
	edge_cov: 100.00% (0/0) opcode_cov: 100.00% (0/0)  <safeDiv(uint256,uint256)>
	edge_cov: 66.67% (2/3) opcode_cov: 95.65% (22/23)  <safeSub(uint256,uint256)>
	edge_cov: 66.67% (4/6) opcode_cov: 97.37% (37/38)  <safeAdd(uint256,uint256)>
	edge_cov: 66.67% (6/9) opcode_cov: 96.04% (218/227)  <transfer(address,uint256)>
	edge_cov: 100.00% (3/3) opcode_cov: 100.00% (62/62)  <approve(address,uint256)>
	edge_cov: 54.55% (6/11) opcode_cov: 95.84% (346/361)  <transferFrom(address,address,uint256)>
	edge_cov: 100.00% (5/5) opcode_cov: 100.00% (132/132)  <burn(uint256)>
	edge_cov: 100.00% (5/5) opcode_cov: 100.00% (166/166)  <freeze(uint256)>
	edge_cov: 100.00% (5/5) opcode_cov: 100.00% (166/166)  <unfreeze(uint256)>
	edge_cov: 100.00% (5/5) opcode_cov: 100.00% (75/75)  <withdrawEther(uint256)>
	edge_cov: 100.00% (1/1) opcode_cov: 100.00% (3/3)  <fallback()>

Vulnerabilities: 
	GASLESS_SEND(156)

Acknowledgment

MagicMirror is developed in collaboration with and funded by the National Institute of Standards and Technology (NIST).

Last updated