Bisecting regressions

Bisecting is a way to find regressions in software. After reporting a bug on the Godot repository on GitHub, you may be asked by a contributor to bisect the issue. Bisecting makes it possible for contributors to fix bugs faster, as they can know in advance which commit caused the regression. Your effort will be widely appreciated :)

The guide below explains how to find a regression by bisecting.

What is bisecting?

Godot developers use the Git version control system. In the context of Git, bisecting is the process of performing a manual binary search to determine when a regression appeared. While it's typically used for bugs, it can also be used to find other kinds of unexpected changes such as performance regressions.

Using official builds to speed up bisecting

Before using Git's bisect command, we strongly recommend trying to reproduce the bug with an older (or newer) official release. This greatly reduces the range of commits that potentially need to be built from source and tested. You can find binaries of official releases, as well as alphas, betas, and release candidates here.

例えば、Godot 3.2 に対するバグを報告する場合、まずそのバグを Godot 3.1 (パッチリリースではありません。その理由は以下を参照) で再現してみてください。もしそれでバグが発生しなかった場合は、Godot 3.2 beta 1 (これは利用可能なすべてのテストビルドのほぼ中央に位置しています) でバグを再現してみてください。Godot 3.2 beta 1 でバグが再現できない場合は、より新しいベータや RC ビルドを試してみてください。Godot 3.2 beta 1 でバグを再現できた場合は、より古い alpha ビルドを試してみてください。

警告

For bisecting regressions, don't use patch releases such as Godot 3.1.2. Instead, use the minor version's first release like Godot 3.1. This is because patch releases are built from a separate stable branch. This kind of branch doesn't follow the rest of Godot's development, which is done in the master branch.

The Git bisect command

If you've found a build that didn't exhibit the bug in the above testing process, you can now start bisecting the regression. The Git version control system offers a built-in command for this: git bisect. This makes the process semi-automated as you only have to build the engine, run it and try to reproduce the bug.

注釈

Before bisecting a regression, you need to set up a build environment to compile Godot from source. To do so, read the Compiling page for your target platform. (Compiling Godot from source doesn't require C++ programming knowledge.)

遅いハードウェアでは、Godotのコンパイルに時間がかかることに注意してください (遅いデュアルコアCPUでは、フルリビルドするたびに1時間かかります)。つまり、フルプロセスには数時間かかる可能性があります。ハードウェアの速度が遅すぎる場合は、ここで止めて GitHub issue で "pre-bisecting" の結果を報告すると、別の貢献者がそこからバイセクティングを続けられるようになります。

To start bisecting, you must first determine the commit hashes (identifiers) of the "bad" and "good" build. "bad" refers to the build that exhibits the bug, whereas "good" refers to the version that doesn't exhibit the bug. If you're using a pre-release build as the "good" or "bad" build, browse the download mirror, go to the folder that contains the pre-release you downloaded and look for the README.txt file. The commit hash is written inside that file.

If you're using a stable release as the "good" or "bad" build, use one of the following commit hashes depending on the version:

3.2-stable
3.1-stable
3.0-stable

To refer to the latest state of the master branch, you can use master instead of a commit hash.

Get Godot's source code using Git. Once this is done, in the terminal window, use cd to reach the Godot repository folder and enter the following command:

# <good commit hash> is hash of the build that works as expected.
# <bad commit hash> is hash of the build exhibiting the bug.
$ git bisect start
$ git bisect good <good commit hash>
$ git bisect bad <bad commit hash>

Godotをコンパイルします。これは、ビルド環境を設定したことを前提としています。

# <platform> is the platform you're targeting for regression testing,
# like "windows", "x11" or "osx".
$ scons platform=<platform> -j4

Godot のビルドには時間がかかるので、できるだけ多くの CPU スレッドをタスクに割り当てたいと思います。これが -j パラメーターの役割です。ここでは、このコマンドは4つのCPUスレッドをGodotのコンパイルに割り当てています。

bin/ フォルダにあるバイナリを実行して、バグを再現してみてください。

それでもそのビルドにバグが現れる場合は、以下のコマンドを実行してください:

$ git bisect bad

ビルドしてもバグが発生しない場合は、以下のコマンドを実行してください:

$ git bisect good

上記のいずれかのコマンドを入力すると、Git は別のコミットに切り替わります。もう一度 Godot をビルドしてバグを再現してみて、結果に応じて git bisect good または git bisect bad を入力してください。これを何度か繰り返す必要があります。コミット範囲が長くなればなるほど、より多くのステップが必要になります。ほとんどのリグレッションを見つけるには、通常 5 から 10 のステップで十分です。Gitは、(最悪の場合における)残りのステップ数を教えてくれます。

Once you've completed enough steps, Git will display the commit hash where the regression appeared. Write this commit hash as a comment to the GitHub issue you've bisected. This will help in solving the issue. Thanks again for contributing to Godot :)

注釈

You can read the full documentation on git bisect here.