# Breaking Into the Source With a Debugger

This page explains how to debug the native source code of the PJRT plugin.

## Prerequisites

- Clone and build the TT-XLA project.
    - The build has to be of the `Debug` type, e.g. `-DCMAKE_BUILD_TYPE=Debug`.
    - This is needed for native binaries to have debug symbols.
- Verify `gdb` is installed by running `gdb --version`.
    - Needed for debugging of native code.
- This guide is scoped to Visual Studio Code only.
- Install "C/C++" (by Microsoft) and "Python" (by Microsoft) VS Code extensions.
    - "Python" will auto-install the "Python Debugger" extension as well.
    - "Python Debugger" extension enables `debugpy` debugging.
- Create an empty `launch.json` file.
    - In the repository root, create a `.vscode/` directory (note that this
      directory is ignored by `git`).
    - Create a new file `.vscode/launch.json` with the following JSON content:
      ```json
      {
        "version": "0.2.0",
        "configurations": []
      }
      ```
      This file is used for configuring multiple debugging profiles.

## Debugging Python Integration Tests

### How to run a Python script or test in `debugpy`

Create a new debugging profile called `Python: Current File` in `launch.json`:

```json
{
  "version": "0.2.0",
  "configurations": [
    { // Python: Current File
        "name": "Python: Current File",
        "type": "debugpy",
        "request": "launch",
        "program": "${file}",
        "console": "integratedTerminal",
        "justMyCode": false
    }
  ]
}
```

Verify that the profile works:

1. Create a new Python script and set a breakpoint in VS Code.
2. Run a VS Code command `Debug: Select and Start Debugging` and select the
   `Python: Current File` profile while the Python script tab is open.
3. Validate that the breakpoint will be hit.

Now, replace the `Python: Current File` with a new profile for running tests,
`PyTest: Current File`:

```json
{
  "version": "0.2.0",
  "configurations": [
    { // PyTest: Current File
      "name": "PyTest: Current File",
      "type": "debugpy",
      "request": "launch",
      "module": "pytest",
      "args": [
        "-s",
        "${file}"
      ],
      "console": "integratedTerminal",
      "justMyCode": false
    }
  ]
}
```

Verify that this profile works:

1. Make sure `venv` is activated and `git` submodules are initialized.
2. Open a Python test from the `tests/` directory and set a breakpoint.
3. Run the new `PyTest: Current File` profile and validate that the breakpoint
   will be hit.

### How to attach `gdb` to a running PJRT client

Since running Python tests is the most common way to also test the PJRT plugin,
and because it is common to debug Python and native code side-by-side, this
section will focus on that scenario. However, this step can be applied to
any running process, assuming you have the time to attach the debugger to the
process before it exits.

First, create a new debugging profile `Native: Attach to PJRT Client`
in `launch.json`:

```json
{
  "version": "0.2.0",
  "configurations": [
    { // PyTest: Current File (from previous section)
    },
    { // Native: Attach to PJRT Client
      "name": "Native: Attach to PJRT Client",
      "type": "cppdbg",
      "request": "attach",
      "program": "${workspaceFolder}/venv/bin/python",
      "processId": "${command:pickProcess}",
      "MIMode": "gdb",
      // pjrt_plugin_tt.so is in this location
      "additionalSOLibSearchPath": "${workspaceFolder}/build/pjrt_implementation/src"
    },
  ]
}
```

Verify that this profile works:

1. Make sure `venv` is activated and `git` submodules are initialized.
2. Select a Python test to run from the `tests/` directory and set a breakpoint
   at the beginning of the test.
3. Run the `PyTest: Current File` debugging profile and wait for `debugpy` to
   break into the Python code.
    - At this point, the process running the test is stalled, and you have time
    to attach `gdb` to the process.
4. Open a C++ file that you wish to debug, and put a breakpoint where you
   wish to break. For exercise, almost all tests should pass through
   `ClientInstance::initialize`.
5. Run the `Native: Attach to PJRT Client` debugging profile without stopping
   the existing `PyTest: Current File` profile (that would kill the test driver
   process), which will prompt you to select which process you want to attach to.
   Select the `pytest` process that is running your test. Note that
   when you are in a remote SSH workspace session you will see
   multiple options, and you need to pick the remote one (the server).
6. Resume execution of the `PyTest: Current File` profile to unblock the Python
   interpreter, and wait for the breakpoint in C++ code to be hit in the
   `Native: Attach to PJRT Client` debugger session.
7. Once the breakpoint is hit, you can debug the native PJRT code.

## Debugging PJRT Unit Tests

Create a new debugging profile called `GTest: Filter and Run Tests` in
`launch.json`:

```json
{
  "version": "0.2.0",
  "configurations": [
    { // GTest: Filter and Run Tests
      "name": "GTest: Filter and Run Tests",
      "type": "cppdbg",
      "request": "launch",
      "program": "${workspaceFolder}/build/tests/pjrt/TTPJRTTests",
      "args": [
        "--gtest_filter=${input:gtestFilter}"
      ],
      "stopAtEntry": false,
      "cwd": "${workspaceFolder}",
      "environment": [],
      "externalConsole": false,
      "MIMode": "gdb",
    }
  ],
  "inputs": [
    {
      "id": "gtestFilter",
      "type": "promptString",
      "description": "Enter gtest filter (e.g., TestSuite.TestName or TestSuite.* for all tests in a suite)",
      "default": "*"
    }
  ]
}
```

Verify that this profile works:

1. Set a breakpoint in one of the PJRT unit tests.
2. Run the `GTest: Filter and Run Tests` debugging profile and enter a filter
   that matches the test (e.g. `*TestName*`).
3. Once the breakpoint is hit, you can start debugging the test. Note that you
   can debug multiple tests within the same session, as long as they match the
   given filter.