Lab Exercises

• Lab 1: Single Core Matrix Multiplication
  • Introduction
    • Basic Algorithm
  • Row-Major Memory Layout
    • Cache-Friendly Access Patterns
      • Implications for Matrix Multiplication
  • Linear Transformation and Computational Flexibility
  • Loop Tiling
    • Simple Tiling Example
      • Original Doubly Nested Loop
      • Tiled Version
    • Exercise 1: Tiled Matrix Multiplication
  • Introduction to Tenstorrent Architecture
    • Tile-Based Architecture
      • Memory Layout and Tiling
  • TT-Metalium Programming Model
  • Example TT-Metalium Program
    • Exercise 2: Running the Example Program
    • Program Description
      • Kernel Types and Data Flow
      • Kernel Creation and Configuration
      • Reader Kernel Code
      • Compute Kernel Code
      • Writer Kernel Code
    • Example Program Summary
  • Kernel Compilation and Execution
    • Exercise 3: Observing JIT Compile Errors
  • Debug Facilities in TT-Metalium
    • Debug Print API
      • Printing a Tile in a Dataflow Kernel
      • Caveats and Best Practices
    • Exercise 4: Using DPRINT to Debug a Kernel
    • Debugging Hangs using Stack Traces
      • tt-triage Dependencies
    • Exercise 5: Using tt-triage to Debug a Hang
    • Device Performance Profiling
    • Exercise 6: Using Device Profiling to Profile Kernels
  • Matrix Multiplication in TT-Metalium
    • Exercise 7: Implementing Matrix Multiplication in TT-Metalium
  • Conclusion
  • Troubleshooting and Additional Resources
• Lab 2: Multi Core Matrix Multiplication
  • Introduction
  • From Single Core to Multi Core Matrix Multiplication
    • Work Distribution for Multi Core Programs
    • Work Distribution in TT-Metalium
      • Determine Number of Available Tensix Cores
      • Split Work Among Tensix Cores
      • Create Circular Buffers and Kernels on Multiple Cores
      • Set Per-Core Runtime Arguments
    • Inspecting and Choosing Cores
    • Exercise 1: Multi Core Matrix Multiplication
  • Data Reuse in Multi Core Matrix Multiplication
    • Motivation
    • Blocked Matrix Multiplication
    • Limiting On-Chip Memory Usage
    • Blocked Matrix Multiplication with Data Reuse Pseudocode
    • Data Reuse Evaluation
    • Blocked Matrix Multiplication in TT-Metalium
    • Exercise 2: Multi Core Matrix Multiplication with Data Reuse
  • Potential Additional Optimizations
  • Conclusion
• Lab 3: Multicast for Improved Data Reuse in Multi Core Matrix Multiplication
  • Introduction
    • High-Level Motivation
  • Background: Tenstorrent NoC and Multicast
  • Example Multicast Program
    • Synchronization with Semaphores
    • High-Level Multicast Protocol
    • Overview of Provided Files
    • Logical vs. Device Coordinates
    • Data Movement Processors and NoC Selection
    • Receiver Kernel Overview
      • Semaphores: Local vs. Remote Access
    • Sender Kernel Overview
      • CB Address Synchronization
      • Multicast Operation
    • Compute and Writer Kernels
    • Multicast and Double Buffering
    • Multicast Exclusions
  • Debugging Hangs with Watcher
  • Exercise 1: Debugging Multicast Issues Using Watcher
  • Exercise 2: Extending the Standalone Multicast Example
  • Exercise 3: Batched Multicast for Improved Throughput
  • Applying Multicast to Multi Core Matrix Multiplication
  • Exercise 4: Multi Core Matrix Multiplication with Multicast
  • Conclusion
  • Appendix A: Watcher Log File Format