Alexander Ingare

Language model inference on edge devices is fundamentally limited by thermals before compute. A Jetson system operates within a 2- to 50-watt budget under a 100°C ceiling, frequently without active cooling. Standard frequency governors handle these constraints poorly: they drive the accelerator near its maximum until throttling intervenes, and they remain blind to the distinct compute characteristics of prefill and decode phases. EnergyLLM replaces these governors with a dynamic runtime scheduler. Using a reinforcement learning agent trained via Group Relative Policy Optimization (GRPO), it independently sets prefill and decode GPU frequencies, request batch sizes, and the number of transformer layers to execute, conditioning every decision on real-time thermal and power headroom.

Scheduling is formulated as a Markov decision process. The state space captures normalized prefill and decode frequency bins, batch size, and remaining thermal headroom, mapping to an action space of several thousand discrete configurations. The reward function balances normalized throughput, energy efficiency, and the energy-delay product. By applying exponential penalties on temperature, latency, power, and exit entropy, the system effectively converts a constrained optimization problem into an unconstrained formulation. The agent’s behavior divides into two regimes: it maximizes throughput when headroom is ample, but shifts to preserving energy once effective headroom falls below 33% (roughly 80°C junction temperature). Training proceeds in two phases: supervised pre-training against reward-weighted grid search data, followed by online fine-tuning on hardware using group-relative advantages.

The scheduler controls four primary dimensions. First, independent frequency scaling exploits the asymmetry between the compute-bound prefill phase and the memory-bound decode phase. Second, batch sizing trades per-request energy costs against token throughput. Third, dynamic early exit recovers the throughput lost to larger batches by terminating the forward pass before the final layer. This utilizes a Llama 3.2 1B LayerSkip model, allowing the agent to dynamically select between layers 9 and 16. Unlike quantization or pruning, this requires no specialized hardware or weight reformatting, and unused layers are never transferred to the GPU. To regulate this, an entropy penalty prevents low-confidence early exits when thermal headroom permits deeper execution. Finally, a factorized action head reduces the output layer from 123,904 to 3,872 parameters. This cuts the memory footprint by 96.8% and policy update costs by over 48%, all while maintaining less than a 3% accuracy loss.

Evaluation was conducted on a Jetson AGX Orin using 1,000 prompts from the Natural Questions dataset. The built-in fan was disabled to model passive cooling, and power was sampled via tegrastats at one-millisecond intervals. Compared to a DVFS baseline pinned to maximum frequency and batch size, which quickly hits the thermal limit and crashes, EnergyLLM stabilizes at 92°C against the baseline’s 99°C peak. This reduces end-to-end latency by 87.5% and power draw by 47%, holding latency near 4 seconds against a 15-second Service Level Objective (SLO) that the baseline eventually violates. Furthermore, dynamic early exit drives a 29.3% reduction in total request time relative to the best static configuration, and a 33.1% reduction relative to DVFS at 99°C. Agent overhead remains marginal: action selection consumes under 0.1% of total latency and energy, while online policy updates require under 3%. Compared to an exhaustive grid search, the policy converges on the same optimal configuration while reducing search energy, power draw, and latency by roughly 80%.

Imagine a world where you can receive a professional, personalized massage in the comfort of your home. TherapeuTech brings this vision to life with an automated back massage system powered by a robotic arm, depth-sensing camera, and massage gun. Designed for safe, user-friendly human-robot interaction, the system identifies massage regions, plans a trajectory, and executes the massage with intuitive user input. Built around the Kinova Jaco2 arm and Intel RealSense L515 depth camera, its modular software can be adapted to other robotic platforms supporting Cartesian planning and point cloud sensing.

To reduce model complexity and support efficient instance segmentation, the system converts 3D point clouds into 2D binary occupancy grids. We generated 200,000 synthetic, auto-labeled training samples using human-body heuristics and fine-tuned a YOLOv11 model. Detected regions in 2D are mapped back to 3D space for Cartesian trajectory planning via the RRTConnect algorithm.

The system relies on two key extrinsic transforms: one from the depth camera to the robot base frame, and another from the massage gun to the robot’s end-effector. The first transform is determined automatically from Iterative Closest Point registration with the point cloud from the L515, and a synthetic point cloud derived from the known robot collision model. The latter transform is found from CAD-based values. By integrating these transforms, we ensure that all perception and motion modules operate in a unified single spatial world frame.

The hardware setup includes a tripod-mounted top-down depth camera, a massage gun, and a custom-designed 3D-printed mounting bracket for the Kinova arm. To gain full control over massage actuation, the built-in controller of the massage gun was bypassed using an Arduino Uno R3 and L298 motor driver, enabling a custom motor control stack within our ROS-based system. A touchscreen and ESP32-based voice control interface enable wake-word detection, emergency stop activation, and dynamic user input for adjusting massage strength and regions. This integrated multi-platform design offers hands-free safety, real-time customization, and a user-centric experience.

The K-Nearest Neighbors (KNN) algorithm is a foundational supervised learning method that classifies data points based on their proximity to stored training instances. Unlike parametric models, KNN lacks a dedicated training phase and defers all computational work to classification time. To evaluate a new instance, the algorithm computes a distance metric against the entire training dataset, sorts the results, and assigns a class based on a majority vote among the closest k neighbors. For image classification, this involves flattening images into one-dimensional feature vectors and calculating Euclidean distances across thousands of data points. Because this workload consists almost entirely of independent distance calculations, it is an excellent candidate for GPU acceleration and a direct instrument for evaluating GPU memory hierarchies.

To robustly assess performance, this project processed three datasets of increasing complexity: MNIST at 28 by 28 pixels in grayscale, CIFAR-10 at 32 by 32 pixels in color, and a labeled subset of STL-10 at 96 by 96 pixels in color. Execution was evaluated across three generations of NVIDIA GPUs chosen for their varying shared memory capacities per streaming multiprocessor: the Tesla P100 with 64 KB, the Tesla V100 with 96 KB, and the A100 with 164 KB. The evaluation recorded total execution time, GPU kernel time, memory utilization, and classification accuracy across different block sizes and k values, while relying on the NVIDIA Thrust library to handle the distance sorting.

Hardware optimization progressed through three distinct implementations to maximize data reuse. The initial approach loaded testing images into shared memory but yielded no performance gain, as global memory traffic for the training set heavily dominated the workload. Furthermore, because shared memory is localized to a block, threads read the cached test images only once without exploiting any reuse. The second approach attempted to store training images in shared memory but encountered a hard hardware limit: a single STL-10 image requires approximately 110 KB, which exceeds the block capacities of both the P100 and V100. Attempting to work around this limitation by tiling the image resulted in severe slowdowns, extending MNIST processing time to over 1,450 seconds. Optimal performance was ultimately achieved in a third design that holds one training image in shared memory per block and batches all test images against it. By assigning one thread per test image, the kernel ensures the cached training data is read continuously rather than just once.

This batched implementation substantially outperformed the baseline kernel across nearly all configurations. Its structural constraints are clearly visible in the final metrics: STL-10 batching registers zero performance on the P100 and V100 because the training image cannot fit into shared memory. Conversely, the algorithm performs exceptionally well on the A100, where the 164 KB capacity comfortably holds the STL-10 image and allows every available thread to process the batched testing data. Classification accuracy consistently fell as the neighbor count k increased from 1 to 125, including tests using a square-root heuristic scaled to the training set size. Ultimately, the results demonstrate that the speedup potential of GPU shared memory is strictly dictated by whether the dataset’s working unit fits inside it.

The Advanced Encryption Standard (AES) is a symmetric-key block cipher adopted by the National Institute of Standards and Technology to secure sensitive data. It processes 128-bit blocks organized as a four-by-four column-major array of bytes known as the state. To encrypt data, AES applies a substitution-permutation network comprising SubBytes, ShiftRows, MixColumns, and AddRoundKey transformations over 10, 12, or 14 rounds for key sizes of 128, 192, and 256 bits respectively. Because these transformations map cleanly onto hardware, AES is an ideal target for hardware acceleration. However, since an accelerator draws power proportional to its data throughput, it also becomes vulnerable to side-channel power analysis. To explore this relationship, this project implements an AES accelerator alongside a ring-oscillator power monitor on a PYNQ-Z2 FPGA.

The encryption and decryption core is an Electronic Codebook (ECB) mode implementation written in C and adapted for the Vitis compiler. A top-level function manages the bus interface, with the specific key size selected at run time via an AXI4-Lite register. Moving data efficiently across the hardware boundary required transitioning from a standard AXI4 interface to an AXI-Stream protocol. Because the Vivado Direct Memory Access (DMA) blocks support only a single streaming input and output, the design merges the key and plaintext into one combined stream, and likewise the ciphertext and decrypted text into a single output stream. Resolving initial system hangs required fixing two distinct communication faults: aligning the 8-bit character payload with 8-bit Numpy DMA buffers instead of 32-bit ones, and moving the TLAST synchronization signal outside of the main streaming loop. Due to DMA transfer limits, the accelerator processes a single 128-bit block per transfer.

Hardware optimization in Vitis HLS relied on carefully applied directives to minimize latency. Loops with static bounds were unrolled, data-dependent loops were pipelined, and the substitution tables were synthesized as dual-port ROMs to prevent read contention. Because several complex loops had variable trip counts, explicit tripcount directives were necessary for the compiler to report latency figures and target specific initiation intervals. These optimizations reduced the core latency from 11,828 cycles to 7,379 cycles. On the physical board, a single block encrypts and decrypts in approximately 1.78 to 1.81 milliseconds across all key sizes. This hardware execution is roughly 6.3 times faster than the unoptimized baseline and 38 times faster than an equivalent Python implementation. However, when compared to the 0.043 to 0.054 milliseconds required by native C software on the board’s ARM processor, the accelerator is slower, highlighting the severe performance penalty of crossing the DMA bus for single-block transfers.

The second half of the project features a side-channel power monitor constructed from ring oscillators. Each oscillator pairs an AND gate with an inverter to generate a fast oscillating signal. Because gate switching delays increase as board power draw rises, counting these oscillations against the unaffected system clock provides an estimate of dynamic power consumption. Since Vitis HLS cannot synthesize gate-level logic, the monitor was developed as a separate SystemVerilog IP and integrated into the design at the RTL level. Forcing Vivado to implement the unclocked logic resulted in a massive initial array that projected a 125°C thermal overload, requiring the design to be scaled down by a factor of one hundred. When enabled on the hardware, the final scaled monitor drew enough power to trip a board-level protection reset. While this prevented live side-channel data collection, it successfully demonstrated the functional mechanism of the power monitor and clearly defined the electrical thresholds of the FPGA.

Developing audio applications for embedded systems often requires physical proximity to the hardware to verify outputs, tying developers to laboratory workstations. AudioJAK resolves this limitation by providing a dual-path audio visualizer and player designed for the Zedboard. The system routes output simultaneously to a local HDMI display and a remote web interface hosted on a companion Raspberry Pi. Beyond simple playback, the application supports dynamic audio operations such as section looping, reverse playback, and audio cutting. Alongside these tools, it renders a real-time waveform, file metadata, and dynamically updated playback cursors to help visualize the audio data.

To ensure playback remains uninterrupted, the local HDMI display operates on a dedicated update thread and utilizes non-blocking drawing calls. It renders the waveform and, if Python is configured on the board, a detailed audio spectrogram. Graphics refresh tasks are distributed across multiple threads, achieving approximately 12.2 frames per second during standard application use and dropping to roughly 10.5 frames per second under heavy, constant updating. While this frame rate is highly functional for visual content without rapid motion, the multithreaded architecture introduces significant CPU overhead. The standalone audio player consumes roughly 4% of the processor time, but total utilization climbs to approximately 150% once the visualization threads are fully engaged.

The remote visualization pipeline relies entirely on a secondary Linux server running on a Raspberry Pi. Analog audio from the Zedboard’s line-out feeds into a USB capture card on the Pi, where FFmpeg continuously encodes the raw pulse-code modulation into libmp3lame HTTP Live Streaming (HLS) segments. Simultaneously, the HDMI framebuffer data travels to the Pi using either a secure copy protocol (scp) transfer with pre-exchanged keys or a distinct HTTP PUT request. A Mongoose HTTP server hosts the frontend index page, which parses the HLS playlist and draws the raw pixel data directly onto a JavaScript canvas. This server is fortified by signal interrupt handlers for clean shutdowns and a systemd service to automatically restore it in the event of a crash.

Measuring network latency end-to-end revealed that remote audio arrives three to five seconds behind the physical board. Framebuffer transport methods also diverged sharply in performance, with scp transfers completing in around three seconds compared to nearly sixty seconds for HTTP curl. The primary bottleneck proved to be the streaming architecture itself. Because HLS relies on sequential file transfers rather than true live streaming, cross-origin resource sharing (CORS) headers and manifest parsing timeouts frequently delayed or stalled playback on the client script. Alternative transports like RTP were tested and streamed correctly to VLC, but struggled severely when integrated into the web page, managing only one video frame every fifteen seconds. These results firmly established that a genuinely low-latency protocol, such as an ICE server paired with WebRTC, is the necessary foundation for future real-time embedded streaming.

The Reduced Instruction Set Computer (RISC) architecture optimizes processor performance by compressing instructions into as few bits as possible, relying on a small, highly efficient set of core commands to execute complex software. A single-cycle implementation of this architecture executes one complete instruction per clock pulse, requiring a precise orchestration of arithmetic logic, memory access, and control flow. This project constructs a complete 16-bit single-cycle RISC-V processor from the ground up using SystemVerilog, bringing the design to life on a TUL PYNQ FPGA board through a rigorous seven-stage integration and verification process.

Construction of the datapath proceeded sequentially, starting with a 16-bit adder featuring an optimized two-term boolean overflow flag. Because overflow depends only on the most significant bits of the operands and the result, exhaustive verification of the flag was reduced to just eight critical input cases. This adder forms the core of an Arithmetic and Logic Unit (ALU) capable of nine distinct operations, including bitwise logic, shifts, and branch-to-zero comparisons. To allow continuous data processing, the ALU is backed by a register file for operand storage and retrieval. Further expanding the processor’s storage capacity, the design integrates a data memory module configured with a 10-bit address space to hold 1024 words of 16 bits. Because data memory reads must be combinational rather than clocked, this storage was implemented using a Distributed RAM IP core.

To convert the static datapath into a functional computer, a dedicated instruction decoder maps twelve distinct machine code instructions onto the ALU opcodes, register addresses, and multiplexer enables. An instruction memory module and a program counter continuously supply these 16-bit instruction words from a loaded memory image, shifting the system from manual toggle operation to autonomous program execution. Finally, rewriting the program counter logic to support conditional branching closed the loop, enabling true control flow and dynamic hardware decision-making.

Hardware bring-up heavily influenced the system’s final architecture. Because the PYNQ board provides only eight physical switches against the roughly thirty-six inputs required by the ALU alone, a Virtual Input/Output (VIO) IP core stands in for the missing pins and actively reports internal state data back to the user. The processor’s ultimate validation was a custom RISC-V assembly program designed to multiply two signed 8-bit numbers via two’s complement logic loops. Assembled into a .coe memory image on a university compute cluster, the machine code was loaded directly into the Distributed Memory core and successfully run to completion on the FPGA.