Assessment of Generative AI Applications in Edge Systems

Modern heterogeneous compute systems targeting AI-based workloads, especially in low-power, mobile, and edge domains, typically rely upon a dedicated neural compute unit, or NPU, to process inference. NPU architectures typically consist of a local memory and accelerators targeting general matrix operations, however, recent advances in NPU architectures and their wide adoption are driving increasingly heterogeneous architectures with concurrent and multithreading capability. As a result, NPUs are subject to higher-order software effects and issues such as OS scheduling, CPU orchestration, application mapping, concurrency, access to and management of virtual memory, and so on. It is therefore vital to leverage hardware-software co-design techniques to properly assess the performance throughput of current and emerging generative AI applications.

In order to properly assess the performance of various NPU-enabled architectures as well as enable design space exploration for emerging AI workloads, we aim to develop tools and methodologies for modelling these architectures. During this research internship, you will examine publicly available NPU designs and software support, propose architectural features for exploration, and implement them in simulation. 

Key responsibilities will include:

• Researching and learning about state-of-the-art NPU-enabled systems for edge AI inference.
• Developing cycle-accurate, extensible, and tuneable models and/or analytical methodologies for underlying accelerators and memory.
• Conduct explorations into architectural features around NPU-enabled systems.

This role is ideal for someone who is deeply interested in hardware-software codesign, computer architecture, and working in an interdisciplinary environment that values innovation, creativity, and real-world impact.

Profile: You are analytical and detail-oriented, with a strong interest in system simulation and hardware-software codesign. You are adept at or have a keen interest in programming and performance evaluation tools.

Background: You have or are currently pursuing a degree in computer engineering, computer science, or electrical engineering. Knowledge of object-oriented programming, scripting languages, gem5, and memory simulators is an advantage.

Type of Project: Internship; Thesis

Master's degree: Master of Engineering Technology; Master of Science; Master of Engineering Science

Master program: Computer Science; Electromechanical engineering; Electrotechnics/Electrical Engineering

Duration: 6-9 months

Supervisor: Chris Van Hoof (EE, Nano)

For more information or application, please contact the supervising scientist Joshua Klein (joshua.klein@imec.be). 

Imec allowance will be provided.