ESP32-linked musical input and feedback.
ESP32-linked musical input and feedback.
A multidisciplinary piano-learning system combining connected hardware, a web interface and an augmented-reality learning experience. The project explores ESP32 communication, Angular/TypeScript, Unity, Vuforia and cloud-assisted recognition.
*File metrics count tracked files after excluding common generated build folders. The project status is an editable draft classification.
Capabilities evidenced by project documentation and source structure; some behaviours require platform configuration.
ESP32-linked musical input and feedback.
Angular/TypeScript application for interactive practice.
Unity/Vuforia augmented-reality training components.
Exploratory music recognition and connected services.
Technical architecture drawn from the repository and supplied documentation.
The project uses C++, ESP32, Angular, TypeScript as its documented technical foundation.
The source tree and supplied README describe the primary modules; individual files and runtime behaviour may vary by platform.
Live ESP32 serial input driving a Unity AR piano
void Update() {
if (serial == null || !serial.IsOpen || piano == null) return;
try {
while (serial.BytesToRead > 0) {
string raw = serial.ReadLine();
string note = Normalize(raw);
if (string.IsNullOrEmpty(note)) continue;
piano.SimulatePressByName(note);
}
} catch (TimeoutException) { }
}
The cloud assistant creates a thread, submits the user message, launches a run and polls for a finished response. Secret validation occurs before the client is created.
AssistantThread thread = await _assistantsClient.CreateThreadAsync();
await _assistantsClient.CreateMessageAsync(thread.Id, MessageRole.User, message);
ThreadRun run = await _assistantsClient.CreateRunAsync(thread.Id, new CreateRunOptions(_id));
do {
run = await _assistantsClient.GetRunAsync(thread.Id, run.Id);
await Task.Delay(TimeSpan.FromMilliseconds(_DELAY_IN_MS));
} while (run.Status == RunStatus.Queued || run.Status == RunStatus.InProgress);
Screenshots from supplied documentation and repository media. Missing visuals remain placeholders rather than invented screens.
To fill this space, add an approved screenshot with the exact filename shown below. The site currently displays a neutral placeholder.
assets/img/projects/interactive-piano-hci/screenshot-01.webp
Technology overview: C++, ESP32, Angular, TypeScript, Unity, Vuforia, Azure. The list reflects documented tools, dependencies and proposed components; for planned projects it does not prove implementation.
File types identified: 27 .cs, 19 .json, 10 .ts, 10 .c
Explore the public repository, source code and project documentation on GitHub.