A smartphone-based collective display system aims to synchronize light effects across devices using Morse code blinking, overcoming challenges in network conditions, positioning, and compatibility, with real-world tests showing 14-15 meter detection ranges. The project highlights AI limitations in vision tasks, stresses observability through real-world telemetry data, and advocates open-source development combining technical rigor with iterative human-driven debugging.

Into The MoTaverse

Rosie Sherry interviews people involved in testing. Video only interviews. Available on youtube or the homepage. Each episode has a full transcript if you find it on the main site.

• https://www.ministryoftesting.com/podcasts/into-the-motaverse
• https://www.youtube.com/playlist?list=PLbdLjg29s9lCY4hspzj3AGdAL7Vr2ys1B

Episode Details

• Show Notes: https://www.youtube.com/watch?v=e2Erxa8_xMw
• Published: 2026-06-17T15:28:15Z
• Duration: 00:50:29
• Author: MoTaverse

Overview

The podcast discusses various technical projects and challenges, focusing on the development of a smartphone-based collective display system designed to replicate concert-style light effects using individual devices. Key challenges include synchronizing animations across devices in varying network conditions, determining crowd positioning for cohesive visuals, and ensuring compatibility across different phone models and environments. The system initially used barcodes for tracking user positions but evolved to Morse code-based light blinking, which improved range and accuracy. Testing in real-world scenarios, such as a Manchester meetup, demonstrated successful detection of users up to 14-15 meters apart, with potential for even greater ranges under optimal conditions. The project emphasizes low-cost, accessible solutions leveraging widely available technology and real-time spatial mapping for applications like crowd-based visual effects.

The discussion also highlights limitations in AI tools like ChatGPT and Claude, particularly their struggles with vision tasks and high costs for image analysis, which can hinder practical applications. A critical focus is on the importance of observabilitycollecting detailed telemetry data (e.g., pixel-level camera feeds, environmental metrics) to debug systems and improve AI reasoning. This includes using real-world data from test events as benchmarks, iterating on code changes, and implementing guardrails to prevent unintended modifications. The challenges of AIs limited understanding of physical contexts, such as lighting conditions or human movement, are emphasized, underscoring the need for human-driven debugging and iterative testing over relying solely on prompts or synthetic data. The project also explores the shift from prompt-centric AI design to data-informed approaches, advocating for systems that prioritize context and real-world adaptability.

Additionally, the discussion addresses broader implications for AI development, including the need for ethical considerations in training data and the balance between technical innovation and societal responsibility. Practical challenges, such as detecting devices in low-light environments or accounting for environmental variables like camera interference, are detailed alongside strategies like storing raw pixel data for cost-effective analysis. The project aims to open-source its code after refinement, reflecting a commitment to sharing insights for community-driven development. Ultimately, the narrative underscores the value of combining technical rigor with human input, emphasizing that real-world validation and iterative problem-solving are essential for refining systems that operate in dynamic, unpredictable environments.

What If

• What if you optimized your smartphone-based collective display system by leveraging environmental sensors for real-time adjustments?

  • Move: Integrate environmental sensors (e.g., light meters, proximity sensors) into your Morse code detection system to dynamically adjust brightness and focus parameters based on venue conditions.
  • Why Now?: The Manchester event demonstrated extended range capabilities, but low-light conditions and ambient interference remain barriers. Real-time adaptation could mitigate these issues.
  • Expected Upside: Enables reliable performance in diverse venues, reducing manual calibration and improving user experience during live events.

• What if you prioritized raw pixel data over AI-driven image analysis to cut costs and improve debug reliability?

  • Move: Build a streamlined data pipeline to store and process raw pixel feeds from cameras, bypassing AI tools like Claude for initial diagnostics.
  • Why Now?: The text highlights the high cost of AI image analysis (e.g., $100 for 25 minutes) and the lack of AIs ability to rationalize physical issues. Raw data is cheaper and provides higher fidelity.
  • Expected Upside: Cuts AI costs by 70%+ while enabling deeper debugging, such as identifying why a phones light pattern failed to decode.

• What if you used the Manchester events recorded data as a unit test suite for future code changes?

  • Move: Create a version-controlled repository of the Manchester events debug data (pixel captures, JSON maps) and automate test runs against new code updates.
  • Why Now?: The text emphasizes the value of real-world data over synthetic tests, and the Manchester data already includes edge cases like device tilt and lighting variability.
  • Expected Upside: Ensures new code changes dont break existing functionality, reducing post-deployment bugs and accelerating iteration cycles.

Takeaway

• Implement observability systems early: Build telemetry logging, video capture, and pixel-level data recording into your project from the start to enable post-event debugging, troubleshoot AI limitations, and refine detection logic iteratively.
• Leverage widely available technology for scalability: Use smartphones as core components for low-cost, scalable solutions (e.g., light displays) instead of relying on specialized hardware, reducing barriers to entry and deployment.
• Test with real-world edge cases: Create test scenarios that simulate diverse environmental conditions (e.g., low light, screen orientation, crowd movement) and use recorded real-event data (like the Manchester meetup) to validate system reliability and adaptability.
• Limit AI use for expensive vision tasks: Avoid using AI tools like Claude for image analysis due to high token costs (e.g., analyzing large images). Instead, prioritize raw pixel data storage and manual debugging for vision-related diagnostics.
• Adopt iterative testing with real-world data: Treat real-world events or user data (e.g., recorded phone blinking patterns from live tests) as unit tests to validate code changes, ensuring your system works in unpredictable, physical environments.