Skip to content

Connectivity Drop with More Than 5 Devices in Micro-ROS Agent on Docker (Raspberry Pi 5) #233

Open

Description

@SpaceBod

Description:
I am running a micro-ROS agent in a Docker container using the prebuilt image (humble) on a Raspberry Pi 5. I have several ESP32 devices with micro-ROS (Humble). When I attempt to connect more than five devices to the agent, I experience connectivity dropouts, where some devices fail to stay connected. With 5 devices, they are successful in subscribing to topics.

I suspect this issue might be related to the configuration flags in the micro-ROS build, particularly those related to client and resource limits. I came across the following parameters:

  • RMW_UXRCE_MAX_CLIENTS
  • RMW_UXRCE_MAX_SUBSCRIPTIONS
  • RMW_UXRCE_MAX_SESSIONS
  • RMW_UXRCE_MAX_NODES
  • RMW_UXRCE_MAX_PUBLISHERS
  • RMW_UXRCE_MAX_SERVICES
  • RMW_UXRCE_MAX_WAIT_SETS
  • RMW_UXRCE_MAX_GUARD_CONDITION
  • RMW_UXRCE_MAX_TOPICS

These parameters seem critical to configuring the agent, especially as I'm dealing with multiple devices where each subscribes to three topics. I suspect increasing the RMW_UXRCE_MAX_CLIENTS value might help, but I'm not entirely sure.

What I've Tried:

  • I attempted to experiment by manually setting RMW_UXRCE_MAX_CLIENTS to 1 (as a test), but it did not yield the expected results, as multiple clients could still connect.
  • I've tried searching through the documentation, including Vulcanexus for guidance, but I haven't been able to solve the issue.

Question:
In order to adjust these flags and rebuild the micro-ROS agent from source, do I need to modify the build configuration in the following file?

Or should the changes be made in this file instead?

I believe the latter might be where the flags should be set, but I'm uncertain. Has anyone else experienced similar connectivity issues when connecting more than five devices to a micro-ROS agent on Docker? If so, how did you resolve it?

(I've also noticed a colcon.meta file in the /src folder of my workspace, should the be used?)

Any guidance on where exactly to set the flags for a manual source installation would be greatly appreciated - Or even the correct approach to setting up an agent to allow for more than 20 clients!

Thanks in advance for your help!

System Details:

  • Hardware: Raspberry Pi 5
  • Docker Image: Prebuilt micro-ROS agent
  • Micro-ROS Version: Humble on ESP32 devices
  • Number of Devices: 5+ ESP32 devices
  • Topics per Device: 3 Topics per device

Links to Documentation:

Thank you!

Activity

  1. simonbogh commented on Jan 26, 2025

    @simonbogh

    Do you connect your devices over serial or udp/tcp?

  2. SpaceBod commented on Jan 28, 2025

    @SpaceBod
    Author

    We are using UDP/TCP.

  3. SpaceBod commented on Jun 17, 2025

    @SpaceBod
    Author

    Update, we have tried to use serial running on one agent via multi-serial:

    • When implementing a reconnection mechanism, issues emerged:
      • Devices pinged the Agent to verify liveliness before initiating reconnection.
    • Latency scaled poorly with device count:
      • At low device counts (≤3), latency was stable (e.g., 9 ms).
      • As device count increased (≥4), ping responses fluctuated dramatically (up to 600+ ms).
    • Even with a 3-second timeout window, devices incorrectly assumed disconnection, triggering unnecessary reconnects.
    • Reconnect attempts would conflict, breaking otherwise healthy connections and destabilising the system.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions