0
\$\begingroup\$

I am looking for the most efficient solution for transferring data from ~100 individual ESP8266 WiFi modules, that have sensors connected to them, to my Raspberry Pi Pico W.

This will be used for a watering system for an outdoor garden. Each plant will gave its own temperature sensor and water valve.

I would like to constantly retrieve the sensor data from all of the devices, as well as send them information if need be (i.e. watering cycle time). I would like to go wireless as it seems to be the cheapest/easiest option for me, however, I am unsure what would be the right framework for this. I started out with a simple socket connection, but it quickly became a nuisance once all devices needed to make a connection.

I also tested out MQTT, however, the big issue I had was if two sensors sent data at the same time, (which I would like them to be able to do) the Broker may only receive one request. I am looking for a solution that has, possibly, more of a queuing system, such as SMTP.

\$\endgroup\$
5
  • \$\begingroup\$ The 802.11 protocol supports 100+ clients but many AP default configurations won’t allow that many on the same network. Check yours? Spread them out over a few different APs? \$\endgroup\$
    – winny
    Commented Oct 31, 2022 at 15:20
  • 2
    \$\begingroup\$ MQTT is a good choice. Regarding collisions, read about Quality of Service (QoS) levels in MQTT. At levels 1 and 2 the client will resend a message if receipt is not confirmed. One tutorial at this link: hivemq.com/blog/… \$\endgroup\$
    – Mark Leavitt
    Commented Oct 31, 2022 at 17:04
  • \$\begingroup\$ Why a Raspberry Pi Pico W as the master? Anything is possible these days but setting up a server for 100 clients will be hard on a microcontroller. Why not use a Pi4 and run mosquitto, do your watering logic in Python, etc? \$\endgroup\$
    – tomnexus
    Commented Oct 31, 2022 at 17:12
  • \$\begingroup\$ PI4s are great if you can get them. Element-14 has stock arriving in 11 months. \$\endgroup\$
    – Jasen Слава Україні
    Commented Oct 31, 2022 at 22:12
  • 1
    \$\begingroup\$ Vital lesson: don’t have all of your nodes transmit at the same time! Use a pseudorandom algorithm to avoid this. As others have mentioned, a pico w is woefully underpowered for handling 100 wifi connections or even 100 socket connections. Tplink and others have commercial grade access points that are reasonably priced but probably slightly more that a pi4. A pi4 might even have trouble supporting 100 wifi connections as its wifi is not intended to be high performance. MQTT is not an unreasonable choice and should work well with a decent broker - not something on a pico w! \$\endgroup\$
    – Kartman
    Commented Nov 1, 2022 at 11:51

1 Answer 1

Reset to default
2
\$\begingroup\$

Consider using a simple UDP protocol. Your sensors might need constant measurement, but it doesn't need to be that fast: perhaps once a minute per plant? Something like temp 12.2 C water 1 reporting their current state.

At one message per sensor per minute you're at about 2 packets/sec, which should be easily management. As it's UDP, there's no connection to maintain.

You could even use straight Syslog protocol (RFC 5424) as it's text based and that means all your sensor data can go straight into a log.

Commands in the other direction could be as simple as water on and water off, or water 60 (seconds) depending what you want. Having an automatic timeout is good (so it doesn't matter when a packet gets missed).

\$\endgroup\$
4
  • \$\begingroup\$ Udp doesn’t get around the need for a wifi connection and this would be the bottleneck methinks, so the message protocol is secondary. Using MQTT is a no brainer. \$\endgroup\$
    – Kartman
    Commented Nov 1, 2022 at 11:56
  • 1
    \$\begingroup\$ @kartman I did mean UDP over IP over wifi. My understanding was that the problem was the number of simultaneous TCP connections maxing out the TCB limit on the central machine, and UDP would address that perfectly. MQTT is a layer above. It's excellent but normally over TCP so may fall foul of the same problem MQTT-SN would certainly be worth looking into, but I haven't used it myself. \$\endgroup\$
    – jonathanjo
    Commented Nov 1, 2022 at 12:29
  • \$\begingroup\$ Thankyou for your insights, however i feel both these methods will fall short. UDP or MQTT would require me to have some sort of connection between the devices, which is severely limited, especially on an esp8266 (ive seen up to 20 UDP Connections at most) I would have to cycle through connections. \$\endgroup\$
    – drew wood
    Commented Dec 5, 2022 at 21:27
  • \$\begingroup\$ There's not really any such thing as a "connected" UDP link. It most socket libraries, there's a faked "connected" state, which usually just means the IP address and ports are associated with a given socket id. Each slave unit is listening on a port and transmitting on the same on; the central unit ditto. Nothing actually has lots of connections "open". If the Pi Pico thinks it has lots of connections, consider open UDP, transmit, close. \$\endgroup\$
    – jonathanjo
    Commented Dec 5, 2022 at 21:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.