It is just a little over a year since the first Village Telco workshop in June 2008 and having recently completed second Village Telco workshop from the 20th to the 24th of July, it has taken a little longer than I expected to get this post out about the event. Similar to the first workshop we managed to assemble a remarkable group of people. Many from last year and some new faces as well. A notable addition this year was Sigqibo Pangabantu of Silulo Technologies. Alan Levine and I met Sigqibo last year when we went looking for likely Village Telco entrepreneurs in Khayelitsha. With his business partners, Sigqibo runs a cyber cafe, an ICT training centre, and computer shop. Alan and I chatted with him last year about the Village Telco but there wasn’t much to show him at the time. This year, we have something quite tangible, the Mesh Potato. Another notable addition was Edwin Chen of Atcom. Edwin is the technical sales manager for Atcom, who are manufacturing the Mesh Potato. Having Edwin at the workshop was both a great sign of support from Atcom but also invaluable in terms of being able to access practical manufacturing knowledge as we debated the finer points of the Mesh Potato design. Finally, in stead of just visiting Antoine van Gelder and David Carman in Scarborough, as we did last year, where they run a 250 node mesh network, it was great to have that at the workshop helping us work through design challenges. Antoine is the principal author of the Afrimesh software project which will be running on all the Mesh Potatoes.
Last year a bold and creative group of geeks conceived of an Open Hardware / Open Software device that would both simplify and lower the cost of deploying wireless mesh voice networks. We called it the Mesh Potato which is a mashup of Mesh + POTS + ATA. Well , it works in Spanish at least. A year later we had working Mesh Potato prototypes to hack at the workshop. And that is what the workshop focused on, getting the Mesh Potato into production and helping it go to scale. The rest of this post is a summary of the key elements of the workshop.
Technical Design Trade-offs
One of the first conversations we had at the workshop was to review the technical design decisions to date and discuss the pros and cons of each decision. From the beginning we have tried to balance a desire to make the Mesh Potato as hard-to-break (or brick for that matter) as possible with a need to keep it affordable.
Antenna Type
Based on her experience in Bangladesh, Elektra proposed the more robust N-type antenna for the Mesh Potato as opposed to the smaller but less rugged Reverse-SMA antenna typically found on low cost wireless APs. Because the N-type was more expensive there was some discussion as to which was the best option. This debate was rendered somewhat moot when we realised that we could both save money and increase robustness by going with an internal antenna etched on the motherboard of the Mesh Potato. That was a big design step for us as it rule out the flexibility of attaching different types of antennae to the device but it seemed like a reasonable trade-off. Even now though a connector for an external antenna may find its way onto the motherboard somewhere. 🙂
FXS Interface
In the design of the Mesh Potato, we have opted for a compromise of speed in getting the Mesh Potato to market and optimal cost-effective design. David pointed out in the workshop that with another month or so of design work, he would be able to bring the cost of the FXS module by over 50%. Clearly, this is something to prioritise for the next generation of Mesh Potatoes in terms of producing a very affordable device.
AC Adaptor
A surprise for me in the discussions was recognising what a weak point the AC adaptors are in the context of the whole system. We have gone the extra mile to harden the ports on the Mesh Potatoes to handle variable power sources, lightning strikes, etc but we haven’t discussed the AC adaptors at all. These are obviously the first things to go when plugged into an unstable or incorrect power source. While designing a custom AC adaptor would possibly turn out to be too expensive, it is possible that we will be able to get a higher rated power supply for the Mesh Potato. David and Edwin are looking into this.
Power Supply Chipset
We had a good discussion about the trade-offs of between a high quality power supply chipset which would give us a 10% savings on the power consumption versus the cost of the chipset which was almost an order of magnitude more expensive than the cheaper option. One insight for me was that the big win on power for the Mesh Potato is simply that of combining an AP and an ATA in a single device. That halves the typical power consumption right there. A 10% savings on top of that is desirable but has to be balanced against overall cost. There was also a discuss about handling unexpected high voltages such as from an unregulated solar panel and here I have to defer to David and Elektra to chime in as it was beyond my modest capacity to fully grok. The bottom line is that we are opting for the cheaper power chipset but some additional design is required for this chipset to cope with the high voltage scenario.
PoE and PoTL Injectors
The discussion around PoE (Power of Ethernet) an PoTL (Power over Telephone Line) injectors was pretty wide ranging. We’d like to create the most flexible scenario possible for the Mesh Potatoes so that they can be powered by standard power cable or PoE using a single ethernet cable or PoTL using a standard telephone line. We even discussed putting power and ethernet and telephone line through a single cable but the consensus at the end was to be as standard as possible to avoid misconfiguration. PoTL in particular is a little hazardous because of the comparatively high voltage going over the phone wires.
Rael had a very innovative solution in which the Mesh Potato ports could be snapped on or off the Mesh Potato and extended via PoE. This would make the Mesh Potato ideal for indoor or outdoor installation but we decided that it was going to add a level of complexity to the design that would slow getting the MPs to market. Something to look at seriously for the next iteration though.
Case Design
Another significant topic of discussion was the case design. We all agreed that we wanted the following:
- a water resistant case that can withstand a rain storm;
- UV resistant plastic that will work for years under an African sun;
- white or beige case to keep the device as cool as possible;
- ports should be covered inside the device;
- an LED display to aid configuration; and,
- fittings for both pole and wall mountings as well as possibly a desktop mounting.
In the end it transpired that what we were talking about would look remarkably like an Ubiquiti Nanostation Loco. Hopefully we may even improve on Ubiquiti’s mounting bracket design. After seeing some Vodacom payphone units in Khayelitsha we also discussed possibly replacing the LEDs with an LCD display that would open up possibilities for innovation in terms of debugging, configuration, payment status, etc. We’re still investigating this possibility.
Software Development
I’ll break the software discussion down into the Mesh Potato, Afrimesh on the Mesh Potato, Server Configuration, and Miscellaneous. Needless to say the conversation was not that structured. 🙂
Mesh Potato Initialisation
We all agreed that to the extent it is possible, we’d like the Mesh Potatoes to work as a telephone network right out of the box. This applies both to the scenario where they are working in peer mode i.e. not connected to an upstream server and when they are connected to an upstream asterisk server which could be on the local network, at an ISP, or even in the cloud. Key to making that work is to ensure that each Mesh Potato that comes off the production line has a unique IP address. Type wireless AP devices have a standard default IP address. This would not work for us because Mesh Potatoes would then need to be individually configured by the user. Thus, each MP needs to begin its life with a unique IP address. Jeff Wishnie proposed a similar strategy to that used by Meraki which is to generate an IP address from a hash of the MAC address of the devices. We decided that each MP should not only have a unique IP address but also a unique phone number which would be derived from the last 9 digits of the IP address. We even came up with an unused country code (+288) which might be used to give the Mesh Potatoes a fully addressable unique DID at some point in the future. More about this on the wiki.
We also established that it would be desirable to be to switch any Mesh Potato into a ” voice master” mode so that other Mesh Potatoes on the local mesh would automatically recognise a configured Asterisk server on the network. Elektra suggested that this could be achieved by having the “voice master” node announce a voice gateway class to the network in a very similar manner to how the mesh network announces upstream gateways via a class. [Need a link to more info on this]
EasyTAG UI
Finally, because the Mesh Potatoes will have effectively random IP addresses out of the box, which will allow them to work as a network from power-on, the Village Telco entrepreneur is likely to want tools for mass configuration of the network, e.g. putting all the nodes on a single subnet, re-numbering the phone numbers, etc. The whole mass-reconfiguration thing made me think of my music collection and the elegant simplicity of tools like EasyTag which allow you to batch edit an individual attribute of a single mp3 file or a thousand. Having an EasyTag-like editor to edit the attributes of the Mesh Potatoes on your network would make network configuration a breeze.
ToDo: Develop all of the above.
Afrimesh
Over the course of the workshop, Antoine managed to get his very elegant Afrimesh software running on the Mesh Potatoes. He has posted some instructions on how to install Afrimesh on a fresh OpenWRT install. Afrimesh on the Mesh Potato will serve three core functions.
- Default GUI: Afrimesh will be the default web GUI for the Mesh Potato. If you plug your laptop into the Mesh Potato via the ethernet port and point your browser at the MP, Afrimesh is what you should see. Any other software should be accessible via Afrimesh.
- Network performance: Afrimesh provides the user with a dynamic map which shows the active mesh nodes on the network and their relative health.
- Basic configuration: Afrimesh should allow the user to configure basic things like network name, IP address, etc. For more sophisticated configuration, Afrimesh provides a link to Luci which can open a Pandora’s box of configuration options for the more intrepid user.
ToDo:
- Develop an Afrimesh package is easily installable on OpenWRT and which is also part of a Mesh Potato “target” firmware for OpenWRT
- Develop the network monitoring aspect of the Afrimesh software to include visual monitoring of network performance over time
Interactive Voice Response (IVR) Configuration
Naturally we don’t wnat to just have one way to configure the Mesh Potatoes, there should be several. In particular, the Mesh Potatoes should be configurable without a web interface i.e. just by picking up a handset. David made some initial progress in setting up IVR to be able to configure the IP address of the Mesh Potatoes, even working in a 2001 Space Odyssey clip to catch the attention…. “What are you doing, Dave?” The goal is ultimately to have a suite of commands that could be issued via the phone keypad to be able to both diagnose and configure the Mesh Potatoes.
Default Asterisk Configuration
In order to make it as easy as possible for people testing out the Mesh Potatoes to set up their own VoIP networks, we need to develop some basic Asterisk configuration instructions, files, snippets and even bootable ISOs for Mesh Potato networks. Work on that has begun on the VT wiki.
Points of Pain
It seems worthwhile ending off this workshop summary with a review of a sessions we called “points of pain”. That is, what do we need to work on on the Mesh Potato and the Village Telco that will have the biggest impact on both the ease and cost of installing an MP network. Here are what we came with:
- From a cost perspective “ease of installation” as just important as driving down the purchase price of the Mesh Potato. A Mesh Potato that is difficult or simply complex to install can add as much as USD 50 to the total cost of the device. A reminder to make the Mesh Potato blindingly easy to install, monitor, and debug.
- Less to do translates into lower error rates. Pre-configure as much as possible and anticipate as many scenarios as possible.
- Investment in software affects the hardware cost. In this case we are talking about things like have David re-design the FXS interface. In the spirit of hardware is the new software, smarter software/firmware can allow us to use less expensive (but not lower quality) components in the Mesh Potato.
- Scalability is critical. What does an MP network look like at 10, 100, 100, 100 000 devices? All software and design choices need to scale.
- Distinguish between technical infrastructure and payment infrastructure (I actually can’t remember exactly what we meant by this… other than the obvious)
- Protection is key. Ensure that the Mesh Potatoes are as invulnerable as possible to natural (accidents, lightning, surges, brown-outs, etc) and un-natural (misconfigurations, theft, etc) events.
That’s about as much as I am able to capture from the event. I hope participants will chip in the comments to flesh out and/or correct memory blips on my part in the post. It was amazing (again) having such creative and brilliant minds crafting a future for the Mesh Potato and the Village Telco.
-Steve