So the challenge, if there is one, is to drive down the cost to such a level that they can become widespread and almost, disposable.
This also strikes me as the kind of challenge that would react well to an open competition, run by DSTL or one of our many esteemed universities, like a DARPA Grand Challenge.
For any sensor system there are fundamental building blocks, sensor, control, power and communications.
Once they have been perfected, the packaging and form factor needs to be decided.
For an unattended system they are all challenging, for a persistent sensor system, power is probably the single biggest problem.
If we are to make them as ubiquitous as small arms they have to be cheap, to make them cheap they either have to be made in vast quantities or exploit off the shelf technology.
That means a unit price of less than £500
The most obvious type of sensor is an optical one i.e. a camera. Fixed focus or zoom, daylight or infra-red, given the huge development activity with mobile telephones, the capabilities of modern small lens camera modules are simply out of this world compared the hedgerow cam at the top of this page.
Full HD CMOS camera modules are readily available in retail quantities are available for less than £5, VGA sensors, even less.
In the smartphone business, there are a number of key suppliers; Sony, Omnivision, Toshiba, SK Hynix, Samsung, LG and HTC. These manufacturers can supply camera and lens modules that incorporate image stabilisation, optical zoom, autofocus and infra-red colour correction. Wider aperture systems are also becoming more readily available for significant improvements in low lighting performance.
Toshiba produced the record breaking 41 megapixel sensor for the Nokia 808 and also market a number of 8 and 21 megapixel sensors for the Project Ara modular smartphone. As the burgeoning Indian and Chinese smartphone market increases demand and volume, companies like OmniVision are selling 16 megapixel sensors that can capture 30fps HD video for less than a few US Dollars.
Infra-red illuminators would allow usage in night time conditions but an emitter is not a great idea in a passive surveillance system for both detectability and power reasons so the best solution would be to utilise low light models and rely on ambient illumination sources, or simply accept a compromise.
The mobile phone market is also driving innovation here, using twin lens modules and in module noise reduction for example.
Depth of field is reduced but for some applications, this may be acceptable. What is certain though is that small optical sensors from the smartphone market are increasingly capable and power efficient.
A typical overcast day is between 1,000 lux and 20,000 lux intensity, a full moon is less than 1 lux and a quarter moon at 0.01 lux. Low light cameras are generally classified as requiring illumination between 0.1 Lux and 0.01 Lux. Moonlight level cameras between 0.1 lux and 0.002 lux, ultra-low light cameras can operate at less than 0.001 lux
As an example, JVT produce a 2 megapixel module that is 38mm square and can operate in colour at 0.001 lux. It can encode and compress the 1920×1080 image using an onboard H.264/H.265 system. Accept a higher lux threshold and the resolution can be improved, 2560×1920 for example, at 0.05 lux, in colour.
Spend a bit more, get a bit more, but the mass market devices at less than $50 are extremely capable.
Although a day or low light optical sensor might seem like the obvious choice, seismic or even audio sensors could also be considered.
In addition, a signals intelligence payload might also replace the optical sensor.
Software defined radio has its own open source software and hardware community and they do often quite incredible things with low cost TV tuner cards, single board computers and software. SDR simply moves signal processing into software.
SDR defined by Wikipedia
VHF radios work in the 30-300 MHz frequency range. Cellular telephones, depending a number of factors, operate between 850 MHz and 1,900 MHz. WiFi operates between 2.4 GHz and 5.9 GHz, whilst Bluetooth operates at 2.4 GHz. GPS operates at 1.57 MHz and 1.22 MHz.
This presents many challenges as different hardware will be required but the aforementioned $20 TV Tuner card can be used to listen across a 50 to 1,750 MHz frequency range which puts many communication systems in range.
Spend $300 and the open source design HackRF One transceiver is available, this has an incredible 10Mhz to 6Ghz operating frequency. Sample rates are also extremely high.
There are even App Stores for SDR.
Another signals intelligence capability might be derived from proximity advertising and the fast growing ‘smart home’ market.
Proximity advertising is a technique that presents targeted advertising initiated by the presence of a device. When the device approaches it is detected and either a large screen display started or some media ‘sent’ to the device, multimedia files for example.
In ‘Smart Home’ applications the key much functionality is detecting when a specific person enters a specific room or building in order to initiate other things, lighting or door opening for example. Each Bluetooth device has a unique MAC address which is used for proxy identification i.e. if this MAC address is detected this person is detected.
The technology used for both these application is Bluetooth, the well-known short range personal network system. Detection for range for 2.5 GHz Bluetooth signals can be as high as 250 metres, although operating ranges are much smaller, 25 metres is more realistic.
Although 25 metres may seem like a small detection range it might be enough when used at specific choke points.
There are over two billion Bluetooth devices in circulation.
Put a detector close to the entrance of an Army barracks and in peace time, collect all the Bluetooth MAC addresses and build a database. As we have seen from the various social media posts of Russian soldiers in Ukraine maintaining operational security over soldier’s use of Smart Phones is actually quite difficult.
Now position the detector at a critical bridge and one could easily get warning of concentrations of MAC addresses (and therefore units) in space and time for real time intelligence of troop movements.
Bluetooth is not just for Smart Phones either, keyboards, headphones and even fitness trackers are transmitters.
Similar can also be achieved with GSM/3G telephones, hoovering up identification data for future location and correlation.
Cross curing one sensor to or from another has enormous potential.
With a signals payload able to show a concentration of a specific number of MAC addresses from soldiers Smart Phones at a key bridge, positive verification could be obtained by cross cueing an optical sensor at the same location.
The signals payload provides a clue or marker and then the optical sensor provides both confirmation and positive evidence.
Vehicle identification and quantities could confirmed automatic number plate or shape recognition.
In real time, early indications of massing or movement across specific locations could allow counter targeting.
If a network of receivers was available, they could also be used for triangulation of more simple communication devices like VHF ‘walkie talkies’
Control and Processing
After collection of visual or electronic information, something has to be done with it.
The data could be transmitted raw but that would likely come at a cost of unsustainable bandwidth and power issues and so some filtering or processing would need to be conducted on-board the device.
Smart Phones and operating systems like Android would make a quick and relatively easy base on which to generate such processing but as am sure most of us know, they are hardly frugal when it comes to power consumption and for an unattended sensor, this is a bad thing TM
A more realistic alternative, and one which pushes the cost right down, is the many single board computers now available.
The British designed and built Raspberry Pi is an obvious choice but even this might be over-specified, the ridiculously cheap Raspberry Pi Zero might be just as suitable. At a whopping £4 (you read that correctly) the Pi Zero has a 1GHz processor, 512Mb RAM and comes on a package 65mm by 30mm.
There are a huge variety of readymade applications and peripherals for the Pi family including cameras, as shown below on the Pi Zero
Whether it is signals analysis, number plate recognition, scene change detection or storage and compression of images ready for transmission, it is in the software load that the real power of such a device will be realised.
Software should also enable the tidal wave of data to be tamed, this is where the sensor cross cueing and target recognition would be advantageous.
Researchers at Georgia Tech’s School of Electrical and Computer Engineering have approached this task by limiting frame rates and combining this with an intelligent control system that recognises images through a novel pixel tracking algorithm
As we know, with imagination the art of possible is vast.
As with sensors, the idea is to use readily and commercially available components.
Although collection for later retrieval might be of value in some circumstances there is a fundamental requirement to transmit collected sensor information to users.
Communications could be continuous, on a store and forward basis, on a time lapse or in response to some stimulus. The stimulus could be a command signal or other sensor input, movement, seismic or acoustic for example.
Instead of transmitting raw data from whatever sensor is used, the actual data transmitted should be processed, compressed and transmitted sparingly.
For video, compression technology means even full motion high definition video can be transmitted at relatively low bandwidths. The latest HEVC/H.265 standard is aimed at 4k video Raw 4K video needs more than 60Mbs but with H.265, this can be reduced to approximately 20Mbs. Proprietary technology like Beamr also claims to reduce this even further, down to 10Mbs. 1080p video compressed using the older H.264 compression technology consumes roughly 6Mbs.
This is still very high for wireless transmission though and higher efficiency codecs usually require higher power computing, and in turn, this generates high power consumption and greater heat.
If we go much lower, VGA (640×480 pixels), but still use H.265 the bandwidth requirement drops to approximately 0.5Mbs, or double that with H.264.
We would also have to question the need for full motion colour video, reducing colour depth and frame rate provides a massive reduction in bandwidth requirement. Dropping to 1 frame per second at 352×240 would need 6Kbs, 10 frame per second at 640×480, 180Kbs.
As can be seen, transmitting at high definition and high frame rates is probably not practicable for this kind of device but accepting frame rate, colour depth and resolution compromises certainly makes it much more feasible.
A balance would need to be found.
Means of communication…
Although satellite would be excluded on cost grounds, it could be seen as an option.
The Low Profile BGAN terminal from Inmarsat is specifically designed for covert communications, it is state of the art with a state of the art price tag, about $20,000 each.
Offering bandwidths of up to approximately 0.5mbs it could comfortably handle video streaming, has a wake on SMS function, can be laid flat and even covered in a thin layer of soil or sand.
The most obvious means of communication for this type of device is to use cellular networks. Many studies have shown that cellular networks have remained remarkably resilient in conflict situations short of major wars with extremely high levels of infrastructure damage.
It might simply be an acceptable compromise to tolerate the risk of the device being disrupted in return for low cost and relatively low power operation. ‘Machine 2 Machine’, or M2M, terminals are sold in high volumes as everything from vending machines to street signs are increasingly connected.
Yes they would be much less secure than satellite but there is also an advantage in hiding in plain sight.
LTE modules can now be purchased for less than $50
Once connected the data would be transmitted over the cellular network and onto the internet for viewing, collection and analysis.
In built up areas there are likely to be a large number of WiFi routers and hotspots that can be accessed automatically, using automated password cracking where required.
A high gain WiFi antenna does not need to be large and can access WiFi signals from a broad area, hopping from to another to avoid detection and improve diversity.
High Altitude Platform (HAP)
One area that does show enormous potential for low power internet connectivity is the high altitude platform concept. Although this is not new, it is looking increasingly likely with Facebook and Google both investing large sums in the technology needed to realise it. Google Project Loon and Facebook unmanned teams are both collaborating and competing at the same time, but whatever the final outcome, the technology is progressing at pace.
With the Airlander 10 soon to make its debut in the UK and the Airbus Zephyr under contract from the MoD, there are likely to be a number of options for generating high altitude internet connectivity.
WiMAX like bandwidth (approx. 10Mbs) to multiple devices over a 400km2 plus area seems to be the current aim point which would provide more than enough coverage for multiple devices.
Low Power Radio Networks with Backhaul
Once concept that could also be promising in situations where sensor placement would be relatively dense is to generate an ad-hoc mesh network between the devices and then use a single communications hub for backhaul transmission.
Multiple devices connected over low cost and low power radio networks to a single satellite communication uplink would enable the high cost of the satellite equipment to be shared across multiple devices.
As the ‘Internet of Things’ and smart metering progresses, industry is producing a number of Low Power Wide Area Network (LPWAN) technologies.
Although they tend to operate at lower bandwidths they may still be enough and are attractive because of their operating ranges, most over 10km for example, and low power/cost.
Power consumption is very important, the Pi Zero (at idle and with HDMI disabled) sips power at a rate of 0.7 Watts per hour (120 mA). If more power is needed for sensor processing (as I think it would) then one of the more powerful single board computers would be needed. A Pi 2 Model B is much more powerful than the Zero but still only consumes 1.2 Watts per hour (240 mA) at 5 Volts.
This calculation does not take into account any communication transmission, this is likely to be much greater (although the Pi figures above do include WiFi). Real time continuous transmission will of course require much more power than transmit on sensor cue, command or on a timed basis.
Each application would need to balance communication frequency with power available.
A 100,000 mA ‘telephone power bank’ readily available on a certain global auction site for £15 could power a Raspberry Pi for about 10-12 days. (calculator here) Connect multiple such cheap power banks and a reasonable battery life could be generated. After use, the device could be retrieved, the battery changed covertly, or, simply abandoned. Modern power tool batteries are also increasingly efficient and offer significant capacity.
5 to 10 days may be suitable for some scenarios but for persistent surveillance, more would be needed.
Widely available, solar chargers could provide and enduring power system for sensors but they are not always discrete.
Careful placement of the collection panels would be needed but given the potentially small amount f power needed, this could be easier than imagines, especially in urban settings. Collection panels can also be separated from the sensor and associated electronics, they do not need to be in one large box.
Mass market solar phone chargers that output enough power to charge a battery bank suitable for the example processor above can be obtained for £20-30 and are no larger than a couple of smartphones.
Where sunlight cannot be guaranteed and extended durations are needed, a fuel cell might be an effective means of generating sufficient power.
The types shown above are very high power and very high cost but yet again, turning to the mass market reveals some interesting devices, mostly aimed at recharging cell phones. Off grid power systems for CCTV, industrial process control and lighting are also becoming more widespread, although they are still relatively expensive and bulky.
Mains and Parasitic Power
In urban or even semi urban environments, it Is probably just as easy to plug it into a mains socket
This might not be as ridiculous as it seems, again, in many situations, mains power remains operable and one more device plugged in to a house or industrial buildings power cabling might be just as stealthy as if it were hidden and operating on battery power.
The hedgerow cam at the top of this page was fitted into a metal cylinder but that does not mean all similar devices have to be the same.
Put the sensor, processor, power and communications system into a black Peli case and it looks exactly what it is.
Split the four modules or put them into common objects that do not look like Peli cases and they become much easier to conceal. In an urban environment it should be relatively easy to conceal the sensor given it is likely to be no larger than a box of matches.
The principle is to hide in plain sight so attaching boxes to street furniture, bridge supports, cell towers and signage is unlikely to arouse any suspicion as they are already adorned in similar. Think creatively and match those boxes to the intended environment; rubbish containers, rubble, agricultural equipment, abandoned cars, shop signs, drainage pipes, street lights and advertising signs.