Combining RFID and video surveillance

The need in major airports to move a large number of people through a complex environment as efficiently and safely as possible is becoming an increasingly challenging task. The volume of flights is growing along with the size of aircraft, and security has, in recent years, become a critical issue. With this in mind, the EU-funded Optag project was launched in 2004 to study the feasibility of a combined RFID tag passenger location and video surveillance system to enhance airport efficiency, safety and security.

The basic concept, illustrated in Figure 1, is for airports to be fitted with a network of combined RFID tag readers and high-resolution panoramic cameras, spaced at around 15-20m intervals, which are used to monitor the movements of people around the terminal building or buildings. Each passenger carries or wears an RFID tag, which can allow location to an accuracy of around 1m, and the video and tag data merged to give a very powerful surveillance capability with a wide range of potential benefits.

The envisaged system

Figure 2 shows a possible implementation of the Optag system. Passengers are issued with an RFID tag, possibly at check-in, which they carry throughout their time at the airport until arrival at the departure gate or even on boarding the aircraft. A cellular network of combined tag readers and panoramic cameras is then able to locate each and every tag with some precision and help ensure that people are in the right place at the right time. The tags currently under development at UCL are transmit-only devices that do not store any data but emit a beacon with a unique ID at frequent, randomised intervals, at least once per second, and this is cross-referenced to passenger information already stored on the system – such as name, flight number and perhaps, looking to the future, biometric data. This gives the effect of intelligence in the tags – passenger information can appear to be ‘read’ from them though it actually resides on the computer system – and with relatively simple, inexpensive and compact tag electronics, ideally cheap enough to be disposable. In addition, the fact that the tags contain no data should help to allay any privacy concerns. The tags and reader infrastructure allow convenient monitoring of passenger flows and identification of late-running passengers, to which appropriate announcements can be directed.

The system can offer a number of benefits. For instance, it can be used to control entry to secure areas; allow the precise automated-tracking, on occasions, of certain individuals on which there is prior intelligence indicating a possible security threat; help to evacuate the building in the event of an emergency, even with poor visibility due to smoke; provide rapid location and imaging of lost children and help to ensure that large aircraft, such as the new Airbus A380, are boarded efficiently by detecting and locating stray passengers. This latter issue is actually quite a serious problem as a significant proportion of flight delays are due to passengers who simply fail to arrive at their gates on time, giving the airline a stark choice between finding the passenger and offloading their baggage. It is difficult to obtain a precise estimate, but the Optag consortium have calculated that cost of flight delays due to late-running passengers amounts to some 100 million Euros per year in Europe alone, so considerable savings are possible with a system of this nature. A high degree of functionality can be built in to the system, dependent largely on the ingenuity of the user interface. An example of a typical basic interface is given in Figure 3. This illustrates a plan view window (left) with location information derived from the RFID tag system and two separate video windows of the same and adjacent areas of the airport. Each tagged person or item is indicated as a dot in the plan view window, with colour-coding to represent, for instance, green for no issues, red for a late-running passenger and blue for a tag that has perhaps been removed. The person represented by the triangle is being monitored in ‘auto-track’ mode, where the tag location data automatically switches the camera view to maintain the best possible imaging as the person moves through the building and could even highlight the region of the video image in which the person is likely to be. All tag data and a sub-set of the video data can be readily archived if required, something of great value to the security services.

Technical and social challenges

The prototype Optag camera, see Figure 4, comprises a cluster of eight 1600 x 1200 pixel CMOS sensors with real-time geometric correction and stitching to produce a 9600 x 1200 panoramic image. A portion of this image, or a lower-resolution panorama, is streamed to the central monitoring station by means of gigabit ethernet using the UDP protocol. Clearly, very fast real-time processing is required, which is achieved in the prototype system with an Odyssey XPRO framegrabber on a PCI-X bus, but would be implemented on custom silicon in a commercial system. The camera resolution equates to 24 pixels per degree, allowing recognition and identification of a human face to 6 m and detection to around 30 m. However, streaming of complete panoramic data from numerous such cameras over even a fast Ethernet infrastructure is not feasible and so a sub-set of the available video data is instead sent, such as a high-resolution sector from a limited number of cameras, whilst video archiving may be achieved locally to each camera. The challenge is to make as much video data available as possible without undue network problems such as latency and dropped packets. The advantage of using a standard LAN network such as this is that the infrastructure may be shared with other airport services.

The tag system is rather different to the conventional near-field technique used widely in transport ticketing in that it is required to operate at relatively long range (10-20m), perform location estimates and simultaneously identify large numbers (>1000) of tagged people or items in any given cell. To meet these challenges, the Optag team has designed a unique tag protocol that sends short bursts of data, at randomly-varying intervals to avoid the possibility of repetitive data clashes, with a mean update rate of twice per second. Each tag reader uses direction finding to establish the bearing of the tag and then subsequently two or more bearings are used to triangulate and establish the location. The prototype tag board operating at 5.8 GHz with FSK modulation is shown in Figure 5. The peak tag output power is 10 mW, but because of the pulsed nature of the transmission the mean output power is very much lower – around 20 ÌW, many orders of magnitude below the threshold of emissions that would constitute any conceivable health risk. Although the 5.8 GHz band has been chosen for the demonstration system, other bands may also be suitable. The prototype tag is a little larger than a credit card, but with miniaturisation, the tag electronics could be very compact and easily incorporated in a small card or unobtrusive wristband.

The tag readers, shown in Figure 6, are based on four antennas and receivers mounted at 90 degree spacings, which perform amplitude-comparison direction finding on each tag burst. This straightforward approach is expected to provide sufficient accuracy to allow location to a 1m tolerance within a 10m radius. However, the effects of reflections, signal blockage in crowded environments and other propagation artifacts are likely to be significant and will most likely diminish the achievable accuracy.

One issue of direct relevance to this system is the possibility of data loss due to clashes between the tag bursts. Some such data loss is inevitable, depending on the number of tags in use in a given area and the duration of the data bursts. This has been carefully modelled, with the results shown in Figure 7. For our system, with a 1 Mb/s tag data rate and a 150-bit data burst, the results in Figure 7(a) indicate that a tag population of 1000 has a 45 per cent chance of a given data packet being lost due to a clash and, from Figure 7(b), this equates to a mean update interval of 0.9 s – indicating that the position of all tags can be determined and updated on a second-by-second basis. Thus, the system can easily accommodate 1000 tags in any given cell, which is probably close to the limit of the number of people who can possibly be squeezed into a 10 m radius area! In the current design, the tag centre frequency needs to be stabilised, normally requiring the use of a relatively expensive synthesised frequency source, though new receiver and tag techniques are being explored that may relax these requirements and allow for a cheaper tag implementation. Also, these types of tag are generally not self-powered and require the use of a battery, with associated cost and maintenance implications.

As far as propagation effects are concerned, a variety of simulations and trials are being conducted to assess the likely location accuracy in a realistic indoor environment. For illustration, a basic model considers reflections from the walls of a rectangular room, in the absence of any obstacles. The result of such a simulation, assuming a worst-case wall reflectivity of 100%, is given in Figure 8. The bearing error around the room is found to vary from zero to as much as 25 degrees, but with a typical value of 6 degrees RMS – equivalent to 1m error at 10m range. This example uses a pessimistic value for wall reflectivity, but ignores the effect of the tag antenna pattern, polarisation and obstacles within the building. The result, however, should constitute a reasonable indication of the likely performance under realistic conditions. The location approach adopted in Optag is inherently vulnerable to propagation effects, but an entirely different technique, using wideband tag modulation to measure the time-of-flight (and hence distance) between the tag and readers, rather than the tag bearing, is much more resistant to these multipath effects and would allow greater accuracy to be obtained in typical indoor environments. Such an approach is more complex and would require more elaborate and expensive tag hardware, but is certainly worth considering for any future implementation of the system.

A potentially more challenging issue is public sensitivity to a scheme such as Optag, on grounds of personal privacy. Whilst it is true that it will be possible to track the position of all passengers at all times (in 1 second increments), there is certainly no agenda to make use of this information on a personal basis. It is more the statistical nature of the information that is of value – for instance, to make sure all passengers on flight no. BA6902 are at gate 28, 15 minutes before the scheduled departure time, or to record the mean and peak passenger flows through certain parts of the building to identify bottlenecks and feed into future terminal design. On occasions, of course, more personal, individual situations will arise – such as the need to locate a lost child – but in such an instance it is surely an honourable use of the technology? For the system to operate effectively, particularly in terms of security and safety (e.g. lost child location), it is important that each passenger carries or wears their own tag. This can be easily enforced from a technical viewpoint but, again, at greater risk of resistance from the civil liberties groups. Perhaps incentives can be offered to help mitigate any public resistance – such as discounts on presenting tags in certain retail outlets? Ultimately, though, it is a matter for the general public to decide whether the security and safety benefits, and the likely reduction in flight delays is worth the minor inconvenience and perhaps perceived stigma of ‘being tagged’.

The future

The Optag project is nearing completion and will report on the feasibility of the combined RFID tag and panoramic video monitoring approach in an airport environment, including a planned proof-of-concept trial in a Hungarian airport building. All indications are that the concept is sound, though any future adoption will require further development and commercialisation, in particular the network infrastructure and associated software to both operate the Optag system and interface with existing airport computer systems and databases.

The mode of deployment of the tag element of the system is controversial and somewhat critical to certain areas of operation. For instance, in a security context, it would be crucial to ensure that each person carries his/her own tag and does not lose or swap them. One way in which this can be achieved is to incorporate the tag in a wristband that sends an alert code should it be removed. The detection mechanism could be a small capacitively-coupled current across the sealed wrist-band, which is interrupted if it is either cut or removed. With suitable circuit miniaturisation, the wristband could be small and unobtrusive, perhaps made of thin card. However, public acceptance of the use of wristbands for this purpose may well be an issue, so exactly how the tags are deployed remains open at this stage. If the system were widely adopted it might be possible for it to operate through the entire end-to-end journey, from check-in at the departure airport though to arrival at the destination airport and even guiding passengers to their aircraft seats!

One of the areas that has huge potential for future development is the user interface, where a whole host of features could be incorporated including, for instance, an additional, simple interface at departure gates to alert staff to late-running passengers; an automated monitoring and announcement system to contact such late-running passengers as and when required; extensive archiving facilities to store tag and at least a subset of video data; seamless linking of tag ID, personal data and biometric data; and market research analysis of data, to aid the design of airport layouts for instance to optimise passenger flows or to feed into charging models for the various retail outlets. It is clear that, once such an infrastructure is in place, there is huge potential to make use of the capabilities in a variety of different manners, many of which have probably not yet been foreseen. The system may also find application in a range of other arenas, including hospitals, theme parks, exhibition halls and concert venues.