[LINK] Airport security
stephen at melbpc.org.au
stephen at melbpc.org.au
Thu Jan 21 04:00:15 AEDT 2010
Here's a glimpse of airport security in the future:
IBM Patenting Airport Security Profiling Technology
by By Alexander Wolfe InformationWeek January 19, 2010 09:06 AM
A dozen little-known IBM patent applications lay out a sophisticated
computer-analysis-based approach to airport security.
The technology has the potential to apply profiling of passengers, based
on attributes such as age and type of clothing worn.
One of the patents IBM is seeking even appears to go Israeli-style
security one better, by using analysis of furtive glances in their patent
application entitled "Detecting Behavioral Deviations By Measuring Eye
The objective of the technology in the passel of patent applications is
to alert officials to potential terminal and tarmac threats using a
network of video, motion, chemical, and biometric sensors arrayed
throughout the airport.
The sensors feed into a grid of networked computers, which provide high-
powered processing to get results to officials in so-called real time,
yet the systems are compact enough to be located on-site.
The "secret sauce" in the set up is a software "inference engine," which
crunches the data fed in by the multitude of sensors, separating the high-
risk wheat from the false-alarm chaff.
That engine uses heuristics and rules developed by the three co-inventors
behind the patent applications--Robert Angell, Robert Friedlander and
"These patents are built on the inference engine, which has the ability
to calculate very large data sets in real time," Angell told me last
He called me because he was surprised I had uncovered one of the patents,
which I wrote about recently in my blog post, " Obama Security Push
Spurring Scanner Patents (IBM's Seeking One)." That post focused on the
patent application "Risk assessment in a pre/post security area within an
Angell told me he believed the patents were under seal.
That piqued my interest, because it indicated that this technology is
probably more important -- in the sense of being proprietary and cutting
edge -- than I had initially realized. As well, I knew of only the one
patent and hadn't realized that, according to Angell, there were eight.
(Since our conversation, I've uncovered 12 unique applications; the
discrepancy might be due to the presence of duplicates--patent lawyers
often revise and resubmit applications--or spin-offs.)
It turns out that, in fact, the patent applications are not under seal;
that's something I don't think you can do, because the patent process is
by definition open.
Companies which want to shield proprietary technology go the trade-secret
route, which means you keep your cutting-edge technology out of the
public eye and hope no one will reverse-engineer it.
However, IBM has used a perfectly legal subterfuge to make the patent
applications difficult to track down. It didn't put its company name on
any of the applications, listing only the inventor names and that of its
I have now tracked down all the applications, and will go into the
technology details, below.
Angell also said that he's no longer with IBM. "I was laid off last year
along with thousands of other people," he told me. Angell is currently
teaching a computer science course at a community college in Salt Lake
City, Utah, where he lives. I was flabbergasted, wondering how Big Blue
could let go a guy like this, who obviously has heavy duty data-analysis
chops and is behind such seemingly important technology.
Angell called me, he said, because he's concerned that the technology be
applied effectively. "If it's done right, we could do passive profiling
[and] passive detection and do it without a whole lot of fanfare," he
This profiling of potentially dangerous passengers, as outlined in the
applications, appears in many ways to be more neutral than the profiling
currently the subject of widespread public debate, because it's software-
based and runs off of pre-programmed rules which, in general, are
intended to identify suspicious behavior.
(On the other hand, this wouldn't necessarily always apply, since markers
such as a person's apparent age are listed in the patent applications as
potential data points.)
Let's look at the patents in more detail. The profiling, off of sensor
input, is described in patent application number 20090204695, filed last
September. It's entitled "Unique Cohort Discovery From Multimodal Sensory
This patent application describes the use of a large number of sensors of
all types -- chemical, biometric, etc -- around the airport perimeter, so
data can be fed into a computer for analysis to detect threats.
Here's the relevant wording from the patent application:
"[Data processing parses the data to form attributes.] Attributes may
include an individual's age, make and/or model of a vehicle, color of a
hat, breed of a dog, sound of an engine, a medical diagnosis, a date of
birth, a color, item of clothing, walking, talking, running, a type of
food eaten, an identification of an item purchased.
An attribute that is an event may include eating, smoking, walking,
jogging, walking a dog, carrying bags, carrying a baby, riding a bicycle,
an engine running, a baby crying, or any other event.
Sensory data processing categorizes the events. . . For example, a type
of event may include a pace of walking, a companion of the cohort, a time
of day a cohort eats a meal, a brand of soda purchased by the cohort, a
pet purchased by the cohort, a type of medication taken by the cohort, or
any other event."
In terms of the sensors themselves, the system uses lots of diverse data-
gatherers. From the patent application:
"Multimodal sensors comprises at least one of a set of global positioning
satellite receivers, a set of infrared sensors, a set of microphones, a
set of motion detectors, a set of chemical sensors, a set of biometric
sensors, a set of pressure sensors, a set of temperature sensors, a set
of metal detectors, a set of radar detectors, a set of photosensors, a
set of seismographs, and a set of anemometers."
Angell told me that the system can even use olfactory sensors, which
means they'll smell the environment. The patent application also
variously mentions license plate recognition technology, face recognition
software, and retina scanners. Data captured from video streams from
airport cameras is also analyzed.
How does one computer process all this data fast enough to deliver a
threat assessment quickly enough to airport security officials? Remember,
the idea is to do the analysis in real time, as passengers are streaming
through the terminal to board their flights. For a single box, this would
be a processing challenge. However, the inventors envision using a small
grid of computers connected over a network. This'd deliver ample power to
do the real-time data crunching.
"Computers aren't fast enough to do real-time modeling unless the
paradigm shifts," Angell told me. "That's why this inference engine is a
pretty big deal."
That shift is embedded in how inference engine is formulated. It uses
rule sets, designed by Angell, Friedlander, and Kraemer, which enable it
to fairly efficiently query 5 million or 10 million data cohorts, in a
very short period of time.
Analyzing Eye Movements
There is another patent application in the group which takes the analysis
of potential passenger threats to a whole 'nother level. It's
entitled "Detecting Behavioral Deviations By Measuring Eye Movements."
(Patent application number 2009232357, filed September 2009.)
(Friedlander is not involved in this patent; it's Angell and Kraemer
only.) From the filing:
"The ocular metadata [patterns of eye movement] is analyzed. . .In
response to the patterns of ocular movements indicating behavioral
deviations in the member of the cohort group, the member of the cohort
group is identified as a person of interest."
Specifically, eye movement characteristics which are monitored and
analyzed include: change in pupil size (dilation); direction of gaze;
visual line of gaze (where someone is looking); and rate of blinking; and
Profiling is specifically addressed in this patent application, as
"The profiled past comprises data that may be used, in whole or in part,
for identifying the person, determining whether to monitor the person,
and/or determining whether the person is a person of interest. Global
profile data may be retrieved from a file, database, data warehouse, or
any other data storage device. Multiple storage devices and software may
also be used to store identification data 506. Some or all of the data
may be retrieved from the point of contact device, as well. The profiled
past may comprise an imposed profile, global profile, individual profile,
and demographic profile. The profiles may be combined or layered to
define the customer for specific promotions and marketing offers."
However, analysis of eye movements aren't the final word in indentifying
passengers with potential ill intent. Patent application
targets "Detecting Behavioral Deviations by Measuring Respiratory
Patterns in Cohort Groups."
What's your take? Let me know, by leaving a comment below or e-mailing me
directly at alex at alexwolfe.net. Follow me on Twitter: (@awolfe58)
Alex Wolfe is editor-in-chief of InformationWeek.com
More information about the Link