Pion^TM Components

Pion includes an embedded web server that allows you to configure the system from most popular web browsers through a lightweight, AJAX (Web 2.0) interface (no downloads or plugins required). The web server was built from the ground up to be secure to enable Pion’s use in the most security conscious organizations. In addition to the easy to use user interface the web server also provides many different RESTful web services which can be leveraged for automated configuration and to dynamically create event stream connections over which data can be sent into or out of any Reactor. Like all of Pion the web server is built to be flexible so it provides easy to use API’s so that you can also create your own web services using plugins.

The processing engine is the framework that manages Reactors — Pion's configurable plugins that process events and create new output streams. The processing engine provides the following services:

• Loads and manages Reactors
• Describes how they are connected
• Manages the flow of events from reactor to reactor
• Distributes processing load across multiple processing resources
• Enables the system to be easily be reconfigured on-the-fly without interrupting your data processing workflow

The combination of Pion’s codec and vocabulary modules are at the core of Pion’s ability to easily read, process and output any type of information. Both the vocabulary and codecs are easy to define and can be shared with other Pion instances through open XML files making it easy to add to the out of the box codecs and vocabularies Pion comes with.

Pion’s vocabulary is the “what” of data processing The vocabulary describes all of the fields of normalized information Pion knows about and can process. For instance one of the out of the box vocabularies for web clickstreams includes everything that can be known about a web user session such as page content, URI stem, client, authorization ID, cookies, etc.

The codec describes the “how” elements of data processing and pairs tightly with the vocabulary to enable interoperability and integration. Basically the codec describes how information is formatted and communicated. For instance the clickstream data could have come from any number of different log file formats, a live sniffer stream, page tags or a database. The codec acts as a translator for Pion so that everything can be represented in the normalized vocabulary regardless of source or destination.

Pion’s Log Input reactor is able to read in any structured flat file as needed to provide you with easy access to hard to integrate information.

Pion’s Sniffer reactor listens to either real-time network communications or captured traffic files (.cap and .pcap). This enables you to see real time conversations between clients and servers to discern critical elements of the interchange. For instance, the built in HTTP(S) protocol translations enables you to see exactly what a web user is doing, what data was sent and capture their entire session in real time. Pion packet sniffing supports:

• Open source plugin interface for extending additional network protocols
• SSL decryption of traffic using private RSA key files
• Handles gigabit network speeds on a single 1U server
• Data capture and extraction
• Sniffer clustering handling any traffic volume

The Transform reactor enables you to take data input and transform it based on any number of rules or regular expressions. This is an important step because different systems represent data differently for instance a simple salutation may be represented as a “Mr.”, “Mrs.”, or “Dr.” on your web page but your account database refers to these as a salutation “1”, “2” and “3”. Pion uses in-memory mapping tables to make even extremely varied or complex mappings fast and simple to implement for non-technical people. Pion is even able to handle situations where you may not know the correct mapping ahead of time via its full regular expression support. With regular expressions users can do complex content extractions and transformations. For example, extracting out an account number and then replacing the first 12 digits with “x” characters to improve security can be done based on a regular expression.

The Filter reactor allows you to include or exclude specific parts of an input stream based on rules that you define. For instance you might want to filter all user sessions on your web site into various group based on how much they spent so that you can separate your best customers into separate categories or flag those who didn’t buy anything for a follow up call or email.

The Script reactor allows you to retrieve, store or process events using an external script or program.

The Fission Reactor enables you to split complex messages into easy to use information.  For example the Fission Reactor easily splits complex ATOM and RSS feeds into their sub components to open up brand new insights and opportunities for web analytics technologies.

The Aggregation reactor allows you to calculate various statistics in real-time without having to wait for a database or OLAP solution for analysis. This enables you to create powerful real-time dashboards via our reporting interface on things such as the number of users at any moment or how much those users spent today. You can also use the Aggregation reactor to consolidate and summarize data on the fly so that your databases or applications don’t have to deal with masses of raw data.

The Session Filter reactor allows you to scan all traffic for each unique visitor session and either discard or forward the session's traffic based upon configurable triggers and patterns.

The Clickstream reactor takes raw web user data from web logs or the Sniffer reactor and is able to transform the jumble of HTTP hits into a coherent set of user sessions. This allows you to see exactly which pages “Chris” used from the ones that “Mike” visited. The Clickstream reactor also creates higher level session data out of the hits such as session duration, average dwell time, session id, session group, number of pages, order of pages, referrer, abandonment point and more.

The SQL Reactor enables you to query any relational database for historical information based on real time information coming into Pion.  For example if you are monitoring web users shopping you could query an existing customer database to see how much they have spent in the last year or if there are any items they buy consistently to offer recommendations.

The Log Output reactor allows you to send any Pion data to a structured log file for later use. You can use one of the many preconfigured log formats included with Pion out of the box or easily add your own with a few clicks.

Pion includes SQLite as part of the Community Edition and gives you powerful open source relational database capabilities at no cost. You can output any data in any format of your choice for later use.

The Google Analytics reactor enables Pion to feed Google Analytics information about user behavior via their web services interface. This enables you to skip the effort and expense associated with page tagging via our packet sniffing technology while still have access to the same great Google Analytics reports. Better still you’ll gather more information about more customers because Pion cannot be blocked by spyware blockers or harmed by browsers, such as those in mobile phones, that don’t support JavaScript.

The Omniture reactor enables Pion to feed Omniture information about user behavior and historical information via their web services interface in realtime. This enables you to skip the effort and expense associated with page tagging via our packet sniffing technology while still have access to the same great Omniture reports. Because Pion cannot be blocked by spyware blockers or harmed by browsers, such as those in mobile phones, that don’t support JavaScript you’ll be able to see the customers (average 20-28%) that your are missing today if you rely solely on page tagging.

Pion converts information directly into WebTrends via their SDC log format.  This enables you to skip the effort and expense associated with page tagging via our packet sniffing technology while still have access to the same great WebTrends reports. Additionally Pion can add offline information to add context to the user experience by publishing custom parameters to WebTrends for richer reporting then would otherwise be available via page tags.

Pion enables optimized output to many enterprise databases via their native protocol. This increases speed and efficiency at every level and is especially powerful for loading data into existing data bases or data warehouses. Today Pion supports the following enterprise databases:

• Oracle - via native OCI
• Microsoft SQL Server - via native DBlib or OLE-DB
• IBM DB2 - via native DB2 CLI
• Sybase - via native ASE, ASA or Open Client
• Informix - via native Informix CLI
• MySQL Enterprise - via native MySQL CAPI
• PostgresSQL - via native libpq
• Centura - via native CAPI
• Any ODBC compliant database

The Pion platform and many of its reactors are open source and so there is a vibrant development community around Pion that is available to answer many of the questions you might have. Our development and services executives are also active in the community and you can consider it the place to go to with any challenges, ideas or problems you might have.

For those of you that need dedicated support available at a moment’s notice we provide premium support options available via phone and email when you need them to keep your production systems up and running.