HOW STREAM CENTRAL WORKS

 

StreamCentral brings together the ability to model real-time business solutions by introducing an innovative solution designer that simplifies the modeling process, an information warehouse manager that automatically rapidly builds and deploys the model on industry leading data management platforms and an enterprise ready server technology that collects, processes, correlates and publishes data in real-time. Together, these three components drastically reduce time to market, risk and cost associated with building real-time Business Intelligence solutions.

StreamCentral introduces Workbench; a unique easy to use application to design Business Intelligence and Big Data solutions. The Workbench is a no coding required application that makes it easy to create a blueprint for the Business Intelligence or Big Data solution. Some of the key elements of the solution life cycle that can be defined using the Workbench are discussed below.

Data Sources

Once data sources are defined using the Workbench, StreamCentral automatically pulls in the various attributes available within the data source. Here, the Workbench allows creation of derived attributes which gets executed in real-time as data is sourced. Finally, StreamCentral has particular smarts built in regarding time and location. Specific time and location data types can be assigned to data attributes that allow StreamCentral to automatically standardize incoming data across time and location. This is done automatically without having to write any code to do this complex task

Data Source Attributes

Once data sources are defined using the Workbench, StreamCentral automatically pulls in the various attributes available within the data source. Here, the Workbench allows creation of derived attributes which gets executed in real-time as data is sourced. Finally, StreamCentral has particular smarts built in regarding time and location. Specific time and location data types can be assigned to data attributes that allow StreamCentral to automatically standardize incoming data across time and location. This is done automatically without having to write any code to do this complex task

Data Transformation Rules

The Workbench includes an extensive list of easy to use text and numerical data transformations that are applied to data in flight. Examples of transformations include Contains, Substring, Replace, Startswith, Remove, LTRIM, Ignore Special Characters and more.

 

 

 

Entities

Factual data from a data source does not automatically have all context needed for analysis. StreamCentral makes it easy to connect in-flight data to things that are core to your business like customers, products, patients, stores, partners, suppliers, cell towers, water pipes and more. Once an entity is defined, initial loads can be imported to allow StreamCentral to have entity history along with continuous update rules to keep entity data always current. In addition, the Workbench allows definition of how data from a data source is connected to a specific entities. Example in Utilities streaming data from a sensor is connected to a pipe, in Retail sales data is connected to a specific customer, product and store, in Telco, streaming video or voice quality data is connected to a customer and a device.

 

Conditions

Using the Workbench you can monitor for specific conditions within a data source. The conditions rule builder allows complex rules to be defined to define a condition. Conditions that are defined across multiple data sources can be grouped together into what is called a Conditions Set. This is useful to later evaluate incoming data differently for different purposes.

Condition Evaluation

One of the powerful capabilities of StreamCentral is that it can apply different sets of conditions to different types of entities. For example, one set of conditions can be evaluated to measure incoming data for different customers based on customer size or one kind of product or age group of a patient. The options are limitless and provide a powerful mechanism to allow the solution to be representative of real world scenarios.

Real-Time Data Correlation

For real-time analytics and real-time event detection it is necessary to be able to pick exact records in multiple data sources that are related to the same thing. The Workbench correlation modeler is an easy to use tool that helps define how to make sense of the various streams of data using fuzzy matching and probability techniques

Event Detection Rules

An Event happens when patterns that consist of conditions with specific ranges from different data streams and other conditions are detected across the data as it streams in. The event modeler in the Workbench is a powerful tool that allows complex rules to be defined for detection of events

Alerts

Define roles and configure who should be alerted when specific conditions or events are detected

Real-Time Event Data Marts

Aggregate real-time events and bring together data across data sources to analyze conditions that existed when events are detected. The Workbench allows data mart structures to be defined by picking and choosing the right set of attributes from data sources, KPIs, events, attributes from entities, and dimensions

Shared Dimensions

Multiple data sources could have the same attribute. These attributes can be used tor slicing and dicing the data. The Workbench allows creation of shared dimensions while standardizing data across data sources to a common business value

360 Degree Data Marts

Easily bring together and aggregate data across data sources to get 360 degree insight. 360 degree data marts allow associations to be analyzed in data to determine patterns that impact business performance. The Workbench allows data mart structures to be defined by choosing the right set of attributes from data sources, KPIs, attributes from entities, and dimensions in the Workbench

Information Warehouse Manager designs, creates and manages updates of the underlying database schema for the information warehouse and data marts in real-time. StreamCentral supports HP Vertica or Microsoft SQL Server. Schema design and management done by StreamCentral is an extremely valuable step in the process. The deployment takes into account complex industry standard design patterns while injecting powerful techniques to design and implement the most effective solution. Some of the key capabilities of the information warehouse manager are discussed below.

Model Meta Data

As the solution model is defined and created in the Workbench, the model deployment is working to create a meta data database using Microsoft SQL Server as the data repository. The meta data begins to capture all the definitions, relationships and rules defined in the Workbench

Create Database Structure

The main function of model deployment is to create the appropriate database schemas in the underlying data management platforms. There are two distinct concepts to understand about the database structure. First, StreamCentral is constantly evaluating the solution model being defined to build the most effective database design leveraging industry best practice design patterns for data warehouse development. Second, StreamCentral writes the appropriate code based on the underlying data management platform to create the database structures. At this time StreamCentral’s data management support is for HP Vertica and Microsoft SQL Server. At Virtus, we believe this step alone saves significant amount of time and specialist skill requirements for any Business Intelligence and Big Data projects.

Add Context

Generating insights from data requires context to be added to the data. This context is a continuous thread that connects all types of data throughout the Big Data Solution life cycle. Four typical examples of context include who (entities like customer, store, patient, product etc), when (time), where (location) and what (streaming and static data correlation). Adding this context exponentially increases the impact and value of the investments in data. StreamCentral automatically builds and maintains time and location dimensions. Entities can be created and defined in StreamCentral. All data (static or streaming) in StreamCentral is continuously and automatically connected to time, location and defined entities. Resultant real-time events and analytical data marts automatically inherit this context without need for any programming or development work.

StreamCentral Server is a set of services that can be installed and run on any number of commodity hardware making real-time model execution highly scalable and available. The server collects real-time or batch data, applies transformations, adds context to data, correlates multiple streams, detects events, publishes data to underlying database, updates marts in real-time and makes insight available for export.

Collector Service

The collector service captures data (static or streaming), validates data and prepares it for processing. With a built in API to collect stream data, StreamCentral enables you to acquire data from a diverse set of sources like sensors, logs, social feeds, business applications and databases. The collector service also executes continuous data pull rules specified against various static data sources.

Data Processing Service

The data processing service executes transformation rules on incoming data (static or streaming) as defined in the model. Incoming data is also monitored for various conditions and Key Performance Indicators as defined in the model. Alerts are executed based on condition monitoring rules. The processing service constantly adds context to incoming data by looking for time, location, entity and shared dimensions present in incoming data.

Data Correlation Service

The correlation service executes any data correlation rules defined in the model on incoming data. Correlated data is then used to detect events by evaluating event rules defined in the model. Correlated data is also used to update real-time event data marts via the data publishing service.

Data Publishing Service

The data publishing service is responsible for inserting processed data into the underlying data management platform. StreamCentral exploits the capabilities of different data management platforms to deal with inserts of bulk static data and trickle high volume streaming data. Customers can choose either HP Vertica or Microsoft SQL Server as their underlying data management platforms for StreamCentral. With StreamCentral, your team does not need to be an expert in either data management platforms. In addition, enterprise applications can also consume event information through StreamCentral Publishing API.

Data Security

The data security service evaluates data security rules defined in the model to ensure only roles and users that have access to the data will be able to view the data. Reporting applications (like Microsoft SQL Server Reporting Services) or discovery and analysis applications (like Tableau Software or Microsoft Powerview) can use the data security model of StreamCentral to ensure only people that have access can see the data.

Free WordPress Themes, Free Android Games