Welcome to TDI
virtualization
log management
outside in paradigm
console management
scada
corporate overview
 
it security alerts
white papers
news & resources
events
support services
contact us

Log Management




Jump To:
  1. Gathering the log data
  2. Logging when O/S or hypervisor is down
  3. Real time vs. after-the-fact
  4. Identifying the root cause and remediating the problem

Today's log management systems use one of two quite different architectures: "inside out" or "outside in."

From that distinction comes two entirely different solutions.



What's the basic difference between "inside out" and "outside in" systems for LOG Management? There is one key difference:
  • "Inside out" systems completely depend on the infrastructure they are logging or managing.
  • "Outside in" systems have no dependence on the systems they are managing.
From that difference come the following capabilities differences:

CapabilitiesTDILogLogicNetIQArcSightNet Forensics
Bios level data accessYESNONONONO
Real time loggingYESNONOYESNO
Logging when OS is downYESNONONONO
Root Cause AnalysisYESNONONONO
Remediation of problemYESNONONONO
Log hypervisorYESNONONONO
Persistent Compliance—HIPAA, NERC, SOXYESNONONONO


Key differences between "inside out" and "outside in" Gathering the log data

A primary difference is where one gathers data, and quite a difference it is.



"Inside out" logging systems come in at the network port (A) collecting only sources such as SYSLOG data or SNMP traps, after an event has occurred.

"Outside in" systems come in at the console port or craft port level (B). Thus, they see all levels of the hardware and software technology stack beginning at the BIOS.

Both systems collect network level data such as SYSLOG streams and SNMP traps. But "outside in" systems gather traditionally ignored data such as crash dumps, software and firmware exceptions, and hardware failures. Thus "outside in" systems can determine why something happened from data which is invisible to "inside out" systems. They can determine root cause and remediate the problem before it becomes critical.

"Outside in" logging thus gathers intelligence and understanding of:
  • Data sources that have disappeared.
  • Data sources that have become idle.
  • Data sources that are active determining what each source means. (IEMs)
  • Virtual data sources that moved to different hypervisors.
  • Data sources not available to higher level software programs such as operating systems or CIMOMs.
  • Data sources that have no on-board intelligence, such as HVAC, water detectors, telephone switches, and physical perimeter security censors.
  • Data sources that have no meaning themselves but become critical when time-correlated with other related data sources.
What does this mean to virtualization?

Where logging data is gathered is of critical importance in a virtualized world. Note the location of the hypervisor in the technology stack above. It is below what the "inside out" technologies can see. Thus they are blind to one of the most important components of the virtualized environment.

"Outside in" management not only sees the hypervisor problem, it can monitor the individual virtual machines and their guest operating systems at the logical serial port and determine in advance when it is likely to go down. More importantly, TDI logging systems use a patented IEM to determine why a VMware or XEN guest went down and recommends how to fix it.

Additionally, TDI technology monitors the hypervisor itself as well as the hardware on which it runs providing a 360 degree view of the entire virtualized world. Add the complexity of Virtual Center and VMotion, and TDI's Virtualized Logging becomes a critical part of the monitoring, management and remediation strategy of any corporation.

Logging when the O/S or hypervisor is down

Traditional "inside out" logging is completely useless when the O/S or hypervisor goes down. When the O/S goes down, the logging system gets no data.

But the clever inside criminal is taking all the payroll data from the system that is either off the network or is temporarily down. When the machine comes back up, there is no record of the intrusion and the traditional "inside out" log management system tells the user there is no problem.

This fails to meet HIPAA, NERC and SOX compliance tests. If one is managing SCADA devices, this can cause entire energy grid to be compromised.

"Outside-in" logging systems like TDI maintain a persistent connection to the machine or virtual O/S at all times. Any intrusion is reported immediately. Every event is logged and time stamped. TDI will even shut out users who the corporation does not want to access the system---even when it is down.

That is why so many of the country's energy firms and governmental security agencies use TDI for their most secure tasks.

Because TDI is "outside in", the TDI user can access the virtual guest and restart it or apply any fix recommended by the vendor—even if the hypervisor or OS is down. The TDI user can follow a guest OS as it moves from one physical device to another.

Real time vs. after-the-fact

TDI applies both "persistent monitoring" and "real time" analysis on log files for all managed devices in the infrastructure. Because TDI does not require any agents, it has no network traffic footprint, nor does it add to the already overwhelming version management and deployment strategies of IT staffs.

TDI delivers persistent monitoring, rather than polling, and determines a problem immediately upon it showing an "out of profile" characteristic.

Identifying the root cause and remediating the problem

TDI has always believed logging without remediation is like a house without a roof.

Why would anyone buy a logging system that did not enable automatic remediation?

Well, "inside out" logging systems cannot remediate problems because they cannot see the root cause because they are too high in the technology stack. (Figure 1)

TDI's "outside in" logging provides an indication of problems/correlation across all managed devices.

TDI's customers can remediate their problems because TDI enables them to see the root cause, time stamp the problem with a natural clock and use IEMs to tell them what to do to each impacted device.

TDI believes in the "self healing infrastructure" and enables its customers to automatically have the system apply changes to the compromised infrastructure components with little or no human intervention.
  1. TDI Logging provides an automatic failover of log files from the primary location (SAN, NAS, or JBOD) to a recovery location should the primary logging location become unavailable. This feature ensures that TDI Logging never loses any important data and alerts the management staff that a failure in the critical logging infrastructure has occurred.

  2. TDI Logging provides many hot, warm and cold failover capabilities to meet every customer's need in its disaster recovery and disaster tolerance methodologies.

  3. TDI Logging provides long-distance, geographically dispersed data mirroring capabilities for customers who need secure, reliable disaster recovery solutions.
This is why so many of the world's most mission critical infrastructures run on TDI. For them, infrastructure failure is not an option.