Difference between revisions of "Cartographer"

From Krupczak.org

Revision as of 19:25, 30 March 2007

1 Cartographer
2 Overview
3 Supported Platforms
4 Managers and Agents
5 Management Protocol
6 Peer-to-Peer
7 Screen Snapshots
8 Example Statistics

Cartographer

Why the name Cartographer? We chose this name to reflect the fact that our software builds and analyzes maps of distributed systems. It does not build maps in the traditional sense -- topography is not collected nor stored. Instead, the relationships between systems and applications are continuously discovered and identified. These relationships are then measured and analyzed in order to assess the health and availability of the distributed system as a whole.

Overview

Cartographer implements a novel approach to managing distributed systems by automatically discovering and tracking the relationships between its component systems and applications. Cartographer does so via specially designed agents -- residing on clients, servers and (potentially) network devices -- that detects, identifies, and tracks the inter and intra-system dependencies or relationships. Dependencies include network level services like DNS, DHCP, and SMTP as well as higher-level application abstractions like filesystems and databases.

These relationships are modeled as a dependency graph borrowed from the Graph Theory branch of mathematics. In our model, systems and applications are represented as vertices and dependencies are represented as edges. More specifically, we use directed graphs to indicate dependencies are between clients and servers.

Once dependencies are discovered, Cartographer agents automatically organize systems and applications into peer-to-peer overlays. Then, peers exchange management information amongst themselves to detect and correct service problems.

Example Dependency Graph 1 represents a small slice of a distributed system containing systems S1, S2, and S3. These systems (and the applications residing on each) possess several dependencies amongst themselves. Systems S3 and S2 rely on system S1 for DNS service while S3 mounts the /www filesystem from S2. Finally, system S3 utilizes a database also residing on S3.

Supported Platforms

Cartographer is supported on the following platforms:

Solaris 9+ on Sparc
Solaris 9+ on x86
Linux systems containing 2.4+ kernels on x86
Windows 2000/XP/2003

Managers and Agents

Cartographer contains agents and managers just like traditional management architectures (e.g. SNMP and the Internet Management Framework). However, in the Cartographer architectures, managers have been de-emphasized and exist merely to inject management policies and updates and exist to provide views into the system. Agents, on the other hand, are intelligent, self-organizing, and self-distributing, and act in a dual-role. When agents are collecting dependency data and measuring performance, they are acting in their traditional role. When agents communicate with each other, run distributed decision-making algorithms, and self-propagate, they are acting more as managers do in traditional frameworks.

Management Protocol

Cartographer's management protocol is a custom-designed, XML-based, management protocol transported via SSL over TCP. The protocol adapts and extends the Internet Management Framework's structure of management information(SMI). Further, the protocol utilizes XML for both data modeling and transport. The choice of XML provides more future-proofing than the original Basic Encoding Rules (BER) that SNMP utilizes. Further, choosing XML allows us to take advantage of all the XML tools and code libraries in existence today. For now, the protocol is called the XML management protocol or XMP for lack of a better name. This name should not be confused with the IETF XMPP Extensible Messaging and Presence Protocol.

Peer-to-Peer

Cartographer agents self-organize into P2P overlay networks in order to exchange management information, software updates, and events. Systems, in Cartographer's model, can be both clients and servers just as in real life. A system is considered a server if it provides some service to a client. A system is considered a client if it utilizes a service from some server. Systems are considered peers if they both utilize the same server for some particular service.

In the future, these overlay networks will be utilized to run distributed, decision-making algorithms.

Screen Snapshots

Cartographer main window

Cartographer's main window includes facilities for loading and unloading topographies, launching views, and tools for mainipulating the database of systems and dependencies.