Open Observability with the Pixie Plugin System

Michelle Nguyen

May 12, 2022 • 2 minutes read

Principal Engineer @ New Relic, Founding Engineer @ Pixie Labs

At Pixie, we believe that the future of observability lies in open source.

With the rise of open standards, such as OpenTelemetry, it is easier than ever to go from a fresh cluster to full observability. It's as simple as deploying standard-friendly agents and sending data to the tool(s) of your choice. Want to analyze metrics in Prometheus and explore your traces in Jaeger? That’s supported. Want to track those traces in Zipkin instead? Easy. Gone are the days of restrictive, proprietary agents designed to work with one specific tool. The interoperability of open source tooling makes it painless to adopt and switch out products that best fit a developer’s needs.

With these beliefs in mind, we built the Pixie Plugin System. At its core, Pixie focuses on providing a no-instrumentation, real-time debugging platform. However, effectively monitoring clusters and applications requires more. Developers need long-term data retention to track historical trends and perform comparisons overtime. They want alerts to notify them when something has gone wrong.

Rather than building out all of these capabilities into Pixie, we recognize there are already other excellent tools which offer these features. We took inspiration from Grafana’s Plugins which allows users to enhance their Grafana experience by integrating with other tools as datasources and panels. The Pixie Plugin System is designed to embrace the interoperability of open source software and leverage the strengths of other tools in the ecosystem.

The initial version of the Pixie Plugin System aims to address Pixie’s data storage limitations. Pixie stores its collected data in-cluster for performance and security benefits. Retention time depends on the level of traffic in your cluster, but will generally be on the order of hours.

The Pixie Plugin System enables long-term retention for Pixie data by providing export to other tools. By relying on OpenTelemetry as Pixie’s export format, Pixie’s metrics and traces can be ingested by any OpenTelemetry-supported product. You can enable the plugin for your favorite tool, and Pixie will immediately start egressing data. Each plugin provider has pre-configured a set of default scripts to export data best complemented by the tool. However, you can also write custom export scripts to send any Pixie data you want.

With Pixie’s Plugin System, we envision a future where Pixie’s telemetry data can be consumed anywhere. However, integrating across many tools has its drawbacks. Navigating and context-switching from tool to tool is cumbersome and inefficient. The future of Pixie’s Plugin System aspires to unite these tools in a central location. All from within the Pixie UI, developers will be able to configure alerts across products, query long-term data from different sources using Pixie’s scriptable views, and more. Pixie’s goal has always been to make gaining visibility into developers’ clusters and applications as simple as possible. By leveraging the benefits of open source and open standards, we hope to make observability more powerful, easy, and accessible for all developers.

Here are some materials to get started using the Pixie Plugins:

• Read the Export OpenTelemetry Data tutorial
• Check out the Plugin System reference docs
• See the list of available plugins
• Learn how to contribute a plugin

---

Related posts

I shouldn't be seeing this: anonymize sensitive data while debugging using NLP

It's 10 pm and you're on-call. A few minutes ago, you received a slack message…

Benjamin Kilimnik
Nov 04, 2022 • 8 minutes read

Pixie Team BlogsPIISecurity

Come meet us at Kubecon NA 2022

Pixie contributors will be attending this year’s KubeCon + CloudNativeCon North…

Zain Asgar, Michelle Nguyen
Oct 18, 2022 • 2 minutes read

Pixie Team BlogsAnnouncementsKubernetes

How we automated Java profiling: in production, without re-deploy

The Java ecosystem offers many options for profiling Java applications, but what…

Pete Stevenson
Sep 22, 2022 • 6 minutes read

Pixie Team BlogseBPF