Pipeline Data Processing

Shape telemetry before teams query and retain it

Guance Pipeline parses, extracts, converts, filters, masks, and normalises telemetry before analysis. Teams can turn unstructured messages into consistent fields for search, alerting, correlation, and data governance across supported logs, metrics, traces, network data, and objects.

Shape telemetry before teams query and retain it

What Pipeline Data Processing helps you solve

Turn raw telemetry into fields teams can reliably query and correlate

When incoming records have inconsistent formats or expose sensitive values, Pipeline applies parsing and governance before those records become a dependency for searches, alerts, dashboards, and investigations.

Raw messages are hard to search. Extract the fields investigators need
Use Pipeline processors to parse, extract, modify, filter, and aggregate collected data. Turn values such as service, host, status, duration, request path, and trace ID into stable fields or tags that searches, monitors, and dashboards can reuse.
View documentation
Raw messages are hard to search. Extract the fields investigators need
A shared schema makes cross-signal investigation possible
A shared schema makes cross-signal investigation possible
Normalise field names and types before teams depend on them. Consistent service, host, environment, status, and trace context lets an investigator move between logs, APM traces, infrastructure metrics, network evidence, and events without reconstructing identity at every step.
View documentation
Sensitive values should be handled before they spread downstream
Apply replacement, deletion, or other configured transformations to selected phone numbers, identity values, tokens, accounts, IP addresses, or business fields. Earlier handling reduces the number of downstream searches, exports, and shared views that can expose the original value.
View documentation
Sensitive values should be handled before they spread downstream

Frequently asked questions

What does Guance Pipeline Data Processing do?

It parses and transforms supported telemetry before analysis, including field extraction, type conversion, filtering, masking, and normalization. The resulting fields are easier to search, aggregate, alert on, and correlate.

Which data types can Pipeline process?

Guance documents Pipeline processing for DataKit-collected categories including Logging, Metric, Tracing, Network, and Object data. Exact processors and safe transformations depend on the source structure and Pipeline configuration.

Why handle sensitive data before storage or sharing?

Early masking or removal limits how often the original value appears in queries, dashboards, exports, archives, and collaboration workflows. Teams should still combine Pipeline with appropriate access and retention policies.

Related reading

Build a queryable log investigation workflow

Build a queryable log investigation workflow

Structure log fields and keep service, host, status, duration, and trace context available for analysis.

Product features
Plan an enterprise log management architecture

Plan an enterprise log management architecture

Connect collection, processing, storage, search, access, and investigation as one governed data path.

Best practices
Use Pipeline to improve telemetry quality

Use Pipeline to improve telemetry quality

Parse, normalize, filter, and protect data before downstream teams depend on it.

Product features
Compare managed log analytics with an ELK workflow

Compare managed log analytics with an ELK workflow

Review the operational questions involved in collection, storage, querying, visualisation, and ownership.

Industry insight

Continue from clean telemetry to analysis and governance

Connect Pipeline to logs, sensitive-data controls, dashboards, documentation, and pricing.

Get started > View documentation > View pricing >